Hostname: page-component-8448b6f56d-jr42d Total loading time: 0 Render date: 2024-04-23T21:57:40.445Z Has data issue: false hasContentIssue false

The why, what, when, who and how of assessing CBT competence to support lifelong learning

Published online by Cambridge University Press:  02 December 2022

Kate Muse*
Affiliation:
Psychological Sciences, University of Gloucestershire, UK
Helen Kennerley
Affiliation:
Oxford Cognitive Therapy Centre, Oxford, UK
Freda McManus
Affiliation:
British Association for Behavioural and Cognitive Psychotherapies (BABCP), UK
*
*Corresponding author. Email: kmuse@glos.ac.uk
Rights & Permissions [Opens in a new window]

Abstract

Assessment of cognitive behaviour therapy (CBT) competence is a critical component in ensuring optimal clinical care, supporting therapists’ skill acquisition, and facilitating continuing professional development. This article provides a framework to support trainers, assessors, supervisors and therapists when making decisions about selecting and implementing effective strategies for assessing CBT competence. The framework draws on the existing evidence base to address five central questions: Why assess CBT competence?; What is CBT competence?; When should CBT competence be assessed?; Who is best placed to assess CBT competence?; and How should CBT competence be assessed? Various methods of assessing CBT competence are explored and the potential benefits and challenges are outlined. Recommendations are made about which approach to use across different contexts and how to use these effectively to facilitate the acquisition, enhancement and evaluation of CBT knowledge and skills.

Key learning aims

After reading this article you will be able to:

  1. (1) Identify key issues about why, what, when, who and how to assess CBT competence and use this framework to guide decisions about the best strategy to use.

  2. (2) Be aware of the range of methods for assessing CBT competence and consider the main benefits and potential challenges of these.

  3. (3) Consider the most effective ways to implement CBT competence assessment strategies as a tool for evaluation and learning.

Type
Invited Paper
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of British Association for Behavioural and Cognitive Psychotherapies

Introduction

CBT competence assessments involve making judgements about the quality or skilfulness of therapists’ performance and abilities in delivering cognitive behaviour therapy (CBT). This article provides a framework designed to support trainers, assessors, supervisors and therapists when making decisions about selecting and implementing effective strategies for assessing CBT competence. It focuses on how competence assessments can be used to facilitate the acquisition, development and evaluation of CBT knowledge and skills, both within training settings and in supporting continued professional development within clinical practice. The framework centres on five central questions of assessing CBT competence: why, what, when, who and how competence is assessed. The evidence base is used to explore each of these key questions and make recommendations (see Fig. 1 for an overview of key guidelines).

Figure 1. Recommendations to guide effective and useful strategies for assessing CBT competence.

Why assess CBT competence?

There are several reasons why the assessment of CBT competence is a critical component in maintaining ongoing delivery of high-quality CBT. First, effective assessment of CBT competence provides a means of monitoring standards of practice, thus ensuring treatment provision continues to be delivered in line with current best practice guidelines and in a way that is optimally effective for clients (Kazantzis, Reference Kazantzis2003). Second, competence assessments facilitate evaluation of the training of CBT therapists and ensure newly qualified therapists have not only acquired the necessary knowledge and skills but can also apply these in clinical practice (Decker et al., Reference Decker, Jameson and Naugle2011). Third, competence assessments play a vital role in supporting acquisition and development of the knowledge and skills required to effectively deliver CBT by offering targeted, structured and focused feedback, promoting self-reflection, and guiding future learning (Bennett-Levy, Reference Bennett-Levy2006; Laireiter and Willutzki, Reference Laireiter and Willutzki2003; McManus et al., Reference McManus, Rosen, Jenkins, Mueller, Kennerley, McManus and Westbrook2010a). These issues have been highlighted by the United Kingdom’s (UK) initiative to Improve Access to Psychological Therapies (IAPT) (Clark, Reference Clark2018), which has necessitated the large-scale training of psychological therapists to deliver evidence-based interventions in routine care. Such initiatives have relied heavily on assessment of competence in CBT to inform and evaluate the training of therapists as well as their subsequent provision of evidence-based interventions.

Assessments of CBT competence can serve a range of individual, institutional and societal purposes (see Fig. 2). Careful consideration of the purpose of an assessment of competence is an essential first step in deciding what, when, who and how to assess CBT competence. A helpful distinction is between formative and summative functions. Assessments are considered summative when used to establish if a therapist has reached a specified standard of competence (Harlen and Deakin Crick, Reference Harlen and Deakin Crick2003). This is important when certifying therapists’ completion of qualifications, granting accreditation, and monitoring quality of practice. The primary goal of formative assessments, however, is to facilitate learning by using structured feedback, which helps those being assessed recognise their strengths and needs, and identify ways of improving practice (Sadler, Reference Sadler1998). Formative assessment plays an important role not only in training but also in continued professional development and supervision. While formative and summative functions of assessments often overlap, identifying a primary focus as formative or summative will influence decisions about the optimal assessment strategy in a given circumstance. Where the assessment is summative, it is most important to provide a robust assessment, often made on the basis of a range of validated assessment tools that provide overall numerical ratings across all key areas of competence (Epstein, Reference Epstein2007). Where the primary purpose is formative, priority can be given to assessments providing in-depth and individually targeted feedback that encourages self-reflection and guides strategies for further development (Juwah et al., Reference Juwah, Macfarlane-Dick, Matthew, Nicol, Ross and Smith2004; Nicol and Macfarlane-Dick, Reference Nicol and Macfarlane-Dick2006). Whilst many summative assessment methods also offer opportunities for formative feedback, some formative assessment methods may not be suitable to meet summative requirements (Muse and McManus, Reference Muse and McManus2013).

Figure 2. An overview of key individual, institutional and societal purposes of conducting CBT competence assessments.

Key stakeholders’ perspectives of the purpose of the assessment must also be considered. Although often aligned, it is important to be aware of, respond to, and manage discrepancies between key stakeholders’ goals for the assessment of competence when they arise. For example, trainees understandably prioritise optimising their chances of passing anxiety-provoking summative assessments (Van der Vleuten et al., Reference Van der Vleuten, Schuwirth, Scheele, Driessen and Hodges2010). Consequently, there may be a temptation to self-select their ‘best’ therapy sessions for review, to strategically and narrowly focus on learning and delivering the knowledge and skills being assessed by the measures of competence, or to attend only to negative feedback or the overall numeric feedback grade awarded. Anticipating such conflicts of interest allows for management strategies to be put in place to support formative development. This may include developing non-judgemental learning environments in which trainees are well-prepared for, understand, and are actively engaged in the process of how, why and when they will be assessed; ensuring feedback is delivered in a non-threatening, engaging manner within the context of a supportive supervisory or training relationship; clearly highlighting the most important ‘take home’ messages; ensuring feedback is strengths-based; allowing time and space for trainees to digest and respond to both positive and corrective feedback, and implementing formative as well as summative evaluations.

What is CBT competence?

A critical issue when assessing CBT competence is for all parties involved to agree a clear working definition of what is understood by the term CBT competence. This is not necessarily straightforward, as CBT competence is a complex construct involving multiple distinct aspects of knowledge and skill. For example, it needs to be decided whether to include protocol-specific elements, or non-CBT-specific general therapeutic skills. Broadly speaking, competent delivery of CBT can be defined as the degree to which a therapist demonstrates the knowledge and skills necessary to appropriately deliver CBT interventions in line with the current evidence base for treatment of an individual client’s presenting problem and with sensitivity to the patient’s assets, needs and values (Barber et al., Reference Barber, Sharpless, Klostermann and McCarthy2007; Kaslow, Reference Kaslow2004). However, the broad range of treatment approaches under the CBT umbrella and constantly evolving nature of CBT means that there is a lack of agreement about what constitutes effective CBT and therefore what is considered competent CBT practice. Roth and Pilling (Reference Roth and Pilling2007) presented a framework of competences required to deliver effective CBT. While this framework has not been empirically investigated, it is reflective of expert opinion at that time and offers the most comprehensive overview of CBT competence available, identifying over 50 inter-related competencies.

The first group of Roth and Pilling’s (Reference Roth and Pilling2007) competences are generic competences applicable across psychological therapies, including aspects of generic therapeutic knowledge (e.g. of mental health problems, ethical guidelines, etc.) and skills (e.g. ability to foster therapeutic alliance, warmth, empathy, listening skills, etc.). Four further aspects of competence relate to the knowledge and skills specific to the domain of CBT. Basic CBT competences are those used in most CBT interventions (e.g. knowledge of CBT principles, ability to explain CBT rationale, etc.). Specific behavioural and cognitive techniques relate to the use of specific techniques employed in most interventions (e.g. exposure techniques, working with safety-seeking behaviours, etc.). Problem-specific CBT skills are those used to deliver treatment packages for a particular problem presentation (e.g. understanding the role of hypervigilance in anxiety disorders, behavioural experiments to modify catastrophic misinterpretation). Finally, metacompetences are those needed to flexibly apply, adapt and pace CBT according to individual client needs (e.g. capacity to select and apply most appropriate CBT method, manging obstacles, etc.). Although typically not evaluated within CBT competence assessments, this set of knowledge and skills also sits within a broader framework of professional competences, ranging from effective use of supervision to research engagement and self-reflection (Barber et al., Reference Barber, Sharpless, Klostermann and McCarthy2007; Kaslow, Reference Kaslow2004). These key areas of knowledge and skills required to competently deliver CBT are outlined in Fig. 3.

Figure 3. A framework of clinical knowledge and skills required to competently deliver CBT drawn from Barber et al. (Reference Barber, Sharpless, Klostermann and McCarthy2007), Kaslow (Reference Kaslow2004), and Roth and Pilling (Reference Roth and Pilling2007).

Examining the full range of CBT competences and using this to specify a working definition of CBT competence allows a decision to be made about which aspects of CBT competence should be assessed. Given the broad range of competences required to deliver CBT and the variety of ways of working within the CBT framework, assessments typically focus on a specified subset of generic and CBT-specific knowledge or skill. Given that it is essential that the methods used to assess competence are well aligned with the competences being evaluated, this decision directly influences decision about how competence will be assessed. This may differ depending on the context and situation. For example, within a CBT training setting it may be important to implement a broad multi-method assessment strategy that assesses the breadth of a therapist’s generic and CBT-specific knowledge and skills. Conversely, developmental feedback within routine CBT supervision may focus on a specific skill or on the ability to deliver a specific treatment strategy. Regardless of the decision made, it is important that those completing the assessment, those being assessed, and those reviewing assessment outcomes are aware of, and understand, what aspects of competence are being examined and why.

When should CBT competence be assessed?

The ‘when’ of CBT competence assessment is important both within and outside of formal training settings. CBT competence assessments are firmly embedded within formal postgraduate CBT training settings in the UK, where multi-source assessments are submitted at several points over the course of training. These practice portfolios usually include assessment of several recorded therapy sessions using standardised rating scales, alongside written case reports and essays (Liness et al., Reference Liness, Beale, Lea, Byrne, Hirsch and Clark2019; McManus et al., Reference McManus, Westbrook, Vazquez-Montes, Fennell and Kennerley2010b). This allows evaluation of the direct and indirect of training efforts across key areas of Kirkpatrick’s (Reference Kirkpatrick, Craig and Bittel1967) training evaluation model: acquisition of knowledge and skill, behaviour in practice settings, and client outcomes (Decker et al., Reference Decker, Jameson and Naugle2011). Within training contexts, assessments serve to establish whether trainee therapists meet a sufficient standard of CBT competence against specified benchmarks and learning outcomes. Training courses must therefore outline and gain institutional agreement as to when and how CBT competence is assessed. This relatively prescribed approach is necessary to ensure a robust and consistent strategy for assessing the acquisition and implementation of general therapeutic and CBT-specific knowledge and skills across training and practice settings (Decker et al., Reference Decker, Jameson and Naugle2011; Muse and McManus, Reference Muse and McManus2013). Yet it is important to remember that this close monitoring and regular feedback on performance is also designed to support the development and consolidation of therapeutic competences. Receiving feedback of in-session performance using standardised rating scales provides useful corrective feedback to trainees, leading to further improvements in competence (Weck et al., Reference Weck, Kaufmann and Höfling2017a; Weck et al., Reference Weck, Junga, Kliegl, Hahn, Brucker and Witthöft2021). Thus, for those completing or receiving assessments within training settings, the key issue may be in understanding the reasons why this strategy has been selected and identifying ways to make use of the feedback within these frameworks to support trainees personal learning goals.

Outside of training settings, ongoing monitoring of competence is not only necessary in the early skills-acquisition phase but is also essential for both novice and experienced therapists to maintain knowledge and skills and prevent ‘therapist drift’ away from optimum delivery of evidence-based CBT treatment practices (Waller and Turner, Reference Waller and Turner2016). Evidence suggests that engaging in such regular deliberate practice can lead to better client outcomes in psychotherapy (Chow et al., Reference Chow, Miller, Seidel, Kane, Thornton and Andrews2015). Furthermore, therapeutic regulatory bodies typically use CBT competence assessments to provide evidence of competent practice for accreditation purposes. For example, within the UK and Ireland the British Association of Behavioural and Cognitive Psychotherapies (BABCP) provide CBT practitioner accreditation, which is maintained through periodic re-accreditation. Criteria for ongoing re-accreditation include engagement in ongoing supervision involving regular formal assessment of skill viewed in vivo or via session recordings using standardised rating scales (British Association of Behavioural and Cognitive Psychotherapies, 2012). However, using competence assessments within supervision offers much more than a formal annual proof of competence for accreditation purposes. Good practice guidelines recommend that formative competence assessments should be embedded within routine CBT supervision practices to support life-long competence development. In particular, standardised competence assessments based on direct observation of therapists’ skills within treatment sessions or role-plays provides structured, specific, accessible and accurate supervisor feedback, which is useful for supporting supervisee self-reflection and planning for further competence development (Milne, Reference Milne2009; Padesky, Reference Padesky and Salkovskis1996). Worryingly supervision frequently relies on ‘talking about’ therapy within therapist-selected cases, rather than observation of therapist skill during supervisor-selected treatment sessions or role-plays (Ladany et al., Reference Ladany, Hill, Corbett and Nutt1996; Townend et al., Reference Townend, Iannetta and Freeston2002; Weck et al., Reference Weck, Kaufmann and Witthöft2017b). Pragmatic and flexible use of competence assessments can support these methods to be routinely embedded within supervision. For example, the ‘I-spy’ technique can be used within supervision to focus on developing a specific micro skill, such as agenda setting or reflective summaries (Gonsalvez et al., Reference Gonsalvez, Brockman and Hill2016). This technique involves reviewing segments of a recorded treatment session within which the specific skill is demonstrated to identify opportunities for alternative ways of responding. Using single items from standardised rating scales can provide useful insight into both parties’ perspective on the supervisee’s performance, as well as providing a framework to support identification of potentially more skilful responses.

Who is best placed to assess CBT competence?

After relevant decisions have been made regarding what to assess and when, the next question is who should carry out the assessment. Research suggests that CBT therapists are not accurate when assessing their own competence. Evidence about the direction of self-assessment biases is mixed, with most research reporting over-estimation of performance compared with independent assessors (Brosan et al., Reference Brosan, Reynolds and Moore2008; Hogue et al., Reference Hogue, Dauber, Lichvar, Bobek and Henderson2015; Parker and Waller, Reference Parker and Waller2015; Rozek et al., Reference Rozek, Serrano, Marriott, Scott, Hickman, Brothers, Lewis and Simons2018; Walfish et al., Reference Walfish, McAlister, O’Donnell and Lambert2012) and one study reporting an under-estimation (McManus et al., Reference McManus, Rakovshik, Kennerley, Fennell and Westbrook2012). Thus it is not recommended that self-assessments be used as a formal summative measure of therapist competence (Muse and McManus, Reference Muse and McManus2013). However, self-monitoring can still have a useful role in developing and maintaining CBT competences. Self-assessment is a core aspect of reflective practice that allows therapists to manage their own learning and facilitates identification of professional development needs. The ability to accurately self-assess competence is also a meta-competence that can be improved with training and supervisory feedback (Beale et al., Reference Beale, Liness and Hirsch2020; Brosan et al., Reference Brosan, Reynolds and Moore2008; Loades and Myles, Reference Loades and Myles2016). Self-assessing competence by rating performance within therapy or role-play sessions may be a useful self-development strategy, especially when used in combination with objective assessor or supervisory ratings. The collaborative comparative process may support therapists’ ability to both ‘reflect-on’ and ‘reflect-in’ action, facilitate more accurate self-awareness of competence, and allow assessors to tailor feedback depending on individual therapists’ self-confidence.

There has been a move towards competency-based CBT supervision paradigms that include regular assessment of supervisee competence. However, research suggests that supervisors are compromised by leniency errors and halo effects (Gonsalvez and Crowe, Reference Gonsalvez and Crowe2014), resulting in supervisors assessing CBT competence more positively than independent judges (Dennhag et al., Reference Dennhag, Gibbons, Barber, Gallop and Crits-Christoph2012; Peavy et al., Reference Peavy, Guydish, Manuel, Campbell, Lisha, Le, Delucchi and Garrett2014). This ‘positivity bias’ calls into question the ability of supervisors to accurately complete high-stakes summative assessments (Muse and McManus, Reference Muse and McManus2013). Independent assessors may be less influenced by demand characteristics such as a pressure to award a ‘pass’, relationship dynamics, or the halo effect caused by information beyond assessment material (e.g. prior competence, ability in other domains). Yet the factors that increase supervisory biases may also mean that supervisors are well placed to offer global insight into a supervisee’s ability across a range of competences over time, and to provide formative feedback on these. For example, supervisors have access to a greater wealth of contextual information about the treatment context, client’s history, supervisee’s work across time and different clients, supervisee’s professional interactions, and supervisee’s developmental stage. This allows supervisors to tailor feedback to support ongoing, individualised, developmentally appropriate and scaffolded learning strategies within the context of a supervisory alliance. Peer feedback delivered within the context of group or peer supervision may also be considered as a strategy for formative assessment of CBT competence. Although potentially less accurate and reliable, peer assessment may be less threatening and can equip therapists with the skills to both provide constructive feedback to others and to self-assess their own skills.

An ideal approach to assessment of CBT competence would be to triangulate across assessments completed by supervisors, therapists and independent assessors (Muse and McManus, Reference Muse and McManus2013). However, the time and financial cost of this strategy means it may not be possible, and for some purposes not necessary. Decisions about who should complete assessments will therefore need to appraise these pros and cons and make a pragmatic decision informed by the assessment purpose and cost. Strategies to reduce potential biases should also be considered. For example, where supervisory assessments are used, reliability checks could be completed by independent assessors for a subset of core competences at key points in a therapist’s training or practice. Active awareness of possible biases, engagement in reflexive practice whereby the potential impact of personal beliefs, attitudes, assumptions on assessment is acknowledged and examined, and supervision of supervision may help to overcome biases.

Whether supervisors, independent assessors, peers, or self-ratings are used, it is also necessary to provide appropriate training in how to use the competence measures employed. The provision of training should involve as a minimum developing familiarity with rating guidelines and completing practice ratings to standardise ratings (Milne, Reference Milne2009). However, individual, peer or group ‘supervision of assessment’ may also be beneficial in supporting assessors to complete high-quality assessments (Milne, Reference Milne2009). Supervision of supervision frameworks have been suggested to support supervisor development (Kennerley, Reference Kennerley2019; Milne, Reference Milne2009) and these approaches may also be applicable within broader peer and independent assessor contexts. Training and supervision for assessors not only enhances the inter-rater reliability of assessments (Kühne et al., Reference Kühne, Meister, Maaß, Paunov and Weck2020), but also supports effective delivery and receipt of feedback. Research suggests that therapist competence cannot be reliably rated by trained novices (Weck et al., Reference Weck, Hilling, Schermelleh-Engel, Rudari and Stangier2011) and novices may also be less skilled in providing appropriate formative feedback. Yet it is unclear what expertise is necessary to assess competence, with findings showing mixed evidence about whether accuracy of assessments increases with assessor competence (Brosan et al., Reference Brosan, Reynolds and Moore2008; Caron et al., Reference Caron, Muggeo, Souer, Pella and Ginsburg2020; Hogue et al., Reference Hogue, Dauber, Lichvar, Bobek and Henderson2015; McManus et al., Reference McManus, Rakovshik, Kennerley, Fennell and Westbrook2012). As a minimum, it is recommended that assessor or supervisory based assessments are carried out by therapists who have themselves received formal training or accreditation as a CBT therapist and who have significant experience in CBT practice.

How should CBT competence be assessed?

A broad range of methods can be used to assess CBT competence with different methods assessing distinct aspects of CBT competence and serving different functions (Muse and McManus, Reference Muse and McManus2013). The recommended ‘gold standard’ approach that is typically employed within CBT training courses is to use a multi-source, multi-informant, multi-method practice portfolio to provide a comprehensive and rounded assessment of therapists’ CBT knowledge and skills (Decker et al., Reference Decker, Jameson and Naugle2011; Muse and McManus, Reference Muse and McManus2013). Ensuring that trainees have reached proficiency across a range of core areas of knowledge and skills is central to high-stakes summative assessments that are used to make an overall judgement of CBT competence for qualification or accreditation purposes. However, such a robust, time-consuming and costly method is not always practical, or necessarily needed or useful in routine professional development, where individual methods may be judiciously selected and applied to develop skills and strengthen supervision. Thus, how competence should be assessed will depend on a variety of contextual factors that influence the nature and purpose of the assessment. For example, assessments may be used to give formative feedback within a training setting, to evaluate the impact of training efforts, as evidence for accreditation purposes, to promote self-reflection, to deliver broad feedback within supervision, or to support development of a specific skill within supervision. It is, therefore, necessary to explore the toolbox of methods available, to be aware of the relative strengths and weaknesses of these methods, and to select the right tool(s) for the job.

Miller’s (Reference Miller1990) hierarchy of clinical skill has been used to categorise the different methods of assessing CBT competence into four hierarchical levels (Muse and McManus, Reference Muse and McManus2013; see Fig. 4). This framework identifies which key aspects of competence are assessed by the different methods available. Table 1 outlines each of these methods, provides examples of key tools that can be used, identifies key strengths and challenges of the method, and makes recommendations for appropriate contexts in which to use them.

Figure 4. A framework for CBT therapist competence measures as aligned to Miller’s (Reference Miller1990) clinical skills hierarchy (Muse and McManus, Reference Muse and McManus2013).

Table 1. An overview of available assessment methods for assessing CBT competence drawn from Muse and McManus (Reference Muse and McManus2013)

* This does not provide an exhaustive list of tools available to measure competence but identifies some of the most commonly used tools available in assessing competence in the delivery of individual CBT.

The foundation of CBT competence is a sound understanding of the scientific, theoretical and contextual basis of CBT (‘knows’) and the ability to use this knowledge to inform when, how and why CBT interventions should be implemented (‘knows how’). These foundational aspects of competence can be assessed relatively easily, quickly, inexpensively and reliably using multiple-choice questions, essays, case reports and short-answer clinical vignettes. The upper levels in the hierarchy focus on assessing higher-order skills necessary to draw on and apply this knowledge in clinical situations. Level 3 refers to demonstration of skills within carefully constructed artificial clinical simulations (‘shows how’) and can be assessed using observational scales to rate performance within standardised role-plays. This approach offers the potential for scalable, standardised assessment of a range of clinical skills across varied client presentations and complexities. The highest level is the ability to use these skills within real clinical practice settings (‘does’ independently in practice). This can be assessed using client surveys or outcomes, supervisory assessments, and ratings of treatment sessions (self and assessor ratings). Client outcomes and satisfaction may be problematic as they are not direct measures of CBT competence and are influenced by other factors (e.g. client responsiveness to treatment, quality of the therapeutic relationship). Supervisory assessments offer broad and global insight into therapist competence over time and across situations but are also influenced by a number of biases. Self and assessor ratings of treatment sessions are therefore the most commonly used method for assessing skill in practice.

Several different scales have been developed for assessing CBT skills within routine and role-play treatment sessions. These can be broadly categorised as ‘transdiagnostic’ or ‘disorder-specific’. Transdiagnostic scales are designed to assess competences that underpin most CBT interventions, whilst disorder-specific scales assess the competence with which CBT treatment packages for a specific problem presentation are delivered. Disorder-specific scales may be beneficial within highly specialised training and treatment settings but have less applicability within broader training or practice settings in which therapists work across a range of protocols and approaches. These rating scales are generally designed to apply across any ‘active’ session utilising CBT intervention strategies. This means sessions focusing on other essential CBT activities such as assessment, formulation or relapse management often fall outside the remit of standardised measures. As CBT expands and develops, new measures are also being created to assess competence in these new ways of working (e.g. third-wave interventions, low-intensity interventions etc.).

Caution must be exercised when making judgements about clinical competence using any single scale as it is unlikely that any scale will provide a full and robust measure of CBT competence that is suitable across all settings, contexts, and for all purposes, Nonetheless the CTS-R (Blackburn et al., Reference Blackburn, James, Milne, Baker, Standart, Garland and Reichelt2001) has been widely used as the benchmark for assessing clinical skill across research, training courses and clinical services for more than two decades. However, this was a pragmatically developed tool and there is a lack of empirical evidence supporting this position, thus the CTS-R has been criticised for poor validity, reliability and usability (Muse and McManus, Reference Muse and McManus2013; Rayson et al., Reference Rayson, Waddington and Hare2021). Furthermore, the degree to which any scale provides a valid and reliable measure of CBT competence will be heavily influenced by the way the scale is implemented (e.g. number of assessors, assessor training, moderation, which sessions are selected, etc.) (Muse and McManus, Reference Muse and McManus2013; Roth et al., Reference Roth, Myles-Hooton and Branson2019). Thus it is important to avoid over-reliance on any single measure of CBT competence, to carefully consider not only which scale to use but how to implement it, and to be realistic about what conclusions can drawn about a therapist’s competence as a result of this assessment.

Theoretical models can be helpful in considering how to optimise learning from CBT competence assessments. Although it has received criticism, Kolb’s experiential learning model (Kolb et al., Reference Kolb., Boyatzis, Mainemelis., Sternber and Zhang2001) provides a useful framework to guide therapists through key learning phases and can be easily embedded within CBT training, supervision and professional development practices. Kolb’s model involves four components that interact within a continuous ‘learning cycle’: experience (engaging in an activity), observation (reflection on the experience), abstract conceptualisation (identifying learning), and active experimentation (putting learning into practice). Optimal use of assessments is made when a strategy for delivering and receiving feedback from CBT competence assessments is aligned with this continual process of action and reflection. Within Miller’s (Reference Miller1990) hierarchical framework, the highest and most complex levels of competence to assess are the application of clinical skills in clinical simulations (such as role-plays) and real clinical practice settings. A widely used approach for assessing clinical skills in training and professional development settings is to use observational rating scales such as the CTS-R (Blackburn et al., Reference Blackburn, James, Milne, Baker, Standart, Garland and Reichelt2001) or ACCS (Muse et al., Reference Muse, McManus, Rakovshik and Thwaites2017) to rate performance. The discussion below outlines ways both therapists completing competence ratings (referred to as ‘assessors’) and therapists receiving ratings (referred to as ‘therapists’) can use feedback from observational rating scales to support the cyclical learning process (see Fig. 5 for an overview).

Figure 5. Using the experiential learning model (Kolb et al., Reference Kolb., Boyatzis, Mainemelis., Sternber and Zhang2001) to embed skills-based CBT competence assessments within a continual process of action and reflection.

The first phase of learning is engaging in an activity, in this case a therapeutic encounter within a treatment session or role-play scenario. Experience is a central component of the learning process, with initial engagement in the activity providing the therapist with concrete experience of their actions and an evaluation of the subsequent consequences (Kolb et al., Reference Kolb., Boyatzis, Mainemelis., Sternber and Zhang2001). The therapist and assessor both need to be able to view the session. ‘Live’ in vivo viewing allows for a richer and more authentic insight. However, session recordings are more practical and can also be revisited. Visual recordings are optimal because they provide rich contextual information (e.g. non-verbal behaviour), followed by audio recordings, with transcripts of a real session being useful but less so than recordings. Therapists may be reluctant to record routine therapy sessions due to feeling self-conscious or fearing negative client reactions. An ‘audit-based’ approach involving routine session recording normalises the process as well as providing a pool of recordings. This allows purposeful selection of recordings to demonstrate competence or seek feedback on skills in working within a particular problem area or using a specific CBT model or intervention. This also enables supervisors to reduce therapist selection biases by periodically reviewing supervisee caseloads and reviewing recordings from those not brought to supervision (Padesky, Reference Padesky and Salkovskis1996). Adherence to ethical and legal guidelines is also supported by using standardised procedures to guide clients’ informed consent and storage of recordings. The use of role-plays circumvents these practical issues but does require careful development and adept portrayal of appropriate, relevant, contextualised and realistic scenarios. Additional contextual information is often necessary to enable assessors to make informed ratings. This may include stage of therapy, nature of presenting problem, formulation, client goals, session agenda, outcome data, and relevant homework or in-session materials (e.g. questionnaires, diaries, thought records, etc.). Weck et al.’s (Reference Weck, Grikscheit, Höfling and Stangier2014) findings suggested that ratings can reliably be made based on the middle third of a session, which would reduce time demands. However, it is more challenging to capture or reliably rate specific aspects of competence (Weck et al., Reference Weck, Grikscheit, Höfling and Stangier2014), thus undermining using segments for summative purposes or for providing overall performance feedback. It has been suggested that as many as three clients per therapist and four sessions per client are needed to achieve suitable reliability (Dennhag et al., Reference Dennhag, Gibbons, Barber, Gallop and Crits-Christoph2012). Collecting multiple assessors’ ratings of the same session can also increase reliability (Vallis et al., Reference Vallis, Shaw and Dobson1986) and reduce halo effects (Streiner and Norman, Reference Streiner and Norman2003). Thus the ideal approach to summative assessment would be for multiple assessors to rate a number of sessions, drawn across different clients or role-play scenarios. As this approach is prohibitively resource intensive, competence judgements within training and accreditation contexts are often based on three or four full sessions per therapist.

Phase 2 of Kolb’s learning cycle involves purposeful and reflective observation of what happened within the CBT treatment or role-play session. Observational CBT competence rating scales completed by assessors and therapists (i.e. self-ratings) can support this reflective process. The assessment strategy should outline which scale is most suitable for the given purpose and context. Both parties need training in how to use the scale and need to establish how the scale is being used. For example, whether the scale is used to provide a broad overview of competences or to focus in on a specific skill? It is also important to be clear about what is considered good performance. This will vary according to the developmental level of the therapist and should be tailored to the individual therapist’s learning goals as well as any specified benchmarks. Whichever scale is being used, the following general guidelines can be helpful when using observational rating scales:

Refer to item descriptors to anchor ratings on each item.

  • Make notes whilst viewing the session but wait until the full session recording has been viewed to provide numerical ratings and finalise feedback.

  • Score each item on the scale independently to avoid relying on an overall global impression.

  • Client progress should not influence the rating provided.

  • Refer back to rating manuals to support consistency and prevent assessor drift.

The third phase refers to drawing on reflective observations to form abstract concepts and general principles (Kolb et al., Reference Kolb., Boyatzis, Mainemelis., Sternber and Zhang2001). This involves making sense of and learning from feedback obtained through self and assessor ratings completed using observational rating scales. Giving and receiving feedback can be uncomfortable, especially when feedback is corrective or summative. However, assessors can implement strategies to support this process (James, Reference James2015; Gonsalvez and Crowe, Reference Gonsalvez and Crowe2015; Kennerley, Reference Kennerley2019). Such strategies may include giving a written or verbal feedback summary alongside numerical scale ratings to provide valuable formative feedback. Rating scales can be used to inform this feedback summary by providing a ‘competence profile’ highlighting areas of strength and those where improvement is needed. As well as identifying opportunities for skills development, it is important to recognise and highlight strengths to ensure these are maintained, enhanced and reinforced. The feedback should be specific, giving concrete examples drawn from the rating scale and session material. The summary should also identify key ‘take home’ messages. This is especially important for novice therapists, who may feel overwhelmed and find it hard to identify which the most important issues are. The assessment purpose, context and assessor–therapist alliance will also impact how feedback will be received and should therefore shape the content and tone. For example, a novice trainee completing a summative assessment may feel nervous and less confident in their abilities. Here the assessor–therapist alliance might be used to create a setting where the trainee feels safe enough to hear and engage with the feedback. Then, the key feedback focus may be to encourage motivation and self-esteem. There is a danger that therapists passively ‘receive’ feedback ‘delivered’ by an assessor. Yet feedback should be a collaborative and dynamic endeavour involving Socratic methods of learning (Padesky, Reference Padesky and Salkovskis1996; Kennerley, Reference Kennerley2019; Kennerley and Padesky, Reference Kennerley, Padesky, Padesky and Kennerley2023). The more active and involved in the process the therapist can be, the more they can take control of and reflect on their own learning progress (Nicol and Macfarlane-Dick, Reference Nicol and Macfarlane-Dick2006). Therapist engagement with the learning experience is likely to be enhanced by incorporating therapist preferences about what they would like feedback on, when, and in what format. For example, some may prefer to receive written feedback prior to supervision to allow time to digest this, others may prefer to explore feedback within supervision; some choose to hear critical statements first, others prefer to learn what has been done well before hearing what might be improved. Completing self-assessments allows therapists to identify their own strengths, deficits and needs before receiving external feedback. This also allows comparisons between assessor and therapist ratings, therefore identifying discrepancies in self-view of competence and offering more perspectives for reflection.

The final phase is a future planning process, namely, how to put learning into practice and apply the abstracted principles across contexts (Kolb et al., Reference Kolb., Boyatzis, Mainemelis., Sternber and Zhang2001). This stage is vital for generalisation and application of learning within future therapeutic activities, which can themselves be reflected upon. It can be helpful for therapists to develop a specific learning action plan to support this. Feedback from rating scales can identify which areas of knowledge and skill the therapist wishes or needs to develop, thus shaping individualised learning goals. The plan also needs to outline how these can be achieved. Assessors can facilitate this by including ‘feedforward’: constructive guidance about ways competence could be further developed. This might include corrective feedback, such as offering suggestions of alternative more skilful ways of working. It can also identify specific strategies for supporting development of declarative knowledge (e.g. suggested reading, sharing case studies, etc.) and procedural skills (e.g. experiential training, role-plays in supervision, further clinical experience, etc.). Supervision is essential in supporting competence development (Rakovshik et al., Reference Rakovshik, McManus, Vazquez-Montes, Muse and Ougrin2016; Watkins, Reference Watkins2011) and may be especially helpful in facilitating this final phase of learning. Well-structured supervision offers a secure environment within which therapists can sensitively explore their understandings of, and reactions to, feedback. Supervision also supports movement away from an information transmission model of delivering feedback. Instead, dialogue can be used to collaboratively co-construct meaning, thus supporting the therapist in understanding and internalising feedback (Nicol and Macfarlane-Dick, Reference Nicol and Macfarlane-Dick2006). Through a mutual process of discovery, therapist and supervisor can create and implement a developmentally appropriate learning action plan that will ultimately enhance client care. Although it may be preferrable from a developmental perspective for rating feedback to be provided by the supervisor, ratings completed by independent assessors can be usefully explored. Feedback can also be provided by peer-supervisors, as it is not always possible or necessary for the supervisor to have an ‘expert’ role. Viewing clips from session recordings may further illustrate feedback and aid discussion. Supervision also provides opportunity for modelling particular skills and for experiential activities (e.g. role-play, ‘chair work’). This can help consolidate learning, prime for more fluent performance, and enhance procedural knowledge about how to implement new skills (Bennett-Levy, Reference Bennett-Levy2006).

Conclusions

Much progress has been made in defining the range of competences that constitutes ‘CBT competence’. This has led to examination and evaluation of different ways of assessing, monitoring and enhancing therapists’ competences. Although the assessment of CBT competence is a complex and challenging issue, it remains a vital component in ensuring the ongoing delivery of high-quality CBT. Thus there is a need to encourage a shift in culture, whereby the allocation of resources to the assessment of CBT competence is more clearly recognised and prioritised. There is a particular need for CBT competence monitoring to be more routinely embedded within routine clinical practice to facilitate continued professional development, self-reflection and maximise the efficacy of supervision. A range of different methods can be used to assess therapists’ generic and CBT-specific knowledge and skills, all of which have inherent advantages and disadvantages. There is a need to continue to build upon and improve the reliability, validity and feasibility of current assessment methods as well as exploring innovative methods of assessing competence. Due to the multi-faceted nature of CBT competence and the limitations of existing methods of assessing competence, a standardised ‘one size fits all’ approach cannot be taken when determining how best to assess CBT competence. Instead, a pragmatic assessment strategy needs to be specified and implemented according to the specific context and purpose of the assessment. This must consider what aspects of competence to assess, which method or combination of methods to use, and how these methods will be implemented (i.e. when and by whom). This should also include a strategy for delivering and receiving feedback to enhance formative functions and optimise learning. Formative feedback is important in supporting acquisition and consolidation of competence for trainee or newly qualified therapists and is also essential for more experienced therapists to support lifelong learning and reduce therapist drift. Future development in the area may consider how best to support those assessing competence – that is, how we assess, develop and monitor skill in assessing competence.

Key practice points

  1. (1) There is a need for a culture shift in which ongoing and career-long assessment of CBT competence is supported, encouraged and valued. It is particularly important to consider pragmatic and flexible ways to routinely embed assessments based on direct observation of skills (in sessions or role-plays) into routine supervision or audit practices to support lifelong learning.

  2. (2) Collaboration is key. Therapists should be firmly situated at the heart of CBT competence assessed by others. Approaching assessment as a collaborative venture whereby the therapist being assessed is actively engaged in the process promotes independent learning and fosters self-reflection.

  3. (3) It is important to strengthen the formative function of CBT competence assessments in order to optimise learning. Experiential learning models and effective communication models can be used as frameworks to guide effective delivery and receipt of feedback and support therapists along their individual learning journey.

  4. (4) Caution must be exercised when making judgements about CBT competence based on limited evidence, especially for summative purposes. A multi-source, multi-informant, multi-method practice portfolio provides the most comprehensive and robust assessment of CBT competence.

  5. (5) There is not a ‘one size fits all’ approach to assessing CBT competence. Instead, careful consideration needs to be given to which assessment methods to use and how and when they should be implemented given the specific context and purpose.

Data availability statement

No new data were generated for this publication.

Acknowledgements

None.

Author contributions

Kate Muse: Conceptualization (lead), Project administration (lead), Resources (lead), Writing – original draft (lead); Helen Kennerley: Conceptualization (supporting), Resources (supporting), Writing – review & editing (equal); Freda McManus: Conceptualization (supporting), Resources (supporting), Writing – review & editing (equal).

Financial support

This research received no specific grant from any funding agency, commercial or not-for-profit sectors.

Conflicts of interest

None

Ethical standards

The authors have abided by the Ethical Principles of Psychologists and Code of Conduct as set out by the BABCP and BPS.

References

Further reading

Decker, S. E., Jameson, M. T., & Naugle, A. E. (2011). Therapist training in empirically supported treatments: a review of evaluation methods for short-and long-term outcomes. Administration and Policy in Mental Health and Mental Health Services Research, 38, 254286.CrossRefGoogle ScholarPubMed
McManus, F., Rosen, K., & Jenkins, H. (2010). Developing and progressing as a CBT therapist. In Mueller, M., Kennerley, H., McManus, F. & Westbrook, D. (eds), Oxford Guide to Surviving as a CBT Therapist (pp. 301322). New York: Oxford Universtity Press.CrossRefGoogle Scholar
Muse, K., & McManus, F. (2013). A systematic review of methods for assessing competence in cognitive-behavioural therapy. Clinical Psychology Review, 33, 484499.CrossRefGoogle ScholarPubMed
Muse, K., McManus, F., Rakovshik, S., & Thwaites, R. (2017). Development and psychometric evaluation of the Assessment of Core CBT Skills (ACCS): an observation-based tool for assessing cognitive behavioral therapy competence. Psychological Assessment, 29, 542.CrossRefGoogle ScholarPubMed

References

Barber, J. P., Liese, B. S., & Abrams, M. J. (2003). Development of the Cognitive Therapy Adherence and Competence Scale. Psychotherapy Research, 13, 205221.CrossRefGoogle Scholar
Barber, J. P., Sharpless, B. A., Klostermann, S., & McCarthy, K. S. (2007). Assessing intervention competence and its relation to therapy outcome: a selected review derived from the outcome literature. Professional Psychology-Research and Practice, 38, 493500.CrossRefGoogle Scholar
Barnfield, T. V., Mathieson, F. M., & Beaumont, G. R. (2007). Assessing the development of competence during postgraduate cognitive-behavioral therapy training. Journal of Cognitive Psychotherapy, 21, 140147.CrossRefGoogle Scholar
Beale, S., Liness, S., & Hirsch, C.R. (2020). Trainee self-assessment of cognitive behaviour therapy competence during and after training. the Cognitive Behaviour Therapist, 13.CrossRefGoogle Scholar
Bennett-Levy, J. (2006). Therapist skills: a cognitive model of their acquisition and refinement. Behavioural and Cognitive Psychotherapy, 34, 5778.CrossRefGoogle Scholar
Bennett-Levy, J., & Beedie, A. (2007). The ups and downs of cognitive therapy training: what happens to trainees’ perception of their competence during a cognitive therapy training course? Behavioural and Cognitive Psychotherapy, 35, 6175.CrossRefGoogle Scholar
Blackburn, I. M., James, I. A., Milne, D. L., Baker, C., Standart, S., Garland, A., & Reichelt, F. K. (2001). The revised Cognitive Therapy Scale (CTS-R): psychometric properties. Behavioural and Cognitive Psychotherapy, 29, 431446.CrossRefGoogle Scholar
British Association for Behavioural and Cognitive Psychotherapies (2012). BABCP Minimum Training Standards for the Practice of Cognitive Behavioural Therapy. Retrieved from: https://www.babcp.com/files/Accreditation/General/Minimum-Training-Standards-V7-0215.pdf Google Scholar
Brosan, L., Reynolds, S., & Moore, R. G. (2008). Self-evaluation of cognitive therapy performance: do therapists know how competent they are? Behavioural and Cognitive Psychotherapy, 36, 581587.CrossRefGoogle Scholar
Caron, E. B., Muggeo, M. A., Souer, H. R., Pella, J. E., & Ginsburg, G. S. (2020). Concordance between clinician, supervisor and observer ratings of therapeutic competence in CBT and treatment as usual: does clinician competence or supervisor session observation improve agreement? Behavioural and Cognitive Psychotherapy, 48, 350363.CrossRefGoogle ScholarPubMed
Carroll, K. M., Nich, C., Sifry, R. L., Nuro, K. F., Frankforter, T. L., Ball, S. A., … & Rounsaville, B. J. (2000). A general system for evaluating therapist adherence and competence in psychotherapy research in the addictions. Drug and Alcohol Dependence, 57, 225238.CrossRefGoogle ScholarPubMed
Chow, D. L., Miller, S. D., Seidel, J. A., Kane, R. T., Thornton, J. A., & Andrews, W. P. (2015). The role of deliberate practice in the development of highly effective psychotherapists. Psychotherapy, 52, 337.CrossRefGoogle ScholarPubMed
Clark, D. M. (2018). Realizing the mass public benefit of evidence-based psychological therapies: the IAPT program. Annual Review of Clinical Psychology, 14, 159183.CrossRefGoogle ScholarPubMed
Cooper, Z., Doll, H., Bailey-Straebler, S., Bohn, K., de Vries, D., Murphy, R., … & Fairburn, C. G. (2017). Assessing therapist competence: development of a performance-based measure and its comparison with a web-based measure. JMIR Mental Health, 4, e7704.CrossRefGoogle ScholarPubMed
Cooper, Z., Doll, H., Bailey-Straebler, S., Kluczniok, D., Murphy, R., O’Connor, M. E., & Fairburn, C. G. (2015). The development of an online measure of therapist competence. Behaviour Research and Therapy, 64, 4348.CrossRefGoogle ScholarPubMed
Decker, S., Jameson, M., & Naugle, A. (2011). Therapist training in empirically supported treatments: a review of evaluation methods for short- and long-term outcomes. Administration and Policy in Mental Health and Mental Health Services Research, 38, 254286.CrossRefGoogle ScholarPubMed
Dennhag, I., Gibbons, M. B. C., Barber, J. P., Gallop, R., & Crits-Christoph, P. (2012). Do supervisors and independent judges agree on evaluations of therapist adherence and competence in the treatment of cocaine dependence? Psychotherapy Research, 22, 720730.CrossRefGoogle ScholarPubMed
Dittmann, C., Müller-Engelmann, M., Stangier, U., Priebe, K., Fydrich, T., Görg, N., Rausch, S., Resick, P. A., & Steil, R. (2017). Disorder- and treatment-specific therapeutic competence scales for posttraumatic stress disorder intervention: development and psychometric properties. Journal of Traumatic Stress, 30, 614625.CrossRefGoogle ScholarPubMed
Epstein, R. M. (2007). Assessment in medical education. New England Journal of Medicine, 356, 387396.CrossRefGoogle ScholarPubMed
Gonsalvez, C. J., Brockman, R., & Hill, H. R. (2016). Video feedback in CBT supervision: review and illustration of two specific techniques. the Cognitive Behaviour Therapist, 9.CrossRefGoogle Scholar
Gonsalvez, C. J. & Crowe, T. (2015). Board approved supervisor training. Part 2: Skills based training workshop. Supervisor Training Manual. Melbourne: Australian Psychological Society.Google Scholar
Gonsalvez, C. J., & Crowe, T. P. (2014). Evaluation of psychology practitioner competence in clinical supervision. American Journal of Psychotherapy, 68, 177193.CrossRefGoogle ScholarPubMed
Harlen, W. & Deakin Crick, R. (2003). Testing and motivation for learning. Assessment in Education: Principles, Policy & Practice, 10(2), 169207.Google Scholar
Harstad, S., Bjaastad, J. F., Hjemdal, O., Compton, S., Waaktaar, T., & Aalberg, M. (2021). Competence and Adherence Scale for Cognitive Behavioural Therapy (CAS-CBT) for anxiety disorders in youth: reliability and factor structure. Behavioural and Cognitive Psychotherapy, 49, 745757.CrossRefGoogle Scholar
Hogue, A., Dauber, S., Lichvar, E., Bobek, M., & Henderson, C. E. (2015). Validity of therapist self-report ratings of fidelity to evidence-based practices for adolescent behavior problems: correspondence between therapists and observers. Administration and Policy in Mental Health and Mental Health Services Research, 42, 229243.CrossRefGoogle ScholarPubMed
Haddock, G., Devane, S., Bradshaw, T., McGovern, J., Tarrier, N., Kinderman, P., … Harris, N. (2001). An investigation into the psychometric properties of the Cognitive Therapy Scale for Psychosis (CTS-Psy). Behavioural and Cognitive Psychotherapy, 29, 221233.CrossRefGoogle Scholar
James, I. A. (2015). The rightful demise of the sh* t sandwich: providing effective feedback. Behavioural and Cognitive Psychotherapy, 43, 759766.CrossRefGoogle Scholar
Juwah, C., Macfarlane-Dick, D., Matthew, B., Nicol, D., Ross, D., & Smith, B. (2004). Enhancing student learning through effective formative feedback. The Higher Education Academy, 140, 140.Google Scholar
Kaslow, N. J. (2004). Competencies in professional psychology. American Psychologist, 59, 774781.CrossRefGoogle ScholarPubMed
Kazantzis, N. (2003). Therapist competence in cognitive-behavioural therapies: Review of the contemporary empirical evidence. Behaviour Change, 20, 112.CrossRefGoogle Scholar
Kennerley, H., (2019). CBT Supervision. Oxford: OCTC Booklets.Google Scholar
Kennerley, H., & Padesky, C.A. (2023). Supervision and therapist beliefs. In Padesky, C.A. & Kennerley, H. (eds), Dialogues for Discovery in Psychotherapy. Oxford: Oxford University Press.Google Scholar
Kirkpatrick, D. L. (1967). Evaluation of training. In Craig, R. L. & Bittel, L. R. (eds), Training and Development Handbook. New York: McGraw-Hill.Google Scholar
Kolb., D. A., Boyatzis, R. E., & Mainemelis., C. (2001). experiential learning theory: previous research and new directions. In Sternber, R. L. & Zhang, L. (eds), Perspectives on Thinking, Learning, and Cognitive Styles. New York: Routledge.Google Scholar
Kühne, F., Meister, R., Maaß, U., Paunov, T., & Weck, F. (2020). How reliable are therapeutic competence ratings? Results of a systematic review and meta-analysis. Cognitive Therapy and Research, 44, 241257.CrossRefGoogle Scholar
Kuyken, W., & Tsivrikos, D. (2009). Therapist competence, comorbidity and cognitive-behavioral therapy for depression. Psychotherapy and Psychosomatics, 78, 4248.CrossRefGoogle ScholarPubMed
Ladany, N., Hill, C. E., Corbett, M. M., & Nutt, E. A. (1996). Nature, extent, and importance of what psychotherapy trainees do not disclose to their supervisors. Journal of Counseling Psychology, 43, 1024.CrossRefGoogle Scholar
Laireiter, A. R., & Willutzki, U. (2003). Self-reflection and self-practice in training of cognitive behaviour therapy: an overview. Clinical Psychology and Psychotherapy, 10, 1930.CrossRefGoogle Scholar
Liness, S., Beale, S., Clark, D. M., Salkovskis, P. M., Ehlers, A., & Wild, J. (2021). Assessing panic disorder-specific competencies: evaluation of the Cognitive Therapy Competence Scale for panic disorder. Behavioural and Cognitive Psychotherapy, 49, 197205.CrossRefGoogle ScholarPubMed
Liness, S., Beale, S., Lea, S., Byrne, S., Hirsch, C. R., & Clark, D. M. (2019). Multi-professional IAPT CBT training: clinical competence and patient outcomes. Behavioural and Cognitive Psychotherapy, 28, 114.Google Scholar
Loades, M., & Myles, P. (2016). Does a therapist’s reflective ability predict the accuracy of their self-evaluation of competence in cognitive behavioural therapy? the Cognitive Behaviour Therapist, 9.CrossRefGoogle Scholar
Mathieson, F. M., Barnfield, T., & Beaumont, G. (2010). Are we as good as we think we are? Self-assessment versus other forms of assessment of competence in psychotherapy. the Cognitive Behaviour Therapist, 2, 4350.CrossRefGoogle Scholar
McManus, F., Rakovshik, S., Kennerley, H., Fennell, M., & Westbrook, D. (2012). An investigation of the accuracy of therapists’ self-assessment of cognitive-behaviour therapy skills. British Journal of Clinical Psychology, 51, 292306.CrossRefGoogle ScholarPubMed
McManus, F., Rosen, K., & Jenkins, H. (2010a). Developing and progressing as a CBT therapist. In Mueller, M., Kennerley, H., McManus, F. & Westbrook, D. (eds), Oxford Guide to Surviving as a CBT Therapist (pp. 301322). New York: Oxford Universtity Press.CrossRefGoogle Scholar
McManus, F., Westbrook, D., Vazquez-Montes, M., Fennell, M., & Kennerley, H. (2010b). An evaluation of the effectiveness of Diploma-level training in cognitive behaviour therapy. Behaviour Research and Therapy, 48, 11231132.CrossRefGoogle ScholarPubMed
Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65, 6367.CrossRefGoogle ScholarPubMed
Milne, D. (2009). Evidence-Based Clinical Supervision. Chichester: BPS Blackwell.Google Scholar
Muse, K., & McManus, F. (2013). A systematic review of methods for assessing competence in cognitive–behavioural therapy. Clinical Psychology Review, 33, 484499.CrossRefGoogle ScholarPubMed
Muse, K., McManus, F., Rakovshik, S., & Thwaites, R. (2017). Development and psychometric evaluation of the Assessment of Core CBT Skills (ACCS): an observation-based tool for assessing cognitive behavioral therapy competence. Psychological Assessment, 29, 542.CrossRefGoogle ScholarPubMed
Myles, P. J., Latham, M., & Ricketts, T. (2003). The contributions of an expert panel in the development of a new measure of knowledge for the evaluation of training in cognitive behavioural therapy. Paper presented at the Annual Conference for the British Association of Behavioural and Cognitive Psychotherapy, York University.Google Scholar
Myles, P. J., & Milne, D. L. (2004). Outcome evaluation of a brief shared learning programme in cognitive behavioural therapy. Behavioural and Cognitive Psychotherapy, 32, 177188.CrossRefGoogle Scholar
Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31, 199218.CrossRefGoogle Scholar
Padesky, C. A. (1996). Developing cognitive therapist competency: teaching and supervision models. In Salkovskis, P. (ed), Frontiers of Cognitive Therapy (pp. 266292). New York: Guilford Press.Google Scholar
Parker, Z. J., & Waller, G. (2015). Factors related to psychotherapists’ self-assessment when treating anxiety and other disorders. Behaviour Research and Therapy, 66, 17.CrossRefGoogle ScholarPubMed
Peavy, K. M., Guydish, J., Manuel, J. K., Campbell, B. K., Lisha, N., Le, T., Delucchi, K., & Garrett, S. (2014). Treatment adherence and competency ratings among therapists, supervisors, study-related raters and external raters in a clinical trial of a 12-step facilitation for stimulant users. Journal of Substance Abuse Treatment, 47, 222228.CrossRefGoogle Scholar
Rakovshik, S. G., McManus, F., Vazquez-Montes, M., Muse, K., & Ougrin, D. (2016). Is supervision necessary? Examining the effects of internet-based CBT training with and without supervision. Journal of Consulting and Clinical Psychology, 84, 191.CrossRefGoogle ScholarPubMed
Rayson, K., Waddington, L., & Hare, D. S. (2021). The quality of research exploring in-session measures of CBT competence: a systematic review. Behavioural and Cognitive Psychotherapy, 50, 4056.CrossRefGoogle ScholarPubMed
Roth, A. D. (2016). A new scale for the assessment of competences in cognitive and behavioural therapy. Behavioural and Cognitive Psychotherapy, 44, 620624.CrossRefGoogle ScholarPubMed
Roth, A. D., Myles-Hooton, P., & Branson, A. (2019). Judging clinical competence using structured observation tools: a cautionary tale. Behavioural and Cognitive Psychotherapy, 47, 736744.CrossRefGoogle ScholarPubMed
Roth, A. D., & Pilling, S. (2007). The competences required to deliver effective cognititve and behavioural therapy for people with depression and with anxiety disorders. London: Department of Health.Google Scholar
Rozek, D. C., Serrano, J. L., Marriott, B. R., Scott, K. S., Hickman, L. B., Brothers, B. M., Lewis, C. C., & Simons, A. D. (2018). Cognitive behavioural therapy competency: pilot data from a comparison of multiple perspectives. Behavioural and Cognitive Psychotherapy, 46, 244250.CrossRefGoogle ScholarPubMed
Sadler, D. R. (1998) Formative assessment: revisiting the territory. Assessment in Education, 5, 7784.Google Scholar
Stallard, P., Myles, P., & Branson, A. (2014). The Cognitive Behaviour Therapy Scale for Children and Young People (CBTS-CYP): development and psychometric properties. Behavioural and Cognitive Psychotherapy, 42, 269282.CrossRefGoogle Scholar
Streiner, D. L., & Norman, G. R. (2003). Health Measurement Scales (3rd edn). Oxford: Oxford Medical Publications.CrossRefGoogle Scholar
Townend, M., Iannetta, L., & Freeston, M. H. (2002). Clinical supervision in practice: a survey of UK cognitive behavioural psychotherapists accredited by the BABCP. Behavioural and Cognitive Psychotherapy, 30, 485500.CrossRefGoogle Scholar
Vallis, T., Shaw, B. F., & Dobson, K. S. (1986). The Cognitive Therapy Scale: psychometric properties. Journal of Consulting and Clinical Psychology, 54, 381385.CrossRefGoogle ScholarPubMed
Van der Vleuten, C. P. M., Schuwirth, L. W. T., Scheele, F., Driessen, E. W., & Hodges, B. (2010). The assessment of professional competence: building blocks for theory development. Best practice & research in Clinical Obstetrics & Gynaecology, 24, 703719.CrossRefGoogle ScholarPubMed
von Consbruch, K., Clark, D. M., & Stangier, U. (2012). Assessing therapeutic competence in cognitive therapy for social phobia: psychometric properties of the Cognitive Therapy Competence Scale for Social Phobia (CTCS-SP). Behavioural and Cognitive Psychotherapy, 40, 149161.CrossRefGoogle ScholarPubMed
Walfish, S., McAlister, B., O’Donnell, P., & Lambert, M. J. (2012). An investigation of self-assessment bias in mental health providers. Psychological Reports, 110, 639644.CrossRefGoogle ScholarPubMed
Waller, G., & Turner, H. (2016). Therapist drift redux: why well-meaning clinicians fail to deliver evidence-based therapy, and how to get back on track. Behaviour Research and Therapy, 77, 129137.CrossRefGoogle ScholarPubMed
Watkins, C. E. Jr (2011). Does psychotherapy supervision contribute to patient outcomes? Considering thirty years of research. The Clinical Supervisor, 30, 235256.CrossRefGoogle Scholar
Weck, F., Grikscheit, F., Höfling, V., & Stangier, U. (2014). Assessing treatment integrity in cognitive-behavioral therapy: comparing session segments with entire sessions. Behavior Therapy, 45, 541552.CrossRefGoogle ScholarPubMed
Weck, F., Hilling, C., Schermelleh-Engel, K., Rudari, V., & Stangier, U. (2011). Reliability of adherence and competence assessment in cognitive behavioral therapy: influence of clinical experience. Journal of Nervous and Mental Disease, 199, 276279.CrossRefGoogle ScholarPubMed
Weck, F., Junga, Y. M., Kliegl, R., Hahn, D., Brucker, K., & Witthöft, M. (2021). Effects of competence feedback on therapist competence and patient outcome: a randomized controlled trial. Journal of Consulting and Clinical Psychology, 89, 885897.CrossRefGoogle ScholarPubMed
Weck, F., Kaufmann, Y. M., & Witthöft, M. (2017b). Topics and techniques in clinical supervision in psychotherapy training. the Cognitive Behaviour Therapist, 10.CrossRefGoogle Scholar
Weck, F, Kaufmann, YM, Höfling, V. (2017a). Competence feedback improves CBT competence in trainee therapists: a randomized controlled pilot study. Psychotherapy Research, 27, 501509.CrossRefGoogle ScholarPubMed
Wright, J. H., Wright, A. S., Salmon, P., Beck, A. T., Kuykendall, J., Goldsmith, L. J., & Zickel, M. B. (2002). Development and initial testing of a multimedia program for computer-assisted cognitive therapy. American Journal of Psychotherapy, 56, 7686.CrossRefGoogle ScholarPubMed
Young, J. & Beck, A. T. (1980). Cognitive Therapy Scale: Rating Manual. Unpublished manuscript, University of Pennsylvania, Philadelphia, PA, USA.Google Scholar
Young, J. & Beck, A. T. (1988). Revision of Cognitive Therapy Scale. Unpublished manuscript, University of Pennsylvania, Philadelphia, PA, USA.Google Scholar
Figure 0

Figure 1. Recommendations to guide effective and useful strategies for assessing CBT competence.

Figure 1

Figure 2. An overview of key individual, institutional and societal purposes of conducting CBT competence assessments.

Figure 2

Figure 3. A framework of clinical knowledge and skills required to competently deliver CBT drawn from Barber et al. (2007), Kaslow (2004), and Roth and Pilling (2007).

Figure 3

Figure 4. A framework for CBT therapist competence measures as aligned to Miller’s (1990) clinical skills hierarchy (Muse and McManus, 2013).

Figure 4

Table 1. An overview of available assessment methods for assessing CBT competence drawn from Muse and McManus (2013)

Figure 5

Figure 5. Using the experiential learning model (Kolb et al., 2001) to embed skills-based CBT competence assessments within a continual process of action and reflection.

Submit a response

Comments

No Comments have been published for this article.