Skip to main content Accessibility help
×
Home

The challenge of training supervisors to use direct assessments of clinical competence in CBT consistently: a systematic review and exploratory training study

  • Maria E. Loades (a1) and Peter Armstrong (a2)

Abstract

Evaluating and enhancing supervisee competence is a key function of supervision and can be aided by the use of direct assessments of clinical competence, e.g. the Cognitive Therapy Scale – Revised (CTS-R). We aimed to review the literature regarding inter-rater reliability and training on the CTS and CTS-R to present exploratory data on training raters to use this measure. We employed a systematic review. An exploratory study evaluated the outcomes of a CTS-R supervisor training workshop (n = 34), including self-reported familiarity with and confidence in using the tool, and inter-rater consistency on three CTS-R subscales, pre- and post-training. CTS and CTS-R inter-rater reliability was variable, with evidence of rater training enhancing reliability, although the form, duration and frequency of such training is unclear. The exploratory study found that supervisors rated themselves as more familiar with and confident in using the CTS-R at the end of training compared to at the beginning. However, inter-rater reliability was poor at the beginning and end of the training. Rating competence requires supervisors to make qualitative judgements, which is inherently variable. Training raters has been shown to improve rater reliability, although this was not demonstrated in the exploratory study. Practice implications and future research priorities are identified.

Copyright

Corresponding author

*Author for correspondence: Dr M. E. Loades, Department of Psychology, University of Bath, Claverton Down, Bath BA2 7AY, UK (email: M.E.Loades@bath.ac.uk).

References

Hide All
Milne, DL, Reiser, R (2016). Supporting our supervisors: sending out an SOS. The Cognitive Behaviour Therapist. doi:10.1017/S1754470X15000616.
Muse, K, McManus, F (2013). A systematic review of methods for assessing competence in cognitive-behavioral therapy. Clinical Psychology Review 33, 484499.
Watkins, CE, Milne, DL (2014). The Wiley International Handbook of Clinical Supervision. Oxford, UK: John Wiley & Sons Ltd.
Adler, AD, Strunk, DR, Fazio, RH (2015). What changes in cognitive therapy for depression? An examination of cognitive therapy skills and maladaptive beliefs. Behavior Therapy 46, 96109.
Bennett-Levy, J, Padesky, CA (2014). Use it or lose it: post-workshop reflection enhances learning and utilization of CBT skills. Cognitive and Behavioral Practice 21, 1219.
Bernard, JM, Goodyear, RK (2013). Fundamentals of Clinical Supervision. Boston: Pearson Education.
Blackburn, I-M, James, IA, Milne, DL, Baker, C, Standart, S, Garland, A, Reichelt, FK (2001). The Revised Cognitive Therapy Scale (CTS-R): psychometric properties. Behavioural and Cognitive Psychotherapy 29, 431446.
Branson, A, Shafran, R, Myles, P (2015). Investigating the relationship between competence and patient outcome with CBT. Behaviour Research and Therapy 68, 1926.
Brosan, L, Reynolds, S, Moore, RG (2006). Factors associated with competence in cognitive therapists. Behavioural and Cognitive Psychotherapy 35, 179.
Brown, LA, Craske, MG, Glenn, DE, Stein, MB, Sullivan, G, Sherbourne, C, Bystritsky, A, Welch, SS, Campbell-Sills, L, Lang, A, Roy-Byrne, P, Rose, RD (2013). CBT competence in novice therapists improves anxiety outcomes. Depression & Anxiety 30, 97115.
Bryant, MJ, Simons, AD, Thase, ME (1999). Therapist skill and patient variables in homework compliance: controlling an uncontrolled variable in cognitive therapy outcome research. Cognitive Therapy and Research 23, 381399.
Culloty, T, Milne, D, Sheikh, AI (2010). Evaluating the training of clinical supervisors: a pilot study using the fidelity framework. The Cognitive Behaviour Therapist 3, 132144.
Devane, SM, Haddock, G, Lancashire, S, Baguley, I, Butterworth, T, Tarrier, N, James, A, Molyneux, P (1998). The clinical skills of community psychiatric nurses working with patients who have severe and enduring mental health problems: an empirical analysis. Journal of Advanced Nursing 27, 253260.
Dimeff, LA, Koerner, K, Woodcock, EA, Beadnell, B, Brown, MZ, Skutch, JM, Paves, AP, Bazinet, A, Harned, MS (2009). Which training method works best? A randomized controlled trial comparing three methods of training clinicians in dialectical behavior therapy skills. Behaviour Research and Therapy 47, 921930.
Dimidjian, S, Hollon, SD, Dobson, KS, Schmaling, KB, Kohlenberg, RJ, Addis, ME, Gallop, R, McGlinchey, JB, Markley, DK, Gollan, JK, Atkins, DC, Dunner, DL, Jacobson, NS (2006). Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the acute treatment of adults with major depression. Journal of Consulting and Clinical Psychology 74, 658670.
Dobson, KS, Shaw, BF, Vallis, TM (1985). Reliability of a measure of the quality of cognitive therapy. British Journal of Clinical Psychology 24, 295300.
Dreyfus, HL, Dreyfus, SE (1986). Mind over Machine: the Power of Human Intuition and Expertise in the Age of the Computer. Oxford: Blackwell.
Ginzburg, DM, Bohn, C, Hofling, V, Weck, F, Clark, DM, Stangier, U (2012). Treatment specific competence predicts outcome in cognitive therapy for social anxiety disorder. Behaviour Research and Therapy 50, 747752.
Gordon, PK (2006). A comparison of two versions of the Cognitive Therapy Scale. Behavioural and Cognitive Psychotherapy 35, 343.
Haddock, G, Devane, S, Bradshaw, T, McGovern, J, Tarrier, N, Kinderman, P, Baguley, I, Lancashire, S, Harris, N (2001). An investigation into the psychometric properties of the Cognitive Therapy Scale for Psychosis (CTS-Psy). Behavioural and Cognitive Psychotherapy 29, 221233.
Hayes, AF, Krippendorff, K (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures 1, 7789.
Inskipp, F, Proctor, B (1993). The Art, Craft and Tasks of Counselling Supervision, Part 1 – Making the Most of Supervision. Twickenham: Cascade Publications.
Inskipp, F, Proctor, B (1995). The Art, Craft and Tasks of Counselling Supervision, Part 2 – Becoming a Supervisor: Twickenham: Cascade Publications.
Jahoda, A, Willner, P, Rose, J, Kroese, BS, Lammie, C, Shead, J, Woodgate, C, Gillespie, D, Townson, J, Felce, D, Stimpson, A, Rose, N, MacMahon, P, Nuttall, J, Hood, K (2013). Development of a scale to measure fidelity to manualized group-based cognitive behavioural interventions for people with intellectual disabilities. Research in Developmental Disabilities 34, 42104221.
James, IA, Blackburn, I-M, Milne, D, Reichfelt, FK (2001). Moderators of trainee therapists’ competence in cognitive therapy. British Journal of Clinical Psychology 40, 131141.
Keen, AJ, Freeston, MH (2008). Assessing competence in cognitive-behavioural therapy. British Journal of Psychiatry 193, 6064.
Kolb, DA (1984). Experiential Learning: Experience as the Source of Learning and Development. New York: Prentice-Hall Englewood Cliffs.
Kuyken, W, Tsivrikos, D (2009). Therapist competence, comorbidity and cognitive-behavioral therapy for depression. Psychotherapy and Psychosomatics 78, 4248.
Landis, JR, Koch, GG (1977). The measurement of observer agreement for categorical data. Biometrics 33, 159174.
Lysaker, PH, Davis, LW, Bryson, GJ, Bell, MD (2009). Effects of cognitive behavioral therapy on work outcomes in vocational rehabilitation for participants with schizophrenia spectrum disorders. Schizophrenia Research 107, 186191.
Maissi, E, Ridge, K, Treasure, J, Chalder, T, Roche, S, Bartlett, J, Schmidt, U, Thomas, S, Ismail, K (2011). Nurse-led psychological interventions to improve diabetes control: assessing competencies. Patient Education and Counseling 84, e37e43.
McManus, F, Rakovshik, S, Kennerley, H, Fennell, M, Westbrook, D (2012). An investigation of the accuracy of therapists' self-assessment of cognitive-behaviour therapy skills. British Journal of Clinical Psychology 51, 292306.
Milne, D (2007). An empirical definition of clinical supervision. British Journal of Clinical Psychology 46, 437447.
Milne, D (2010). Can we enhance the training of clinical supervisors? A national pilot study of an evidence-based approach. Clinical Psychology & Psychotherapy 17, 321328.
Milne, D, Baker, C, Blackburn, I-M, James, I, Reichelt, K (1999). Effectiveness of cognitive therapy training. Journal of Behavior Therapy and Experimental Psychiatry 30, 8192.
Milne, D, Claydon, T, Blackburn, I-M, James, IA (2001). Rationale for a new measure of competence in therapy. Behavioural and Cognitive Psychotherapy 29, 2133.
Milne, DL, Reiser, R (2016). Supporting our supervisors: sending out an SOS. The Cognitive Behaviour Therapist. doi:10.1017/S1754470X15000616.
Muse, K, McManus, F (2013). A systematic review of methods for assessing competence in cognitive-behavioural therapy. Clinical Psychology Review 33, 484499.
Nichol, D, MacFarlane-Dick, D (2006). Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education 31, 199218.
O'Donovan, A, Kavanagh, DJ (2014). Measuring competence in supervisees and supervisors In: The Wiley International Handbook of Clinical Supervision, pp. 458467. Chichester: John Wiley & Sons, Ltd.
Reichelt, FK, James, IA, Blackburn, I-M (2003). Impact of training on rating competence in cognitive therapy. Journal of Behavior Therapy and Experimental Psychiatry 34, 8799.
Rosenbaum, M, Ronen, T (1998). Clinical supervision from the standpoint of cognitive-behavior therapy. Psychotherapy: Theory, Research, Practice, Training 35, 220230.
Roth, AD, Pilling, S (2008). A competence framework for the supervision of psychological therapies. Recuperado em 10, 2010.
Ryum, T, Stiles, TC, Svartberg, M, McCullough, L (2010). The effects of therapist competence in assigning homework in cognitive therapy with cluster C personality disorders: results from a randomized controlled trial. Cognitive and Behavioral Practice 17, 283289.
Shafran, R, Clark, DM, Fairburn, CG, Arntz, A, Barlow, DH, Ehlers, A, Freeston, M, Garety, PA, Hollon, SD, Ost, LG, Salkovskis, PM, Williams, JM, Wilson, GT (2009). Mind the gap: improving the dissemination of CBT. Behaviour Research and Therapy 47, 902909.
Sharpless, BA, Barber, JP (2009). A conceptual and empirical review of the meaning, measurement, development, and teaching of intervention competence in clinical psychology. Clinical Psychology Review 29, 4756.
Shaw, BF, Elkin, I, Yamaguchi, J, Olmsted, M, Vallis, TM, Dobson, KS, Lowry, A, Sotsky, SM, Watkins, JT, Imber, SD (1999). Therapist competence ratings in relation to clinical outcome in cognitive therapy of depression. Journal of Consulting and Clinical Psychology 67, 837846.
Simons, AD, Rozek, DC, Serrano, JL (2013). Wanted: reliable and valid measures for the science of cognitive behavioral therapy dissemination and implementation. Clinical Psychology: Science and Practice 20, 181194.
Stallard, P, Myles, P, Branson, A (2014). The Cognitive Behaviour Therapy Scale for Children and Young People (CBTS-CYP): development and psychometric properties. Behavioural and Cognitive Psychotherapy 42, 269282.
Strunk, DR, Brotman, MA, DeRubeis, RJ (2010). The process of change in cognitive therapy for depression: predictors of early inter-session symptom gains. Behaviour Research and Therapy 48, 599606.
Trepka, C, Rees, A, Shapiro, D, Hardy, G, Barkham, M (2004). Therapist competence and outcome of cognitive therapy for depression. Cognitive Therapy and Research 28, 143157.
Tversky, A, Kahneman, D (1974). Judgment under uncertainty: heuristics and biases. Science 185, 11241131.
Vallis, TM, Shaw, BF, Dobson, KS (1986). The Cognitive Therapy Scale: psychometric properties. Journal of Consulting and Clinical Psychology 54, 381385.
von Consbruch, K, Clark, DM, Stangier, U (2012). Assessing therapeutic competence in cognitive therapy for social phobia: psychometric properties of the cognitive therapy competence scale for social phobia (CTCS-SP). Behavioural and Cognitive Psychotherapy 40, 149161.
Watkins, CE, Wang, CDC (2014). On the education of clinical supervisors The Wiley International Handbook of Clinical Supervision (pp. 177203): John Wiley & Sons, Ltd.
Weck, F, Grikscheit, F, Hofling, V, Stangier, U (2014). Assessing treatment integrity in cognitive-behavioral therapy: comparing session segments with entire sessions. Behavior Therapy 45, 541552.
Williams, RM, Moorey, S, Cobb, J (1991). Training in cognitive-behaviour therapy: pilot evaluation of a training course using the Cognitive Therapy Scale. Behavioural Psychotherapy 19, 373376.
Wittorf, A, Jakobi-Malterre, UE, Beulen, S, Bechdolf, A, Müller, BW, Sartory, G, Wagner, M, Wiedemann, G, Wölwer, W, Herrlich, J, Klingberg, S (2013). Associations between therapy skills and patient experiences of change processes in cognitive behavioral therapy for psychosis. Psychiatry Research 210, 702709.
Young, J, Beck, AT (1980). Cognitive Therapy Scale: Rating manual (unpublished manuscript).

Keywords

The challenge of training supervisors to use direct assessments of clinical competence in CBT consistently: a systematic review and exploratory training study

  • Maria E. Loades (a1) and Peter Armstrong (a2)

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed

The challenge of training supervisors to use direct assessments of clinical competence in CBT consistently: a systematic review and exploratory training study

  • Maria E. Loades (a1) and Peter Armstrong (a2)
Submit a response

Comments

No Comments have been published for this article.

×

Reply to: Submit a response


Your details


Conflicting interests

Do you have any conflicting interests? *