In the last two decades, many studies have proved the benefit and effectiveness of a multi-element psychosocial approach provided for severe psychiatric disorders like psychosis, especially in the early phases of the illness (Bird et al., Reference Bird, Premkumar, Kendall, Whittington, Mitchell and Kuipers2010; Birchwood et al., Reference Birchwood, Todd and Jackson1998; Dixon et al., Reference Dixon, Dickerson, Bellack, Bennett, Dickinson and Goldberg2010; Dunn et al., Reference Dunn, Fowler, Rollinson, Freeman, Kuipers and Smith2012; Edwards et al., Reference Edwards, Harris and Bapat2005; Kuipers et al., Reference Kuipers, Garety, Fowler, Dunn, Bebbington and Freeman1997; Lecomte et al., Reference Lecomte, Leclerc and Wykes2012; Morrison et al., Reference Morrison, Renton, Williams, Dunn, Knight, Kreutz, Nothard, Patel and Dunn2004; Penn et al., Reference Penn, Waldheter, Perkins, Mueser and Lieberman2005; Ruggeri et al., Reference Ruggeri, Bonetto, Lasalvia, Fioritti, De Girolamo and Santonastaso2015; Sarin et al., Reference Sarin, Wallin and Widerlöv2011; Tarrier et al., Reference Tarrier, Yusupoff, Kinney, McCarthy, Gledhill and Haddock1998; Wykes et al., Reference Wykes, Steel, Everitt and Tarrier2008). According to international guidelines [National Institute for Clinical Excellence (NICE), 2014], optimal intervention for early psychosis should include adequate anti-psychotic medication combined with individual structured psychological intervention. Specifically, cognitive behavioalur therapy for early psychosis (CBTep) is indicated as a first-line psychological treatment for patients at the onset of psychosis (NICE, 2003, 2009, 2014). The importance of professional competence in delivering CBTep has been consistently highlighted in the literature (Jolley et al., Reference Jolley, Onwumere, Kuipers, Craig, Moriarty and Garety2012, Reference Jolley, Onwumere, Bissoli, Bahayani, Singh and Kuipers2015; NICE, 2009, 2014). In particular, providing mental health staff with advanced and specific skills on assessment and therapeutic techniques should be guaranteed for the effectiveness of the intervention (Fowler et al., Reference Fowler, Garety and Kuipers1995; Rollinson et al., Reference Rollinson, Smith, Steel, Jolley, Onwumere and Garety2008; Ruggeri et al., Reference Ruggeri, Lora and Semisa2008).
While the efficacy of CBT in reducing re-hospitalization and distress associated with both positive and negative symptoms has been recognized (Lecomte et al., Reference Lecomte, Leclerc and Wykes2012; NICE, 2003, 2009, 2014), few published studies have specifically investigated the feasibility and efficacy of a training programme on CBT for early psychosis provided to professionals who work daily with patients at the first period of the illness (Beidas and Kendall, Reference Beidas and Kendall2010; Rakovshik and McManus, Reference Rakovshik and McManus2010). According to this perspective, Beidas and Kendall (Reference Beidas and Kendall2010) underline the importance of evaluating the impact of contextual variables of the training, such as the organizational support, the quality of the programme, and client and therapist variables. Some existing studies (Jolley et al., Reference Jolley, Onwumere, Bissoli, Bahayani, Singh and Kuipers2015; Waller et al., Reference Waller, Garety, Jolley, Fornells-Ambrojo, Kuipers, Onwumere, Woodall, Emsley and Craig2013, Reference Waller, Emsley, Freeman, Bebbington, Dunn and Fowler2015) explored the efficacy and the feasibility of cognitive behavioural therapy for psychosis (CBTp) training for mental health staff. However, in these studies, staff were not specifically trained on early psychosis but more generally on psychotic patients, and the number of staff members involved was exiguous.
Therefore, in this study we wanted to address the shortcomings of the current evidence by evaluating the delivery of a CBTep training to staff working in different Italian Community Mental Health Centres (CMHCs).
The present work can be considered in line with the larger framework of the movement called ‘Capacity Building’, which is a worldwide movement with the aim to fill ‘the mental health treatment gap’ (the increasingly evident mismatch of mental health disease burden with the extant resources; Saxena et al., Reference Saxena, Thornicroft, Knapp and Whiteford2007) and to equip the local workforce to deliver high-quality, innovative, locally relevant, feasible and effective interventions (Becker and Kleinman, Reference Becker and Kleinman2012; NICE, 2009), also in low- and middle-income countries (Fricchione et al., Reference Fricchione, Borba, Alem, Shibre, Carney and Henderson2012; Thornicroft et al., Reference Thornicroft, Cooper, Bortel, Kakuma and Lund2012).
Specifically, the aims of the present study were: (i) to evaluate the feasibility and efficacy of a training programme developed in order to build specific expertise on CBT for early psychosis for staff working in CMHCs, and (ii) to investigate the possible association between the characteristics of the professionals and the competence acquisition.
The present study was conducted in the context of the ‘Psychosis early Intervention and Assessment of Needs and Outcome’ (PIANO) randomized clinical trial.
The PIANO study was part of a more general research project named GET UP (Genetics, Endophenotypes, and Treatment: Understanding early Psychosis), which was a large, multi-element research programme involving 117 CMHCs in Northern and Central Italy, covering a catchment population of approximately 10 million inhabitants.
The primary objectives of the PIANO were to compare the 9-month effectiveness of a multi-component psychosocial intervention versus ‘treatment as usual’ (TAU) for patients with first-episode psychosis (FEP) and their family members recruited from Italian CMHCs. CHMCs randomized in the experimental arm of the trial offered, in addition to the treatment as usual, CBTep for patients, family intervention and case management (Ruggeri et al., Reference Ruggeri, Bonetto, Lasalvia, De Girolamo, Fioritti and Rucci2012, Reference Ruggeri, Bonetto, Lasalvia, Fioritti, De Girolamo and Santonastaso2015). The PIANO study was a pragmatic cluster-randomized controlled trial, with the CMHCs as randomization clusters. In detail, 117 CMHCs were enrolled in the project; of these, 96 CMHCs entered in the randomization procedure because 32 small CMHCs were paired and therefore considered as 16 CMHCs. Forty-eight were randomized in the TAU arm and 48 in the experimental arm. Before the start of the trial, local mental health professionals in the experimental arm of the study received a specific training programme in CBTep, which represents the focus of the present study.
The CBTep training programme was delivered to psychiatrists and psychologists who worked in the 65 Italian CMHCs included in the experimental arm of the PIANO randomized clinical trial. At least two psychiatrists/psychologists per experimental CMHC took part in the training as they are the only professional figures allowed to provide any form of psychotherapy in Italy. Trainees were permanent salaried staff of the CMHC so they were not paid specifically for the delivery of the CBTep intervention.
The only selection requirement for participation in the training was that staff members should have worked for at least 5 years in clinical settings with people with psychosis. Expert psychologists with a specific 4-year CBT masters level qualification also joined the training course to support CMHC staff members in the use of CBTep within the PIANO trial. Expert psychologists also represented the comparison arm in order to compare the specific CBTep skills they developed during the training course with those acquired by CMHC staff members.
The training course was an innovative, focused and graded CBTep training developed to train local CMHC professionals to deliver CBTep in accordance with the NICE guidelines. It was developed by the staff members of the Italian Center for Early Psychosis ‘Programma 2000’, who had been working in this field for more than 10 years (Cocchi et al., Reference Cocchi, Meneghelli and Preti2008; Meneghelli et al., Reference Meneghelli, Cocchi and Preti2010). The training programme was included in a postgraduate course at Verona University and teachers involved were national and international experts in the field of the psychosis onset. The training was planned and organized on three main levels: teaching modules, practical training and supervision.
Specifically, the course started in June 2009 and finished in December of the same year and consisted of 112 hours of teaching (divided into four modules of 3 or 4 days), 30 hours of small group supervision and 3 months of practical training that included exercises, videos and role-playing.
The contents were conceived starting from the bases of the CBTep model developed by Kuipers and colleagues (Reference Kuipers, Fowler, Garety, Chisholm, Freeman and Dunn1998), Garety and colleagues (Reference Garety, Fowler, Freeman, Bebbington, Dunn and Kuipers2008) and Fowler and colleagues (Reference Fowler, Garety and Kuipers1995), whose efficacy was demonstrated in previous randomized clinical trials (Fowler et al., Reference Fowler, Garety and Kuipers1995; Garety et al., Reference Garety, Fowler, Freeman, Bebbington, Dunn and Kuipers2008; Kuipers et al., Reference Kuipers, Fowler, Garety, Chisholm, Freeman and Dunn1998; Turkington et al., Reference Turkington, Kingdon, Rathod, Hammond, Pelton and Mehta2006). Table 1 reports the contents of the CBTep model used in this project, divided into four different modules.
A detailed CBTep intervention manual was developed according to the content covered in the course and by taking into account the critical aspects raised by the staff members. The intervention manual was provided to each participant to guide the use of CBT for early psychosis in clinical practice. Finally, after the course, supervision groups were carried out for the entire duration of the GET UP programme, in order to monitor the use of CBT and to maintain the skills acquired by the clinicians involved.
Assessments and measures
At the end of each module and end of the entire course, all participants (both CMHC staff members and CBT expert psychologists) were evaluated on acquired CBTep theoretical knowledge and practical skills.
The measures of training feasibility were: staff member drop-out, reasons for staff member drop-out, and the percentage of sessions attended.
The measures of training efficacy were: the written examination scores obtained after each course module, including theoretical and clinical CBT skills (maximum score: 40) and the supervisor’s judgement (maximum score: 10) – see the Supervisor assessment schedule in Appendix 1 of the Supplementary material. The final overall score to pass the course was 35/50.
Examination sessions included theoretical examination and case reports for all modules, except for the examination session of module 2 that included only questionnaires. Independent blinded expert trainers evaluated the participants’ results by assigning a score on a scale from 0 to 50.
Theoretical examinations consisted of multiple-choice questionnaires concerning the conceptual contents of the training. Case reports included case analysis, formulation and treatment strategies. In particular, experts assessed each participant’s level of knowledge of CBT theory and their skills to identify clinical problems, conceptualize the case following the CBT model (Fowler et al., Reference Fowler, Garety and Kuipers1995; Garety et al., Reference Garety, Fowler, Freeman, Bebbington, Dunn and Kuipers2008; Kuipers et al., Reference Kuipers, Fowler, Garety, Chisholm, Freeman and Dunn1998), recognize relevant goals and carry out a treatment programme.
Moreover, supervisors expert in CBTep at the end of the practical training and during the supervision period assessed each participant’s capacity building with regards to ‘Participation’, ‘Attitude’, ‘Therapeutic style’ and ‘Congruency with CBT contents’, by using the Supervisor Assessment Schedule (see Appendix 1 in Supplementary material). In order to avoid differences in the assessment style, supervisors were selected on purpose from the Programma 2000 team as they all shared a consistent theoretical and clinical approach and had long experience in working together on FEP cases within a comprehensive multi-modal protocol of early intervention in psychosis (Cocchi et al., Reference Cocchi, Meneghelli and Preti2008; Meneghelli et al., Reference Meneghelli, Cocchi and Preti2010).
To pass the final exam of the course, participants had to achieve a minimum total score of 35/50 at the end of the training. This score was made up by the combination of the final exam score and supervisory judgement. Lastly, participants’ course satisfaction was evaluated by using an ad hoc questionnaire for each training session on a score from 0 to 10.
At the end of the training, CBTep intervention manuals per international standards were given to the staff members as treatment references. During the 9 months of the trial, staff members were supported by CBT expert psychotherapists assigned to each CMHC and written reports of each session were produced (see the Schematic Report Schedule in Appendix 1). They were also supervised by external supervisors expert in CBTep who held 1-day meetings every 2 months, and were regularly available for consultation to ensure fidelity of the intervention. A random sample of sessions was audio-recorded to allow further fidelity measurement by independent raters.
The whole sample of participants was divided into two groups: group 1: CMHC staff members (psychiatrists and psychologists) and group 2: CBT expert psychologists, as a comparison group. Comparison was executed by t-test for independent groups (continuous variables) and Fisher’s exact test (categorical variables). All tests were bilateral at p = <0.01 (due to multiple testing). Data were analysed using Statistical Package for the Social Sciences (SPSS).
To test the role of demographic and background characteristics of CMHC staff members on competence acquisition, participants were dichotomized as follows: age <40 vs ≥40; years from graduation <10 vs ≥10; years from qualification test <10 vs ≥10; years of work experience <10 vs ≥10; theoretical orientation: CBT No vs Yes; and professional background: psychiatrists vs psychologists. Due to the descriptive nature of the study, no correction for multiple testing was applied and a conservative p-value was chosen (p < 0.01).
One hundred and five CMHCs staff members were invited to join the course. Eight of them dropped out before the beginning of the course (four moved to other services and four for other reasons) and one after the beginning of the course (because of serious family health reasons), leading to a drop-out rate of 8.6%. Therefore, a total of 96 CMHCs staff members (group 1) and 24 CBT expert psychologists (group 2) attended and completed the course. The sample had a mean age of 41.47 years (SD = 8.16), and there was a preponderance of females (68.3%). Participants attended 93% of the course (minimum frequency required 70%, corresponding to 104 hours). The majority of the CMHC staff members were psychiatrists (75%), with no previous experience in CBTep and with a mean of 11.78 years (SD = 7.52) of work in CMHC. The CBT experts were all psychologists, on average younger [mean age: 33.21 years (SD = 3.43) vs 43.53 years (SD = 7.69), respectively; p < .001] and with less working experience in mental health clinical setting compared with CMHC professionals (mean years of work: 2.32 (SD = 2.67) vs 11.78 (SD = 7.52), respectively; p < .001). Among the 24 CBT expert psychologists, 21 had completed a specific 4-year Masters level in CBT and three had attended specific training courses with well-documented experience in CBT for psychosis. Table 2 shows in detail the demographic and background characteristics of the participants.
Theoretical capacity building
Overall, all trainees passed each examination session positively, and at the end of the training most of them (80.21% of the CMHC staff members and 95.65% of the CBT expert psychologists) reached medium–high total scores level (41–50/50) (see Table 3).
The data show that there was no significant difference between the two groups both in ongoing exam scores and in the final exam total score [group 1 = 34.94 (SD = 2.04) vs group 2 = 36.08 (SD=2.45), p = .020]; only in the ongoing exam of the training session 2 did we find a significant difference in favour of group 2 (p = 0.002) (see Table 4; Fig. 1).
Clinical CBT capacity building
Considering supervisors’ judgements, we found a difference between the two groups in the final total score [total score: group 1 = 7.71 (SD = 1.23) vs group 2 = 9.30 (SD = 0.82), p < .001] and in two of the four evaluation areas: ‘Therapeutic style’ [group 1 = 1.39 (SD = 0.50) vs group 2 = 1.91 (SD = 0.29), p < .001] and ‘Congruency with CBT’ [group 1 = 1.83 (SD = 0.65) vs group 2 = 2.63 (SD = 0.58), p < .001] (see Table 5; Fig. 2). However, the score difference was small, suggesting no major difference between the capacity acquired by the participants of the two groups.
After the training, the number of sessions delivered by the trained staff members during the 9 months has been recorded. In total, 272 patients were enrolled in CBTep. Out of the 96 staff members trained, 74 were actively working with the patients. A mean number of 65.4 sessions for each trained professional was done.
Impact of sociodemographic characteristics in CMHC staff members group
Descriptive analysis does not show a significant difference on the basis of age, years from graduation, years of work, and theoretical orientation, in training sessions 1 (fundamental principles of CBT) and 2 (theoretical basis of CBT for psychosis), both in the ongoing exam scores and in the final exam total scores. In training session 3 (specific CBT for first episode psychosis), we found a significant difference in favour of younger participants, with fewer years from graduation and less work experience; see Table 6). We detected some minor differences in training session 3 among professionals with different backgrounds. In detail, psychologists showed slightly higher scores than other professionals. Interestingly, previous knowledge in CBT did not strongly affect the outcome (Table 6).
Satisfaction with the course
The course was well received by trainee CMHC staff, with high scores on course satisfaction [minimum 0; maximum 10; mean 7.81, SD 1.14).
The aim of this study was to evaluate the feasibility and efficacy of a brief and intensive CBTep training provided to mental health professionals of CMHCs.
The importance of providing this type of specific training to professionals has been pointed out by the NICE guidelines: ‘Trusts should provide access to training that equips healthcare professionals with the competencies required to deliver the psychological therapy interventions recommended in this guideline’ (NICE 2009, 2014).
As mentioned in the introduction, many studies evaluated the efficacy of CBT treatment for patients with psychosis or schizophrenia, but few described the results of a specific training for staff. For instance, Rakovshik and McManus (Reference Rakovshik and McManus2010) conducted a review where they included 34 trials on CBT, emphasizing that only seven studies were specifically focused on training efficacy and underlining that an ‘extensive’ (more than 137 hours) and ‘graded’ training (in which the stages of therapist involvement progressed as competence developed) seems to increase training effectiveness, although having the drawback of being costly and often impractical. Furthermore, experiential and interactive approaches using clinical cases, co-therapy and supervision added to traditional type of training (workshop, reading, web-based instruction), are important to maintain competency over time.
Recently, Jolley and colleagues (Reference Jolley, Onwumere, Bissoli, Bahayani, Singh and Kuipers2015) ran a pilot evaluation of therapist training in a CBT programme for psychosis targeted to a small number of professionals (n = 9) working in mental health services. They found that high-quality training in CBT had a beneficial effect on clinical outcomes of patients. In contrast to our study, they trained different-type mental health professionals (nurses, occupational therapists, clinical psychologists), and directed the therapy to patients in different phases of the psychosis illness.
The training course used in our study has been organized according to the main requirements that have been proven to be associated with better outcomes (Jolley et al., Reference Jolley, Onwumere, Bissoli, Bahayani, Singh and Kuipers2015), such as graded approach, extensive duration, interactive training and sustained supervision. Concerning the quality of training, the content and method used, we included the best evidence in this field according to the NICE guidelines (CBTep and interactive learning); similarly, organizational support was considered, with sustained supervision to facilitate the use of CBTep in routine clinical practice. The low drop-out rate and the successful completion of the course achieved by the majority of participants (120/129), along with the high frequency (104.46 hours; 93% vs 70% required) and the good performance (medium–high scores obtained by 80.21% of CMHC staff members), suggest the feasibility of the CBTep training programme.
Concerning the efficacy, the similar scores reported in the ‘ongoing’ and ‘final’ exam tests in the two groups (CMHC staff members vs CBT expert psychologists) support the efficacy of the course in terms of development of theoretical knowledge and competence. Furthermore, the sensitivity of the assessment method to test the capacity building of this brief and intensive course is proved by the difference detected between CMHC professionals and expert psychologists in the supervisor judgement, which probably reflects the effect of specific competences previously acquired by CBT expert psychologists, who already had a CBT Masters qualification or CBT-specific clinical experience. Interestingly, however, this difference is small and only concerned some evaluations such as ‘Congruency with CBT’ and ‘Therapeutic style’, which are probably more sensitive to previous specific training. Overall, these results are in line with those of Jolley et al. (Reference Jolley, Onwumere, Bissoli, Bahayani, Singh and Kuipers2015), which proved that it is possible to train mental health staff in the field of CBTep to a high standard of competence, despite different professional backgrounds.
We found differences in the impact of other trainee characteristics (age, years from graduation, years of work, theoretical orientation and professional background) on competence acquisition only in the most specific part of the training (module 3), with a better performance of younger CMHC professionals. This observation, i.e. impact of age, years from graduation, years of work and theoretical orientation, is consistent with the reports of some previous studies (Brosan et al., Reference Brosan, Reynolds and Moore2006; Sholomskas et al., Reference Sholomskas, Syracuse-Siewert, Rounsaville, Ball, Nuro and Carroll2005) but in contrast with others (James et al., Reference James, Blackburn, Milne and Reichfelt2001; Manassis et al., Reference Manassis, Ickowicz, Picard, Antle, McNeill and Chahauver2009; McManus and Westbrook, Reference McManus and Westbrook2010). This controversy in literature may be due to differences in methodology, settings and countries of the training, suggesting that the effects of such sociodemographic variables should be better addressed in future multi-sites studies.
Our findings, in conclusion, seem to confirm the possibility of delivering specific knowledge through a relatively short and intensive training programme to qualified mental health professionals, with no (or non-specific) previous knowledge in CBT.
Limitations and recommendations for future research
The CBTep training course presented here is the first attempt, conducted on a large scale, to build high-quality capacity in CBTep in CMHC staff working in a clinical practice setting, through a brief and intensive course. The results achieved by CMHC staff members, whose capacity building did not remarkably differ from CBT expert psychologists, seem to confirm not only the feasibility but also the efficacy of the course.
However, our results should be interpreted also in the light of some limitations. Firstly, course participants did not complete a baseline assessment, and therefore it is not possible to estimate the relative improvement in CBT specific skills from baseline, as only the number of working years in a CMHS was collected. However, if we consider that the baseline level of CBTep knowledge and skills of CBT expert psychologists were potentially higher than those of CHMC staff members, and that the final results observed in the two groups were not meaningfully different, the good performance achieved by CHMC staff members at the final examination is even more notable. Secondly, we did not perform a cost-efficacy analysis of the course, which should be a point to address in future research. Finally, as we did not use a standardized method to assess inter-rater reliability of supervisor judgements and nor did we use a blinding procedure in the assessment of the trainee by the supervisors, we cannot exclude a possible bias of their ratings.
This paper opens the way to future directions, especially in terms of implementation. After having passed the course examination, the staff members of CMHCs provided the CBTep in their local service during the 9 months of the PIANO study. The results and outcomes of the study have been reported in the paper from Ruggeri et al. (Reference Ruggeri, Bonetto, Lasalvia, Fioritti, De Girolamo and Santonastaso2015). Data proved the efficacy of CBTep psychotherapy in reductions in overall symptom severity, while no difference could be found for days of hospitalization (the second primary outcome). Improvements were also found for global functioning, subjective well-being and subjective burden of delusions. In particular, the patient subgroup of the experimental arm with more severe psychotic symptoms, showed a significant reduction in the subjective appraisal of delusions (emotional and cognitive components), which is a specific focus of the CBTep intervention. However, it must be said that as the trial design was based on a multi-element intervention (including – beside CBTep – also family intervention and case management), we could not disentangle the specific impact of the CBTep.
After the trial, many and different initiatives have been adopted by participating CMHCs in terms of clinical and training experiences arising from the GET-UP project. However, the implementation of the CBTep training for CMHC staff outside the trial context is still to be explored in a systematic way. As such, future investigations in this direction should centre on two main aspects. The first concerns the exploration of the possible implementation of the training as part of the routine clinical practice of services and the potential difficulties for its implementation. Barriers to the implementation of training that equips professionals with the competencies required to deliver high-specialist interventions may be fuelled by the lack of resources in terms of organizational factors (staff availability) and system factors (allocation of funding). This may be particularly true in the context of Italian generalist mental health services, which have fewer human resources for mental health care compared with other high-income countries, especially in terms of clinical psychologists working in the public mental health sector (Barbui et al., Reference Barbui, Papola and Saraceno2018). The second aspect concerns the impact of CBTep training course on long-term outcomes, including the maintenance of the acquired skills over time and the correct use of CBTep in clinical practice outside the trial.
To view supplementary material for this article, please visit https://doi.org/10.1017/S1754470X20000355
We are grateful to the professionals involved in the study who voluntarily participated.
The present study was included on a more general research project named the GET UP (Genetics, Endophenotypes, and Treatment: Understanding early Psychosis) Program. The financial support of this project was provided by Ministry of Health, Italy – Ricerca Sanitaria Finalizzata, code H61J08000200001.
Conflicts of interest
The authors declare that there are no conflicts of interest in relation to the subject of this study.
The Ethical Principles of Psychologists and Code of Conduct have been complied with. Ethical approval was not required because the sample was not composed of patients.
Key practice points
(1) The training course presented in this study proved feasible to deliver, and efficient in building CBTep specific skills in staff of CMHCs.
(2) Interactive training with a graded approach, intensive duration and sustained supervision are the crucial characteristics of the course.
(3) Trainees’ sociodemographic characteristics and professional background may only marginally affect the outcome.