Skip to main content Accessibility help

The CORE service improvement programme for mental health crisis resolution teams: results from a cluster-randomised trial

Published online by Cambridge University Press:  14 February 2019

Brynmor Lloyd-Evans
Senior Lecturer, Division of Psychiatry, University College London, UK
David Osborn
Professor of Psychiatric Epidemiology, Division of Psychiatry, University College London; and St Pancras Hospital, Camden and Islington NHS Foundation Trust, UK
Louise Marston
Principal Research Associate, Department of Primary Care and Population Health, UCL Medical School (Royal Free Campus); and Priment Clinical Trials Unit, UK
Danielle Lamb
Research Associate, Division of Psychiatry, University College London, UK
Gareth Ambler
Associate Professor in Medical Statistics, Department of Statistical Science, UCL, UK
Rachael Hunter
Senior Research Associate, Department of Primary Care and Population Health, UCL Medical School (Royal Free Campus); and Priment Clinical Trials Unit, UK
Oliver Mason
Reader in Clinical Psychology, School of Psychology, University of Surrey, UK
Sarah Sullivan
Research Fellow, Epidemiology and Health Services Research, CLAHRC West, UK
Claire Henderson
Clinical Senior Lecturer, Institute of Psychiatry, Psychology and Neuroscience, Kings College London, UK
Steve Onyett
previously Chief Executive, Onyett Entero, UK
Elaine Johnston
Principal Research Associate, Division of Psychiatry, University College London, UK
Nicola Morant
Associate Professor, Division of Psychiatry, University College London, UK
Fiona Nolan
Professor of Nursing, School of Health and Human Sciences, University of Essex, UK
Kathleen Kelly
Consultant Psychiatrist, Barnes Unit, John Radcliffe Hospital, Oxford Health NHS Foundation Trust, UK
Marina Christoforou
Research Assistant, Division of Psychiatry, University College London, UK
Kate Fullarton
Research Assistant, Division of Psychiatry, University College London, UK
Rebecca Forsyth
Research Assistant, Division of Psychiatry, University College London, UK
Mike Davidson
Research Assistant, Division of Psychiatry, University College London, UK
Jonathan Piotrowski
Research Assistant, Avon and Wiltshire Mental Health Partnership NHS Trust, Research & Development Office, UK
Edward Mundy
Research Assistant, Division of Psychiatry, University College London, UK
Gary Bond
Professor of Psychiatry, Westat, Rivermill Commercial Center, UK
Sonia Johnson
Professor of Social and Community Psychiatry, Division of Psychiatry, University College London; and St Pancras Hospital, Camden and Islington NHS Foundation Trust, UK
E-mail address:



Crisis resolution teams (CRTs) offer brief, intensive home treatment for people experiencing mental health crisis. CRT implementation is highly variable; positive trial outcomes have not been reproduced in scaled-up CRT care.


To evaluate a 1-year programme to improve CRTs’ model fidelity in a non-masked, cluster-randomised trial (part of the Crisis team Optimisation and RElapse prevention (CORE) research programme, trial registration number: ISRCTN47185233).


Fifteen CRTs in England received an intervention, informed by the US Implementing Evidence-Based Practice project, involving support from a CRT facilitator, online implementation resources and regular team fidelity reviews. Ten control CRTs received no additional support. The primary outcome was patient satisfaction, measured by the Client Satisfaction Questionnaire (CSQ-8), completed by 15 patients per team at CRT discharge (n = 375). Secondary outcomes: CRT model fidelity, continuity of care, staff well-being, in-patient admissions and bed use and CRT readmissions were also evaluated.


All CRTs were retained in the trial. Median follow-up CSQ-8 score was 28 in each group: the adjusted average in the intervention group was higher than in the control group by 0.97 (95% CI −1.02 to 2.97) but this was not significant (P = 0.34). There were fewer in-patient admissions, lower in-patient bed use and better staff psychological health in intervention teams. Model fidelity rose in most intervention teams and was significantly higher than in control teams at follow-up. There were no significant effects for other outcomes.


The CRT service improvement programme did not achieve its primary aim of improving patient satisfaction. It showed some promise in improving CRT model fidelity and reducing acute in-patient admissions.

Creative Commons
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (, which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright © The Royal College of Psychiatrists 2019

Crisis resolution teams (CRTs) are specialist, multidisciplinary mental health teams that provide short-term, intensive home treatment as an alternative to acute hospital admission.1 CRTs were implemented nationally in England following the National Health Service (NHS) Plan2 in 2000 and have been established elsewhere in Europe and in Australasia.Reference Johnson3 Trial evidence suggests CRTs can reduce in-patient admissions and improve patients' satisfaction with acute care.Reference Johnson, Nolan, Pilling, Sandor, Hoult and McKenzie4, Reference Murphy, Irving, Adams and Waqar5 When CRTs were scaled-up to national level in England, however, patients reported dissatisfaction with the quality of careReference Hopkins and Niemiec6, 7 and CRTs' impact on admission rates has been disappointing.Reference Jacobs and Barrenho8 English CRTs' service organisation and delivery is highly variable and adherence to national policy guidance is only partial.Reference Lloyd-Evans, Paterson, Onyett, Brown, Istead and Gray9, Reference Lloyd-Evans, Lamb, Barnby, Eskinazi, Turner and Johnson10 In initial stages of the Crisis team Optimisation and RElapse prevention (CORE) research programme (as part of which the trial reported in this paper was conducted, trial registration number: ISRCTN47185233), a measure of model fidelity for CRTsReference Lloyd-Evans, Bond, Ruud, Ivanecka, Gray and Osborn11 and a service improvement programme for CRTsReference Lloyd-Evans, Fullarton, Lamb, Johnston, Onyett and Osborn12 were developed. This followed a model for supporting the implementation of mental health complex interventions widely and successfully used in the USAReference McHugo, Drake, Whitley, Bond, Campbell and Rapp13 (but not so far in Europe). The trial reported in this paper evaluated whether the CORE CRT service improvement programme increased model fidelity and improved outcomes in CRT teams over a 1-year intervention period.


A non-blind cluster-randomised trial evaluated whether a CRT service improvement programme improved patients' experience of CRT care, reduced acute service use and improved CRT staff well-being. The trial also investigated whether the Fidelity Scale scores of CRTs receiving the service improvement programme increased over the 1-year intervention period compared with control sites, and whether change in team Fidelity Scale score was associated with change in service outcomes. Cluster randomisation was used because the trial involved a team-level intervention. The primary hypothesis was that patients' satisfaction with CRT care, measured using the Client Satisfaction Questionnaire (8-item version, CSQ-8)Reference Atkisson and Zwick14 was greater in the intervention group than the control group at end-of-intervention 1-year follow-up.

The trial was registered with the ISRCTN registry (ISRCTN47185233) in August 2014 and the protocol has been published.Reference Lloyd-Evans, Fullarton, Lamb, Johnston, Onyett and Osborn12 This paper reports the main trial results and relationships between teams' model fidelity and outcomes. Economic and process, and qualitative evaluations will be reported elsewhere. Ethical approval was granted by the Camden & Islington Research Ethics Committee (Ref: 14/LO/0107). Trial reporting in this paper conforms to extended CONSORT guidelines for cluster-randomised trialsReference Campbell, Piaggio, Elbourne and Altman15 – see supplementary data 1 available at Teams were recruited to the study between September and December 2014, with a 1-year trial intervention period.


The trial involved 25 CRT teams in eight different health regions (NHS trusts) across Southern England and the Midlands, selected to include inner-city and mixed suburban and rural areas. CRT teams were eligible if no other major service reorganisations were planned over the trial period. At least two participating CRTs were required from each NHS trust, to ensure teams from each trust could be allocated to each group.


We recruited: (a) CRT patients, (b) CRT staff, and (c) anonymised data about use of acute care from service records.

  1. (a) Patient experience outcomes: we aimed to recruit a cohort of recently discharged CRT patients: 15 per team each team (n = 375) at baseline, and another cohort, 15 per team (n = 375) at follow-up (between months 10 and 12 of the study period). All participants admitted to the CRT during these 3-month periods were eligible if they: had used the CRT service for at least 7 days; had ability to read and understand English and capacity to provide informed consent; and were not assessed by CRT clinical staff to pose too high a risk to others to participate (even via interview on NHS premises, or by phone).

  2. (b) Staff well-being outcomes: all current staff in participating CRTs were invited to complete study questionnaires at baseline and follow-up (months 10–12 of the study intervention period).

  3. (c) Patient-record data were collected for two separate cohorts at each time point: (i) anonymised data about all admissions to in-patient services were collected retrospectively from services' electronic patient records at two time points: 6 months prior to study baseline, and months 7–12 of the study intervention period (in-patient service-use outcomes); (ii) for all patients admitted to the CRT during two 1-month periods, anonymised data about readmissions to any acute care service (including CRTs or in-patient wards) over a 6-month period were collected retrospectively from electronic patient records: the first cohort at 6 months prior to baseline; the second at month 7 of the study follow-up period (readmission following CRT care outcome).

Randomisation and masking

The 25 teams were randomised, stratified by NHS trust, after a baseline fidelity review had been conducted for all teams within each NHS trust. In order to maximise learning about implementation of the service improvement programme within available study resources, more teams (n = 15) were randomly allocated to the service improvement programme than to a control group (n = 10). A statistician independent of the study generated allocation sequence lists and conducted randomisations. Researchers and staff in participating services were aware of teams' allocation status in this non-masked trial. Patient-participants and trusts' informatics teams providing data from patient records were not informed of teams' allocation status.

The intervention

The team-level service improvement programme supported CRT teams to achieve high fidelity to a model of best practice, defined in the CORE CRT Fidelity ScaleReference McHugo, Drake, Whitley, Bond, Campbell and Rapp13 and informed by a systematic evidence review,Reference Wheeler, Lloyd-Evans, Churchard, Fitzgerald, Fullarton and Mosse16 a national CRT surveyReference Lloyd-Evans, Paterson, Onyett, Brown, Istead and Gray9 and qualitative interviews with stakeholders.Reference Morant, Lloyd-Evans, Lamb, Fullarton, Brown and Paterson17 The service improvement programme was delivered over 1 year and consisted of: (a) ‘fidelity reviews’ at baseline, 6 months and 12 months: teams were assessed and given feedback on adherence to 39 best practice standards for CRTs; (b) coaching from a CRT facilitator (an experienced clinician or manager) 0.1 full-time equivalent, who could offer the CRT manager and staff advice and support with developing and implementing service improvement plans; (c) access to an online resource kit of materials and guidance to support CRT service improvement for each fidelity item; and (d) access to two ‘learning collaborative’ events where participating teams could meet to share experiences and strategies for improving services. Structures to support service improvement in each team included: an initial ‘scoping day’ for the whole team to prioritise and plan service improvement goals; and regular meetings of a CRT management group and the CRT facilitator, to develop and review detailed service improvement plans.

Through these resources and structures, the service improvement programme constituted a sustained, multicomponent programme of support to CRTs, which aimed to address the different domains of implementation support identified as contributing to attainment of high fidelity in the US Implementing Evidence-Based Practices ProgramReference Torrey, Bond, McHugo and Swain18 in a tailored way to meet individual services' needs: prioritisation of the programme, leadership support, workforce development; workflow re-engineering; and practice reinforcement.

Teams in the control group received a fidelity review and brief feedback at baseline and 12-month follow-up, but no other aspects of the study intervention.


Patient experience outcomes

Data were collected using two self-report structured questionnaires: the CSQ-8,Reference Atkisson and Zwick14 which assesses satisfaction with care; and Continu-um,Reference Rose, Sweeney, Leese, Clement, Burns and Catty19 which assesses perceived continuity of care.

Staff well-being outcomes

Staff burnout was assessed using the Maslach Burnout InventoryReference Maslach and Jackson20 (emotional exhaustion, personal accomplishment and depersonalisation subscales were scored and reported separately, as recommendedReference Maslach and Jackson21); work engagement, using the Work Engagement Scale;Reference Schaufeli and Bakker22 general psychological health using the General Health Questionnaire;Reference Goldberg and Williams23 and psychological flexibility, measured using the Work-Related Acceptance and Action Questionnaire.Reference Bond, Lloyd and Guenole24

Team outcomes

Participating teams' CRT model fidelity was assessed at baseline and 12-month follow-up using the CORE CRT Fidelity Scale,Reference Lloyd-Evans, Bond, Ruud, Ivanecka, Gray and Osborn11 which scores teams from one to five on 39 fidelity items in four subscales (referrals and access, content and delivery of care, staffing and team procedures, timing and location of care), yielding a total score ranging from 39 to 195.

In-patient admissions

Patient data from patient records for all patients from the catchment area of each participating CRT during baseline and follow-up periods (whether or not they used the CRT itself) were used to generate three team-level outcomes: total number of psychiatric hospital admissions from the catchment area over 6 months, total number of compulsory psychiatric hospital admissions and total occupied in-patient bed days.

Readmission following CRT care

Patient records data for all patients admitted to the CRT during baseline and follow-up periods were used to calculate the number of CRT patients accepted for treatment by any acute care service during the 6-month follow-up, following discharge from the index acute admission.


Patient experience and staff well-being participants

Patients were screened for eligibility and contacted about the study initially by CRT staff. Research staff attempted to contact all identified potential participants consecutively in the order they were discharged, until the site recruitment target was achieved. An information sheet about the study was sent by researchers and participants provided informed consent before completing questionnaires through face-to-face interview, online questionnaire or by phone. Patient-participants were given a thank you gift of £10 cash or vouchers. CRT staff were contacted by study researchers and invited to consent and complete measures using an online questionnaire.

In-patient admissions and readmission following CRT care

Information technology staff from participating NHS trusts provided anonymised data from patient records about all acute service use during data-collection periods. Study researchers calculated study outcomes from these raw data (further details in supplementary data 2).

Fidelity Scale scores were derived for each team from a structured, 1-day ‘fidelity review’ audit following a well-defined protocol, involving three independent reviewers from the study team (including at least one clinician and one patient or carer-researcher). Reviewers interviewed the CRT manager, staff, patients and carers and managers from other local services, and conducted a case-note audit and review of team policies and procedures. They then used the information gathered to score the team on each fidelity item, in accordance with criteria and guidance set out in the measure.

Patient experience and staff well-being outcomes data were entered directly into the ‘Opinio’ UCL secure online database, then downloaded as Excel files. Patient-record data were provided by NHS trusts in Excel or Word files. All data were transferred to Stata 14 software for analysis.


The primary hypothesis, that patients' satisfaction with the CRT, measured by the CSQReference Atkisson and Zwick14 is greater in the intervention group teams than control teams, was analysed using a linear random-effects model (mixed model) with a random intercept for CRT, controlling for mean baseline CSQ score by CRT. Patient-reported perceived continuity of care and measures of staff well-being were analysed similarly using linear random-effects models. Regression coefficients and 95% confidence intervals were reported. The calculated sample size provided 95% power to detect half a standard deviation difference between groups in mean satisfaction measured by the CSQ (3.5 points assuming a typical s.d. of 7.0) using a two-sided test, allowing for within-team clustering (intraclass correlation coefficient  0.05).

Patient outcomes at follow-up (in-patient admissions, bed days and readmissions following CRT care) were compared between intervention and control groups using Poisson random-effects modelling with a random intercept for trust. For each outcome, baseline score was set as the exposure variable, as it accounts for the baseline population and health of the catchment area as well as local admissions policies. Incidence rate ratios and 95% confidence intervals are reported. A second set of analyses was also conducted, using catchment area population as the exposure variable.

At team level, Fidelity Scale scores in the intervention and control groups at follow-up were compared, adjusting for baseline scores. The relationship was explored between change in team Fidelity Scale score from baseline to 12-month follow-up, and changes in five study outcomes: patient satisfaction, staff work engagement, in-patient admissions, in-patient bed use and readmission following CRT care. Relationships at team level between change in outcomes and change in fidelity scale subscale scores were also explored. For patient satisfaction and staff work engagement, we fitted linear regression models relating change in outcome (follow-up – baseline) to change in Fidelity Scale score (follow-up – baseline). For the remaining outcomes, we used normalised change in outcome ((follow-up – baseline)/square-root (baseline)). To aid interpretation, we present correlations to summarise these relationships with the corresponding P-values from the regression/correlation analyses.


Trial recruitment

Recruitment to the trial is summarised in the CONSORT diagram in Fig. 1. All 25 CRT teams were retained in the trial. We did not achieve our recruitment target of 15 patients per team in six teams at baseline and in one team at follow-up. These shortfalls occurred in teams with smaller case-loads, or where eliciting staff help to contact potential participants was problematic. At each time point, 62% of eligible patients whose contact details were provided to researchers agreed to participate (353/567 at baseline; 371/594 at follow-up); however, the proportion of eligible patients in each CRT who were initially approached by CRT staff is unknown. At each time point, 79% of all current staff in trial CRTs participated (441/560 at baseline and 431/544 at follow-up). One NHS trust, covering five participating CRTs, was unable to provide data at follow-up from patient records about whether in-patient admissions were compulsory or voluntary; complete patient records data were otherwise obtained.

CSQ, Client Satisfaction Questionnaire.

Fig. 1 Crisis team Optimisation and RElapse prevention (CORE) crisis resolution team (CRT) service improvement programme cluster randomised trial – CONSORT flow diagram.

The characteristics of participants recruited for patient experience and staff well-being outcomes are summarised in Tables 1 and 2 (further information in supplementary data 3). More patient-participants were men in the control teams than the intervention teams at follow-up (43% compared with 34%); more staff participants were men in control teams than intervention teams at both time points. No other marked differences between groups were apparent. No serious adverse events were identified during participant recruitment or data collection; no study-related harms were reported by participating CRTs.

Table 1 Patient-participant characteristics in the Crisis team Optimisation and RElapse prevention (CORE) crisis resolution team service improvement programme trial

Table 2 Staff characteristics in the Crisis team Optimisation and RElapse prevention (CORE) crisis resolution team service improvement programme trial

Intervention delivery

Figure 2 describes the implementation of the trial intervention. The CRT manager was unable to organise a scoping day for the whole team in one CRT. Otherwise, all the main components of the intervention were provided in all teams, but not always as promptly or completely as planned. Teams targeted a median of eight fidelity items each (selected as priorities by the team) over the course of the year in their service improvement plans. Further information about the content of intervention group teams' service improvement plans is provided in supplementary data 8.

Facilitator in post: green, yes, throughout; amber, yes, but with a change in facilitator during intervention year; red: no facilitator for full-year. Team scoping day: green, held within first 3 months; amber, held later than 3 months; red, not held. Service improvement plan (SIP) made: green, within first 3 months; amber, later than 3 months; red, plan not made. SIP reviewed regularly: green, reviewed at least 3 times during study year; amber, reviewed fewer than 3 times; red, not reviewed. Interim fidelity review: green, held in month 6 or 7; amber, held later than month 7; red, no reviewed. Attendance at learning collaboratives: green, facilitator and CRT team members attended events; amber, just facilitator attended.

Fig. 2 Implementation of the crisis resolution team (CRT) service improvement programme trial intervention.

Fidelity Scale scores

Teams' baseline scores for fidelity to a model of good CRT practice ranged from 97 (low fidelity) to 134 (moderate fidelity) – compared with a median score from a 75-team national survey in 2014 of 122.Reference Lloyd-Evans, Bond, Ruud, Ivanecka, Gray and Osborn11 Eleven out of 15 teams in the intervention group improved their Fidelity Scale score from baseline to follow-up, including teams from seven of the eight participating trusts, and teams with higher and lower model fidelity at baseline. The range in change scores on the CORE CRT Fidelity Scale in intervention group teams was from −22 to +37, with a mean change of 8.1 points. (The mean baseline Fidelity Scale score in the intervention teams was 116.4 and the mean follow-up score was 124.5). This contrasts with the control group, where none of the ten teams increased their Fidelity Scale score from baseline to follow-up, with a range in change scores from −20 to 0, with a mean change of −9.7 points. (The mean baseline Fidelity Scale score in the control teams was 122.2 and the mean follow-up score was 112.5.) There was a significant difference in outcome Fidelity Scale scores between intervention and control groups, adjusting for baseline scores (P = 0.0060). Further details of participating teams' Fidelity Scale scores are provided in supplementary data 4.

Trial outcomes

There was no significant difference between the intervention and control group teams for the trial's primary outcome of patient satisfaction: regression analysis suggested slightly higher satisfaction in the intervention group (coefficient 0.97, 95% CI −1.02 to 2.97) but this was not significant (P = 0.34). There was also no statistically significant difference between groups in patient-rated continuity of care, or four of the six staff well-being measures (with significantly better staff psychological health and psychological flexibility in the intervention group). The trial outcomes are summarised in Table 3. Descriptive data for patient experience and staff well-being outcomes at team level are provided in supplementary data 5.

Table 3 Crisis team Optimisation and RElapse prevention (CORE) crisis resolution team (CRT) service improvement trial results – patient, staff and service-use outcomes

IQR, interquartile range.

a. Staff and patient analysis: mixed modelling (CRT as random effect).

b. Incidence rate ratio (IRR) baseline score on outcome measure as exposure variable (trust as random effect).

c. Compulsory admissions data missing for five teams at follow-up (all from the same National Health Service trust: three in the intervention group and two in the control group).

At team level, there were significantly fewer total in-patient admissions and in-patient bed days in the intervention group than the control group over 6 months, after adjustment for baseline rates, suggesting that admissions were reduced more in intervention teams than in controls during the study period. These results were not replicated in a second analysis that adjusted for catchment area population instead of baseline rates, suggesting that admission rates relative to the size of the local population may not have been significantly lower in intervention teams than controls at follow-up, although we note that this second analysis does not adjust for differences in patient case-mix across areas. There was no difference between groups in rates of compulsory in-patient admissions or in rates of readmission to acute care following an episode of CRT care. Further details are provided in supplementary data 6.

Relationship between outcomes and model fidelity

There was a weak positive correlation of 0.34 between change in Fidelity Scale score and change in patient satisfaction, which corresponds to a mean increase of 0.65 points on the CSQ Scale for a ten-point increase in Fidelity Scale score. There was a weak negative correlation (i.e. in the expected direction) of −0.38 between Fidelity Scale score and readmissions following CRT care. There was no evidence of associations between change in total Fidelity Scale score and change in in-patient admission rates, bed days or staff work engagement. In post hoc exploratory analyses of subscale scores, a relationship between reduction in in-patient admissions and increase in access and referrals fidelity subscale score was apparent (correlation  −0.32). Readmissions following CRT care, by contrast, were most closely correlated with the timing and location of care (−0.45), and content of care (−0.34) subscales. Patient satisfaction was most closely correlated with the content of care subscale (correlation 0.36). Illustrative graphs are provided in supplementary data 7.


Main findings

For the primary outcome, patient satisfaction was not significantly greater in teams receiving the programme than in control teams. Model fidelity improved in 11 of 15 teams receiving the CRT service improvement programme over the study period, compared with none of the 10 control teams. There was some indication of significantly better results over the study period for the intervention teams compared with controls regarding hospital admission rates and in-patient bed use. Staff psychological health and psychological flexibility were higher at follow-up in the intervention group. There were non-significant trends favouring the intervention group teams regarding patient satisfaction, readmission to acute services following CRT care, and staff morale and job satisfaction. There was no evidence that the trial intervention reduced rates of compulsory admissions.

Altogether, this suggests the intervention was insufficient to achieve all its intended service improvements, but did achieve some, notably better model fidelity and reduced in-patient admissions. It may thus help unlock the potential benefits of CRTs in reducing the high costs and negative experience for patients associated with in-patient admissions. Positive results from our study also provide international validation for the process developed by the US Implementing Evidence-Based Practice projectReference McHugo, Drake, Whitley, Bond, Campbell and Rapp13 but not previously trialled in a UK NHS context, as a means to support implementation of complex interventions in mental healthcare.


Three limitations of the study relate to its design. First, other local and national service initiatives that arose during the year of the trial may have influenced CRT implementation and outcomes independently of the trial intervention. Second, because CRTs in each trust share senior managers and communicate regularly at management level, there is a possibility of contamination, where elements of the trial intervention were also accessed by control teams. Third, the 1-year follow-up period may have been too short for teams to fully enact their service improvement plans and to capture all changes in model fidelity and outcomes resulting from the trial intervention.

Four further limitations of the study relate to data collection. First, for patient experience outcomes, the participants recruited were highly satisfied with care, compared with previous studies,Reference Johnson, Nolan, Pilling, Sandor, Hoult and McKenzie4, Reference Johnson, Nolan, Hoult and White25 and may well over-represent those who were best engaged and most easily contactable by CRT staff. CRTs may have varied in the proportion of patients they approached and asked about taking part in the trial, with possible resulting bias. There appears to have been a ceiling effect in our sample with the trial primary satisfaction outcome measure (CSQ-8): 26% of participants in the treatment group at follow-up gave a maximum score of 32, compared with 12% in the control group. There may have been differences between groups in patients' satisfaction with CRTs that our evaluation failed to capture. We further note that fidelity gains regarding service organisation or the extent of care, even where achieved, may not have translated into increased patient satisfaction as broadly measured by the CSQ-8, if not accompanied by improvements in staff's clinical competenceReference Bond and McGovern26 and reduction in negative interactions with individual staff,Reference Sweeney, Fahmy, Nolan, Morant, Fox and Lloyd-Evans27 both of which have been identified as important to patient experience.

Second, data regarding acute service use were provided in anonymised form from NHS patient records and could not be verified as complete by researchers. Information about whether in-patient admissions were compulsory or voluntary was not available for five teams at follow-up.

Third, neither fidelity reviewers nor participating services could be masked to teams' trial allocation status during follow-up CRT fidelity reviews. Intervention group teams may have been more motivated to prepare thoroughly for their review and thus maximise their score. Reviewers may have unconsciously favoured the intervention group when assessing fidelity.

Fourth, it was not possible to confirm wholly accurate data regarding CRTs' catchment area population size, some of which were based on general practitioner registration rather than geographical area (see supplementary data 2). This possible measurement error does not affect the results for service-use outcomes reported in the main text of this paper – which are in any case better able to assess change in service use (and therefore the impact of the intervention) during the study period, through adjusting for baseline service use in each team – but may have affected the second analysis of service-use outcomes (supplementary data 6), which adjusted for catchment area population.

Implications for research

A future economic evaluation of the study will explore the cost-effectiveness of the intervention as delivered in this trial. Qualitative and process evaluations (also to be reported separately) will explore in more depth the content of support provided by facilitators in the project, the focus of teams' service improvement plans, the organisational contexts in which the intervention was delivered, and how these factors may relate to the extent of teams' success in improving model fidelity during the project. This may inform the development, and then evaluation, of a revised, CRT service improvement programme that better targets critical components of care, engages stakeholders and addresses organisational barriers to service change.

Our trial sought to improve outcomes in CRTs through the mechanism of increasing teams' model fidelity, but only weak relationships between changes in teams' overall Fidelity Scale score and outcome were established, and not for all outcomes. Three possible reasons for this could be explored in future research. First, the reliability of the CORE CRT Fidelity Scale has not been unequivocally established.Reference McHugo, Drake, Whitley, Bond, Campbell and Rapp13In vivo testing of interrater reliability is desirable. Second, some critical components of CRTs may not be assessed by the CORE CRT Fidelity Scale. All fidelity scales are better able to audit team organisation and the extent of service provision than to assess the clinical competence with which care is delivered,Reference Bond and McGovern26 and relationships between fidelity and outcomes are moderate even for the most well-established scales.Reference Kim, Bond, Becker, Swanson and Reese28 Other methods, possibly involving direct observation of practice; may be required to evaluate clinical competence in CRTs. Third, critical components of effective CRT services may be present but insufficiently weighted in teams' overall Fidelity Scale score for it to relate closely to service outcomes. The closer relationships with some outcomes we found for specific fidelity subscales provide some support for this idea. A large observational study, evaluating model fidelity and outcomes across many CRTs, could test hypotheses about the relationship between team outcomes and Fidelity Scale item or subscale scores.

Implications for policy and practice

In general, for the control teams in our trial, Fidelity Scale scores dropped, readmissions following CRT care rose, and in-patient service-use outcomes were worse than for the intervention group teams. This suggests that there is a pressing need for effective CRT service improvement support. This trial suggests that considerable input is needed to improve service quality in CRTs: our successfully implemented, multicomponent programme of sustained support for CRTs did not demonstrate improved patient satisfaction, and only partially achieved its aims. The CORE CRT service improvement programme provides a useful starting point for developing future CRT service improvement initiatives, having achieved improvements in model fidelity in teams from varied geographical and provider-trust contexts and a range of baseline levels of fidelity, and having some evidence of effectiveness in reducing admissions and in-patient service use. It is informed by a model of implementing complex interventions in mental health settings with prior evidence of effectiveness in US contextsReference McHugo, Drake, Whitley, Bond, Campbell and Rapp13 that now also has evidence of applicability to English services, with potential usefulness beyond CRTs. The CORE service improvement structures and the online CRT Resource PackReference Lloyd-Evans and Johnson29 are publicly available and provide guidance, materials and case examples to support good practice in CRTs. Clear specification of a CRT service model and development of effective resources to support CRT service improvement can help ensure that consistent provision of acceptable and effective home treatment for people experiencing mental health crisis is fully achieved.


This study was undertaken as part of the CORE Programme, which is funded by the UK Department of Health National Institute for Health Research (NIHR) under its Programme Grants for Applied Research programme (Reference Number: RP-PG-0109-10078). The views expressed in this paper are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. Neither the study funders not its sponsors had any role in study design, in the collection, analysis or interpretation of data, or in the writing of this report or the decision to submit it for publication. S.J. and D.O. were in part supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) North Thames at Bart's Health NHS Trust.


Our colleague Steve Onyett died suddenly in September 2015. He contributed to the design of this study and oversaw delivery of the service improvement programme trial intervention. We miss his warmth and wisdom. Thank you to the members of the CORE Study service user and carer working groups, who contributed to planning the trial and provided peer-reviewers for the trial fidelity reviews: Stephanie Driver, Pauline Edwards, Jackie Hardy, David Hindle, Kate Holmes, Lyn Kent, Jacqui Lynskey, Brendan Macken, Jo Shenton, Jules Tennick, Gen Wallace, Mary Clarke, Kathleen Fraser-Jackson, Mary Plant, Anthony Rivett, Geoff Stone and Janice Skinner. We are grateful for the help of staff and patients from all the CRT teams that participated in this study.

Supplementary material

Supplementary material is available online at


Declaration of interest None.




1Department of Health. Crisis Resolution/Home Treatment Teams. The Mental Health Policy Implementation Guide. Department of Health, 2001 ( Scholar
3Johnson, S.Crisis resolution and home treatment teams: an evolving model. Br J Psychiatry 2013; 19: 115–23.Google Scholar
4Johnson, S, Nolan, F, Pilling, S, Sandor, A, Hoult, J, McKenzie, N, et al. Randomised controlled trial of acute mental health care by a crisis resolution team: the north Islington crisis study. BMJ 2005; 331: 599.CrossRefGoogle ScholarPubMed
5Murphy, SM, Irving, CB, Adams, CE, Waqar, M.Crisis intervention for people with severe mental illnesses. Cochrane Database Syst Rev 2015; 12: CD001087.Google Scholar
6Hopkins, C, Niemiec, S.Mental health crisis at home: service user perspectives on what helps and what hinders. J Psychiatr Ment Health Nurs 2007; 14: 310–8.CrossRefGoogle ScholarPubMed
7MIND. Listening to Experience: An Independent Inquiry into Acute and Crisis Mental Healthcare. MIND, 2011.Google Scholar
8Jacobs, R, Barrenho, E.The impact of crisis resolution and home treatment teams on psychiatric admissions in England. J Ment Health Policy Econ 2001; 14: S13.Google Scholar
9Lloyd-Evans, B, Paterson, B, Onyett, S, Brown, E, Istead, H, Gray, R, et al. National implementation of a mental health service model: a survey of Crisis Resolution Teams in England. Int J Ment Health Nurs 2017; January 11 (Epub ahead of print).Google ScholarPubMed
10Lloyd-Evans, B, Lamb, D, Barnby, J, Eskinazi, M, Turner, A, Johnson, S.Mental health crisis resolution teams and crisis care systems in England: a national survey. BJPsych Bull 2018; 42: 146–51.CrossRefGoogle ScholarPubMed
11Lloyd-Evans, B, Bond, G, Ruud, T, Ivanecka, A, Gray, R, Osborn, D, et al. Development of a measure of model fidelity for mental health crisis resolution teams. BMC Psychiatry 2016; 16: 427.CrossRefGoogle ScholarPubMed
12Lloyd-Evans, B, Fullarton, K, Lamb, D, Johnston, E, Onyett, S, Osborn, D, et al. The CORE Service Improvement Programme for mental health crisis resolution teams: study protocol for a cluster-randomised controlled trial. BMC Trials 2016; 17: 158.CrossRefGoogle ScholarPubMed
13McHugo, GJ, Drake, RE, Whitley, R, Bond, G, Campbell, K, Rapp, C, et al. Fidelity outcomes in the national implementing evidence-based practices project. Psychiatr Serv 2007; 58: 1279–84.CrossRefGoogle ScholarPubMed
14Atkisson, C, Zwick, R.The client satisfaction questionnaire: psychometric properties and correlations with service utilisation and psychotherapy outcome. Eval Programme Plann 1982; 5: 233–37.CrossRefGoogle Scholar
15Campbell, MK, Piaggio, G, Elbourne, DR, Altman, DGfor the CONSORT Group. Consort 2010 statement: extension to cluster randomised trials. BMJ 2012; 345: e5661.CrossRefGoogle ScholarPubMed
16Wheeler, C, Lloyd-Evans, B, Churchard, A, Fitzgerald, C, Fullarton, K, Mosse, L, et al. Implementation of the crisis resolution team model in adult mental health settings: a systematic review. BMC Psychiatry 2015; 15: 74.CrossRefGoogle ScholarPubMed
17Morant, N, Lloyd-Evans, B, Lamb, D, Fullarton, K, Brown, E, Paterson, B, et al. Crisis resolution and home treatment: stakeholders’ views on critical ingredients and implementation in England. BMC Psychiatry 2017; 17: 254.CrossRefGoogle ScholarPubMed
18Torrey, WC, Bond, GR, McHugo, GJ, Swain, K.Evidence-based practice implementation in community mental health settings: the relative importance of key domains of implementation activity. Admin Policy Ment Health 2012; 39: 353–64.CrossRefGoogle ScholarPubMed
19Rose, D, Sweeney, A, Leese, M, Clement, S, Burns, T, Catty, J, et al. Developing a user-generated measure of continuity of care: brief report. Acta Psychiatr Scand 2009; 119: 320–4.CrossRefGoogle ScholarPubMed
20Maslach, C, Jackson, SE.The measurement of experienced burnout. J Organ Behav 1981; 2: 99113.CrossRefGoogle Scholar
21Maslach, C, Jackson, S.The Maslach Burnout Inventory. Consulting Psychologists Press, 1981.Google Scholar
22Schaufeli, W, Bakker, A.The measurement of work engagement with a short questionnaire. Educ Psychol Meas 2006; 66: 701–16.CrossRefGoogle Scholar
23Goldberg, D, Williams, P.The General Health Questionnaire. NFER-Nelson, 1988.Google Scholar
24Bond, F, Lloyd, J, Guenole, N.The Work-related Acceptance and Action Questionnaire (WAAQ): initial psychometric findings and their implications for measuring psychological flexibility in specific contexts. J Occupational Organisational Psychol 2013; 86: 331–47.CrossRefGoogle Scholar
25Johnson, S, Nolan, F, Hoult, J, White, R.Outcomes of crises before and after introduction of a crisis resolution team. Br J Psychiatry 2005; 187: 6875.CrossRefGoogle ScholarPubMed
26Bond, GR, McGovern, MP.Measuring organizational capacity for integrated treatment. J Dual Diagn 2013; 9: 165–70.CrossRefGoogle Scholar
27Sweeney, A, Fahmy, S, Nolan, F, Morant, N, Fox, Z, Lloyd-Evans, B, et al. The relationship between therapeutic alliance and service user satisfaction in mental health inpatient wards and crisis house alternatives: a cross-sectional study. PLoS One 2014; 9: e100153.CrossRefGoogle ScholarPubMed
28Kim, SJ, Bond, GR, Becker, DR, Swanson, SJ, Reese, SL.Predictive validity of the individual placement and support fidelity scale (IPS-25): a replication study. J Vocat Rehab 2015; 43: 209–16.CrossRefGoogle Scholar
29Lloyd-Evans, B, Johnson, S.CORE CRT Resource Pack. UCL, 2014 ( Scholar

Lloyd-Evans et al. supplementary material

Lloyd-Evans et al. supplementary material 1

[Opens in a new window]
File 242 KB
Submit a response


Re: CORE Study - different interpretation of the results

22 March 2019
Brynmor Lloyd-Evans, Senior Lecturer, University College London
We agree with the thoughtful letter by Wong and colleagues up to a point. The CORE fidelity scale for Crisis Resolution Teams (CRTs) was based mainly on stakeholders’ opinions rather than robust empirical evidence regarding components of effective crisis care [1]. Some fidelity items may be more important than others, and some items may not constitute critical ingredients of effective CRTs. The CORE... More
Conflict of interest: None declared

CORE Study: Different Interpretation of the Results.

25 February 2019
Pang Loong Wong, Specialty Registrar in Psychiatry, South West London and St George's NHS Trust
Lloyd-Evans et al.1 published results from a cluster-randomised trial looking at the effect on patients of an improvement programme for mental health crisis resolution teams, in which the aim was to increase fidelity with the crisis resolution team model. In the intervention group, the authors found a reduction in admissions and in-patient bed days but no increase in average patient satisfaction.... More
Conflict of interest: None declared

Altmetric attention score

Full text views

Full text views reflects PDF downloads, PDFs sent to Google Drive, Dropbox and Kindle and HTML full text views.

Total number of HTML views: 218
Total number of PDF views: 989 *
View data table for this chart

* Views captured on Cambridge Core between 14th February 2019 - 4th December 2020. This data will be updated every 24 hours.

Open access
Hostname: page-component-b4dcdd7-nf2kx Total loading time: 0.83 Render date: 2020-12-04T14:58:54.542Z Query parameters: { "hasAccess": "1", "openAccess": "1", "isLogged": "0", "lang": "en" } Feature Flags last update: Fri Dec 04 2020 14:00:26 GMT+0000 (Coordinated Universal Time) Feature Flags: { "metrics": true, "metricsAbstractViews": false, "peerReview": true, "crossMark": true, "comments": true, "relatedCommentaries": true, "subject": true, "clr": false, "languageSwitch": true }

Send article to Kindle

To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

The CORE service improvement programme for mental health crisis resolution teams: results from a cluster-randomised trial
Available formats

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

The CORE service improvement programme for mental health crisis resolution teams: results from a cluster-randomised trial
Available formats

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

The CORE service improvement programme for mental health crisis resolution teams: results from a cluster-randomised trial
Available formats

Reply to: Submit a response

Your details

Conflicting interests

Do you have any conflicting interests? *