Study design and randomisation procedure

The study was designed as a multisite randomised controlled trial (ISRCTN57292778) with a CBT and a TAU condition. The Central Committee for Research involving Human Subjects approved the study. When the inclusion criteria were met, written informed consent was requested. The randomisation was stratified over six regions in The Netherlands and performed independently by University Medical Center Groningen.

Study population

The study population comprised people with schizophrenia or schizoaffective disorder with persistent symptoms despite adequate antipsychotic medication. Two university hospitals and seven mental health centres in The Netherlands participated in the study.

The inclusion criteria were:

1. (a) age 18–64;

2. (b) diagnosis of schizophrenia or schizoaffective disorder (DSM–IV–TR, 295.xx) ¹⁶

3. (c) Positive and Negative Syndrome Scale (PANSS) ^{Reference Kay, Fiszbein and Opler17} scores of delusions ≥4 OR hallucinations ≥4 OR suspiciousness ≥4) AND Psychotic Symptoms Rating Scale (PSYRATS) ^{Reference Haddock, McCarron, Tarrier and Faragher18} scores delusions-suffering ≥2 AND delusions-impact ≥2 OR hallucinations-suffering ≥2 AND hallucinations-impact ≥2;

4. (d) treatment resistance defined as failure of two or more antipsychotic treatments of at least 6 weeks over the past 2 years.

The exclusion criteria were: intellectual disabilities with IQ<80; severe addiction; no competence in the Dutch language; and previous exposure to CBT.

Outcome measures, power analysis and measurements

The primary outcome measure in this study was the number of days that participants functioned within the normal range. In order to function within the normal range they had to fulfil three criteria: the level of social functioning had to be in the 95% range of the normal population (assessed by the Social Functioning Scale (SFS)); ^{Reference Birchwood, Smith, Cochrane, Wetton and Copestake19} there had to be no or minimal suffering from residual symptoms; and there had to be no or minimal affect on daily living of residual symptoms on the PSYRATS.

The secondary outcome measures included symptoms based on the PSYRATS and the PANSS and quality of life assessed with the scale of the World Health Organization (WHO–QoL). ²⁰

A power analysis was conducted. In order to show medium effect sizes (0.50) with an alpha of 0.05 and a power of 0.80, at least 70 participants were required in each condition at the end of the study. As a conservative estimate, we anticipated about a 30% drop-out rate during 18 months and 230 participants were planned to enrol.

All participants had assessments at months 0, 3, 6, 9, 12, 15 and 18. The number of days of normal functioning was estimated at 91 if two consecutive measurements were within the range of normal functioning. The number of days that a participant functioned within the normal range was estimated at 45.5 if the individual moved from below normal functioning to normal functioning or the other way around. The number of days of normal functioning was estimated at zero if the measurements before and after both indicated below normal functioning. The research assistants were trained and had a consensus meeting every 3 months. They were masked to the treatment arm of participants.

Treatments

A CBT therapist (psychologist) and a CBT assistant (nurse) delivered CBT in the experimental condition. The first four sessions were carried out by the CBT assistant and consisted of a pre-therapy training of the concepts ‘antecedents’, ‘thoughts’, ‘feelings’ and ‘behavioural responses’. The goal was to teach participants that people can have different thoughts about an identical incident and consequently experience different emotions and act differently.

The CBT therapist made an individual case formulation and a list of goals with the participant and disputed alternative interpretations of incidents and designed behavioural experiments to test hypotheses. The CBT assistant could help with several therapeutic procedures such as exposure to ‘dangerous’ places, improving activity levels, exercises to improve self-esteem and challenging ongoing dysfunctional thoughts.

The therapy was manualised in six steps: pre-therapy training in the concepts: antecedent, thought, feeling and behaviour; consumer information and mutual roles and demands; assessment; shared case formulation and goal setting; changing dysfunctional cognitions into more functional thoughts; and consolidation. Therapy was provided in weekly sessions for 26 weeks but could end earlier when the participant attained the goals set. The therapists met every 6 weeks for supervision and mutual consultation. All sessions with the participants were recorded on audiotape and a random selection of the tapes was scored on treatment validity using the cognitive therapy scale. ^{Reference Young and Beck21,Reference Haddock, Devane, Bradshaw, McGovern, Tarrier and Kinderman22}

The TAU condition was routine care within each site and typically consisted of pharmacotherapy and contacts with a community psychiatric nurse. Some participants followed educational programmes in managing medication, persistent symptoms, addiction and work.

Costs and unit prices

The economic evaluation was conducted from a societal perspective; costs in and outside the healthcare sector were assessed. Medical costs of participants included medication, in-patient care, community care and general healthcare. Costs of the CBT intervention were registered in detail such as training and supervision of psychologists and nurses during the study, the number of sessions per participant, the wages of therapists and additional aspects, such as the costs of therapy office and accommodation. Both antipsychotic medication and non-prescribed medication were included in medication costs.

In addition, various types of costs outside the healthcare sector were included in the analyses. Time costs (related to the CBT intervention) were based on the number of attended training and therapy sessions, combined with information on the net income of participants (shadow prices were used for participants who did not have paid work). Costs of informal care were based on the monetary valuation of the time invested by relatives or acquaintances in helping or assisting the participant (household work, accompanying individuals to healthcare professionals, discussing problems related to the psychiatric condition, etc). Out-of-pocket costs were additional costs generated by the participants, such as the costs of cancelling holidays or other planned activities. The cost of productivity losses due to illness-related absence from work were estimated by means of the friction cost method. ^{Reference Koopmanschap, Rutten, van Ineveld and van Roijen23} Compensation mechanisms were taken into account when estimating these productivity costs. ^{Reference Jakob-Tacken, Koopmanschap, Meerding and Severens24} Furthermore, the costs related to changes in the amount of voluntary (unpaid) work conducted by participants were taken into account.

Information on healthcare consumption was collected by means of a detailed questionnaire that was filled in during an interview covering the previous 3 months. If there was uncertainty the research assistants checked the data using case histories. Information on the use of antipsychotics was collected through the healthcare professionals concerned. In order to facilitate comparisons with other economic evaluations, unit prices (i.e. the price of one unit of each included cost type) were mainly based on Dutch standard prices for the year 2007. ^{Reference Oostenbrink, Bouwmans, Koopmanschap and Rutten25} True costs of used resources were estimated when standard prices were not available.

Design of the economic evaluation

The economic evaluation was designed as a cost-effectiveness analysis. In this type of analysis, costs and the primary health outcome associated with an intervention are used to calculate the incremental cost-effectiveness ratio (ICER) relative to one or more alternatives. ^{Reference Drummond, Sculpher, Torrance, O'Brien and Stoddart26} The formula used for calculating the ICER is:

\batchmode \documentclass[fleqn,10pt,legalpaper]{article} \usepackage{amssymb} \usepackage{amsfonts} \usepackage{amsmath} \pagestyle{empty} \begin{document} \[\ ICER=\frac{(C_{\mathrm{CBT}}-C_{\mathrm{TAU}})}{(DNF_{\mathrm{CBT}}-DNF_{\mathrm{TAU}})}\ \] \end{document}

where C _CBT is mean costs in the CBT group; C _TAU is mean costs in the TAU group; DNF _CBT is the mean days of normal functioning in the CBT group; and DNF _TAU is the mean days of normal functioning in the TAU group.

Both costs and health outcomes were discounted in accordance with current guidelines. The bootstrap method was applied to provide information on the uncertainty of the results of the economic evaluation. ^{Reference Efron and Tibshirani27} The ICERs were calculated for each of the 2000 bootstrap iterations; simulated values of the mean estimates for the cost and outcome differences were added to the cost-effectiveness plane. ^{Reference Black28} Finally, cost-effectiveness acceptability curves ^{Reference Fenwick, O'Brien and Briggs29} were calculated. Cost-effectiveness acceptability curves inform decision makers on the probability that an intervention will be cost-effective, which depends on the willingness to pay per additional unit of health outcome.

Statistical procedures

Kolmogorov–Smirnov tests were done to check the normal distribution of variables. To examine successful randomisation and compare those who dropped out of the study with completers, logistic regression and independent t-tests were used. The EM algorithm with a bootstrap approach ^{Reference Oostenbrink and Al30} was applied to deal with participants for whom not all the data were available at the various time points. The analyses were conducted using SPSS version 14 and R version 2.10.1 on Windows (R Foundation, Vienna, Austria; http://www.r-project.org). ³¹

Baseline characteristics and participant flow

The participant flow can be seen in Fig. 1. Participants were recruited between 1 April 2005 and 30 September 2006. Participants were treated for 6 months and followed-up with assessments at 3-month intervals for 18 months. The 12 participants who stopped therapy prematurely did so after one or two sessions. The eight clerical errors are participants that ought to have been referred to one of the therapists by the coordinating therapist concerned, but who failed to do so. Nine individuals, who with-drew consent, did so after they heard that they were randomised to the TAU condition. The median number of sessions was 3 by nurse therapists during pre-treatment training and 13 by psychologists (intention-to-treat). Only 21 participants were co-treated by a nurse therapist and psychologist; the median number of session by the nurse therapist was 5.

Fig. 1 The flow of the participants in the study. CBT, cognitive–behavioural therapy; TAU, treatment as usual.

The randomisation appeared to be successful. There were no significant differences between the groups in the demographics (Table 1). Logistic regression aimed at possible confounders found no significant effects at baseline and end of treatment. A random sample of four audiotaped sessions for each therapist were rated for therapy fidelity: 97.8% of the recorded interviews were above the fidelity score. The research assistants were asked to guess in which condition the participants were at each assessment and apparently grew more and more unmasked: 70% of the guesses were actually correct.

Table 1 Demographic characteristics of cognitive–behavioural therapy (CBT) and treatment as usual (TAU) groups

Characteristic	CBT group (n = 109)	TAU group (n = 97)
Age, years: mean (s.d.)	36.52 (11.18)	37.45 (10.61)
Duration of illness, years: mean (s.d.)	10.14 (7.59)	11.02 (8.37)
Male, %	69	73
Education,b mean (s.d.)	4.20 (1.80)	4.32 (1.93)
Social Functioning Scale, total score: mean (s.d.)	112.17 (21.27)	109.59 (21.85)
Positive and Negative Syndrome Scale, total score: mean (s.d.)	67.71 (13.21)	70.21 (12.81)
Participants with delusions, %	81	86
Participants with hallucinations, %	61	65

Health outcome

The primary outcome measure in this study was the number of days that participants functioned within the normal range. The CBT group was functioning significantly better than the TAU group at most of the time points when assessments were made, except for the 9- and the 18-month assessment. A breakdown of the composite dichotomous normal functioning outcome measure showed that the gain in social functioning (SFS) was comparable in both groups, but the CBT group experienced less suffering and impact of symptoms (PSYRATS) on daily living than the TAU group did. The number needed to treat varied from 5 to 10 on the intention-to-treat analysis (Table 2).

Table 2 Number of participants with normal functioning at different time points for the cognitive–behavioural therapy (CBT) and treatment as usual (TAU) groups: intention-to-treat

Normal functioning at assessment	CBT (n = 109)	TAU (n = 97)	Fisher's exact test	Number needed to treat (95% CI)
3 months	34	10	0.000	5 (3.2–9.7)
6 months	31	14	0.018	8 (4.0–33.1)
9 months	33	20	0.150	10 (ns)
12 months	40	20	0.014	7 (3.5–25.2)
15 months	42	24	0.037	8 (3.8–79.9)
18 months	39	25	0.134	10 (ns)

In the CBT group, the participants had on average 183 days of normal functioning and the TAU group 106 days. The mean difference of 77 days of normal functioning between groups was significant (95% CI 29.7–124.0). The results of the secondary outcome measures are shown inTable 3. The differences between groups on the PANSS and the WHO–QoL were not statistically significant. The subjective rating of symptoms on the PSYRATS was statistically significant.

Table 3 Differential effects of cognitive–behavioural therapy (CBT) and treatment as usual (TAU) on symptoms and quality of life over the study period

	CBT, mean (s.e.)		TAU, mean (s.e.)			Interaction group × time point P a
	Baseline	End of treatment	Baseline	End of treatment	F
Positive and Negative Syndrome Scale, total	67.9 (1.3)	61.2 (1.5)	70.7 (1.3)	62.7 (1.6)	0.51	0.48
Psychotic Symptoms Rating Scale, total	29.9 (1.3)	19.9 (1.6)	32.8 (1.4)	27.8 (1.8)	4.40	0.04
World Health Organization – Quality of Life, total	71.9 (1.0)	75.1 (1.0)	73.5 (1.0)	75.4 (1.3)	0.76	0.39
World Health Organization – Quality of Life, psychological well-being	17.7 (0.3)	18.4 (0.3)	18.1 (0.3)	17.9 (0.4)	3.70	0.06

Costs and healthcare utilisation

The various medical and non-medical costs generated by both groups are presented in online Table DS1. The mean total costs of training therapists in CBT, supervising therapists during the study, and providing pre-therapy to participants was €426. The costs of the CBT sessions during the first 6 months of the study were included in the costs of contacts with psychologists, which were therefore higher in the CBT condition (approximately €550 of these costs were related to CBT sessions). Although the percentage of participants who were admitted during the study was comparable between groups, a small number of participants in the CBT group was admitted during most of the study period, leading to considerably higher costs. An overview of the total costs incurred over the duration of the study, broken down by time period, is provided in Table 4. The differences between the mean costs of both groups were not statistically significant. This should be interpreted with some caution, since the study was powered (as most economic evaluations) to demonstrate differences in health outcomes and not costs.

Table 4 Mean total costs (Euro, price level of 2007) during the study

	Cognitive–behavioural therapy		Treatment as usual
Time period	n	Total costs, mean	n	Total costs, mean	Difference in costs, mean (95% Cl)a
0–3 months	97	7158	77	5084	2074 (–29 to 4275)
3–6 months	94	5824	69	4888	936 (–931 to 2902)
6–9 months	88	5141	75	4840	300 (–1805 to 2282)
9–12 months	89	5222	75	3873	1350 (–314 to 2968)
12–15 months	96	4617	85	4963	– 346 (–2272 to 1521)
15–18 months	99	4301	87	3777	524 (–873 to 1969)
0–18 monthsb	109	33 130	97	29 578	3552 (–5162 to 12 184)

Cost-effectiveness analysis

The calculated value of the ICER was €47 per day with normal functioning gained. The point estimate of the ICER and the results of the bootstrap analyses are presented in the cost-effectiveness plane inFig. 2.

About 21% of the bootstrapped mean ICERs were located in the southeast quadrant, indicating that CBT dominated TAU in 21% of the bootstrapped mean ICERs. The other bootstrap simulations (79%) were located in the northeast quadrant indicating that both costs were higher and health outcomes were better in the CBT group. Interpretation of outcomes in the northeast (and southwest) quadrant depends on how much decision makers are willing to pay for an additional unit of health outcome. InFig. 3, the graph for the standard analysis shows the probability that CBT (compared with TAU) will be cost-effective for increasing willingness to pay per additional unit of health outcome. For instance, a willingness to pay approximately €84 per additional day of normal functioning is associated with a 70% probability that CBT is the optimal intervention.

Sensitivity analyses

In the first sensitivity analysis, participants in one of the six participating centres were left out of the analyses (n = 11; 5%). Outcomes of individuals in this centre were very distinct from the other centres: with considerably more hospitalisations and less experienced therapists. Excluding these participants led to lower average costs in the CBT group (€30 652) and the TAU group (€28 789), and a smaller mean difference of €1863 (CI –6918 to 10 647) between CBT and TAU.

In the second sensitivity analysis, we adjusted the way the number of days of normal functioning was calculated. A more liberal threshold for normal functioning was chosen, allowing one 2-score (moderate severe) on either suffering or the impact of symptoms. The mean difference in days of normal functioning between groups was now 58 days in favour of CBT.

In the third sensitivity analysis, participants who had been admitted to a hospital during the 3 months before the start of the study were excluded from the analysis. Results showed that mean total costs in both groups were much lower (CBT group (n = 81) €24 885, TAU group (n = 73) € 23 585), and the difference between the groups (€1300, CI – 7450 to 9949) was much smaller than in the previous analyses. For this analysis, the incremental costs were €14 per additional day of normal functioning gained. Fig. 3 presents the cost-effectiveness acceptability curves assessed for each of the described sensitivity analyses.

Fig. 2 Results of the cost-effectiveness analysis and bootstrap method.

C_CBT, mean costs in the cognitive–behavioural therapy (CBT) group; C_TAU, mean costs in the treatment as usual (TAU) group; DNF_CBT, mean days of normal functioning in the CBT group; DNF_TAU, mean days of normal functioning in the TAU group. The white circle is the point estimate of the incremental cost-effectiveness ratio.

Normal functioning and costs

This study shows that implementing CBT in routine care can alleviate the suffering of people with persistent psychosis and brings some of them into the range of normal functioning, with low levels of distress and minimal impact of the symptoms on their daily life. However, the additional health gains for participants in the CBT condition were accompanied by higher costs. Decisions on the further implementation of CBT are dependent on what decision makers are willing to pay for the incremental health gains. In this study, CBT was more likely to be cost-effective than TAU when the willingness to pay for an additional day of normal functioning gained would be higher than €47.

The overall mean total cost estimates per participant appeared to be in line with a recent economic study aimed at first-episode psychosis, ^{Reference Stant, TenVergert, Wunderink, Nienhuis and Wiersma3} and were generally higher than other cost results published in the last decade. ^{Reference Wykes, Steel, Everitt and Tarrier10–Reference Stant, TenVergert, Groen, Jenner, Nienhuis and van de Willige14} This may at least be partially because of the widening scope of costs included in economic studies, differences in costing methods applied, and increased use of more expensive antipsychotics in recent years. Furthermore, this study confirmed the widely acknowledged influence of hospital admissions ^{Reference Stant, TenVergert, Wunderink, Nienhuis and Wiersma3,Reference Knapp, Chisholm, Leese, Amaddeo, Tansella and Schene32} on the total costs in patient populations with psychosis.

Fig. 3 Cost-effectiveness acceptability curves based on the standard analysis and the additionally conducted sensitivity analyses. CBT, cognitive–behavioural therapy.

Hospital admissions and CBT

Although the experimental group was more likely to experience normal functioning, they also had double the in-patient costs. We have looked into the distribution of the days spent in hospital. These are skewed and the CBT group had 21 participants and the TAU group 15 participants with more than 60 in-patient days. The top seven outliers in the CBT group consumed 60% of all in-patient days. Five of these people were already long-stay in-patients when they were included in the study. It seems to us that CBT did neither cause admission nor lengthened the hospital stay. Six of the participants had almost no exposure to CBT and stopped after an average of 2.5 sessions with zero days of normal functioning. The one person that did have 19 sessions of CBT also improved to normal functioning for a period of 182 days.

Strengths

Important strengths of this study are the naturalistic study design and the relatively large number of participants. The study can be generalised to people with schizophrenia-spectrum disorders with persistent psychosis, because the number of exclusions is quite small. Medication was not kept fixed during the trial. Many medication regimes were switched during the study or dosages were calibrated to optimum levels. Only 54 people (12.6%) were excluded because of primary addiction, low IQ or no competence in the Dutch language; and 38 potential participants (8.8%) because they had undergone CBT in the past. A total of 120 people (28%) refused to take part in the study, which is comparable with previous findings in such studies. Of the 110 individuals who were randomised to the CBT group, only 12 individuals (11.0%) stopped treatment prematurely, which indicates that the intervention was very well tolerated. Furthermore, the participating therapists all worked in routine care services all over the country. All participants fulfilled broad inclusion criteria. Contrary to efficacy trials, individuals with disorganised and undifferentiated symptoms were not excluded, although communication, memory, learning potential and acting on plans are all compromised in these participants. Individuals in these subgroups had higher symptom scores than the paranoid and schizoaffective subgroups, and although they improved over time, they were still more symptomatic. In this real-life environment the CBT group attained normal functioning more often than the TAU group.

Another strength of the study is the long follow-up period. Importantly, the effects of CBT may wane over time, while the costs are stable after the intervention period. If individuals benefit from CBT after 18 months, making no additional therapy costs, the costs per day of normal functioning would reduce further. Long follow-up periods are more realistic in a relapsing condition such as schizophrenia. There will always be more moderate results on the outcome measures, but the mean accumulating costs will reduce after therapy termination.

Limitations

This study also had several limitations from both a clinical and economic point of view. First, CBT in psychosis is a complex intervention that requires expertise in psychopathology and treatment techniques. Some of the therapists were novices and trained and supervised during the study, but for optimum performance several years of experience are required. ^{Reference Wykes, Hayward, Thomas, Surguladze, Fannon and Landau33} This might explain site differences that resulted from different admission policies and differences in the expertise of the therapists. Also the commitment to supervision varied between centres.

Second, the primary outcome measure applied in this study is of concern. In the field of health technology assessment, there have been debates on the topic of adequate outcome measures for economic evaluations. Currently, most guidelines recommend the use of quality-adjusted life-years (QALYs). The primary outcome measure used in this study, days of normal functioning, does also seem to be relevant for policy decisions. However, the outcome cannot directly be compared with QALY analysis used in other studies. Moreover, there is no consensus on acceptable (cost) benchmarks per additional day of normal functioning gained. For the moment, policy makers will have to decide whether the presented willingness to pay values, and the associated probability outcomes, justify the (further) implementation of CBT in the healthcare system.

The final limitation concerns the generalisability of current findings to other healthcare systems. Directly transferring current results may be troublesome, not only because of substantial differences in healthcare systems but also to (international) variations in the costing methods applied.

When we consider the findings of this study, we must state that effects on symptoms as measured with the PANSS were absent, but that CBT did succeed in reducing the suffering from persistent psychotic symptoms and their impact on daily life and improved social functioning. Cognitive–behavioural therapy is not a quasi-antipsychotic. Psychotic symptoms can persevere, but the individual's appraisal of psychosis can be changed with CBT. Reductions in mood symptoms more than reductions in psychotic symptoms probably facilitate emotional well-being and social functioning in these individuals. This was also found in a recent trial to prevent relapse with CBT. ^{Reference Garety, Fowler, Freeman, Bebbington, Dunn and Kuipers34}