Hostname: page-component-cd4964975-g4d8c Total loading time: 0 Render date: 2023-03-30T08:09:40.939Z Has data issue: true Feature Flags: { "useRatesEcommerce": false } hasContentIssue true

Comparing individually tailored to disorder-specific internet-based cognitive–behavioural therapy: benchmarking study

Published online by Cambridge University Press:  18 July 2018

Martin Kraepelien*
Department of Clinical Neuroscience, Centre for Psychiatry Research, Karolinska Institutet, Sweden
Erik Forsell
Department of Clinical Neuroscience, Centre for Psychiatry Research, Karolinska Institutet, Sweden
Eyal Karin
eCentreClinic, Department of Psychology, Macquarie University, Australia
Robert Johansson
Department of Clinical Neuroscience, Centre for Psychiatry Research, Karolinska Institutet and Department of Psychology, Stockholm University, Sweden
Nils Lindefors
Department of Clinical Neuroscience, Centre for Psychiatry Research, Karolinska Institutet, Sweden
Viktor Kaldo
Department of Clinical Neuroscience, Centre for Psychiatry Research, Karolinska Institutet and Department of Psychology, Faculty of Health and Life Sciences, Linnaeus University, Sweden
Correspondence: Martin Kraepelien, FoU M58, SLSO Psykiatri Sydväst, Huddinge sjukhusområde, 141 86 Stockholm, Sweden. Email:
Rights & Permissions[Opens in a new window]


Disorder-specific internet-based cognitive–behavioural therapy (ICBT) is effective for depression, panic disorder and social anxiety. In this benchmarking study, a new, individually tailored, ICBT programme (TAIL) showed effects on depression (n = 284, d = 1.33) that were non-inferior to disorder-specific ICBT for depression in routine care (n = 2358, d = 1.35). However, the hypotheses that TAIL for individuals with social anxiety or panic disorder is inferior to disorder-specific ICBT could not be rejected (social anxiety: TAIL d = 0.74 versus disorder-specific d = 0.81; panic: TAIL d = 1.11 versus disorder-specific d = 1.47). Our findings strengthen the empirical base for TAIL as an alternative to disorder-specific ICBT for depression.

Declaration of interest


Short report
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (, which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright © The Royal College of Psychiatrists 2018

Disorder-specific, therapist-guided internet-based cognitive–behavioural therapy (ICBT) is established as efficacious and cost-effective for depression, panic disorder and social anxiety.Reference Hedman, Ljótsson and Lindefors1 It has been implemented in routine care in Sweden since 2007.Reference Hedman, Ljótsson, Rück, Bergström, Andersson and Kaldo2Reference El Alaoui, Hedman, Kaldo, Hesser, Kraepelien and Andersson4 However, depression and anxiety disorders are often comorbid.Reference Sundquist, Ohlsson, Sundquist and Kendler5

Individually tailored ICBTReference Johansson, Sjöberg, Sjögren, Johnsson, Carlbring and Andersson6 handles comorbidities within one treatment by prescribing different cognitive–behavioural therapy components matched to the individual's needs. This can manage comorbid conditions more optimally and lessen the need for thorough diagnostic procedures and undertaking several disorder-specific treatments.

One trialReference Johansson, Sjöberg, Sjögren, Johnsson, Carlbring and Andersson6 indicated no difference in reduction of depressive symptoms between individually tailored ICBT and disorder-specific ICBT. In that study, individually tailored treatment was even more effective than disorder-specific treatment in reducing depression in a subgroup with higher levels of depression and comorbidity. However, larger comparison groups are needed to explore the non-inferiority of individually tailored to disorder-specific ICBT. Also, the effects of individually tailored ICBT for depression on comorbid conditions have not been explored.

Our primary aim was to use the basic methodology of benchmarking to compare a new intervention with a more established one. We evaluate if the effects of an individually tailored ICBT programme (TAIL) on depressive symptoms among patients in TAIL with depression (TAIL-depression) were non-inferior to the effects of a disorder-specific ICBT for depression used in routine care (DS-depression). Secondary aims were to use the same benchmarking strategy to compare reduction in anxiety among patients in TAIL with either panic disorder (TAIL-panic) or social anxiety disorder (TAIL-social anxiety) with disorder-specific ICBT in routine care for patients with panic (DS-panic) or social anxiety (DS-social anxiety).


The treatment samples that served as the benchmarks of this study were clinical outcomes from disorder-specific ICBT within a routine care service at the internet psychiatry clinicReference Hedman, Ljótsson, Rück, Bergström, Andersson and Kaldo2Reference El Alaoui, Hedman, Kaldo, Hesser, Kraepelien and Andersson4 in Stockholm, Sweden from October 2007 to July 2017. Comorbid anxiety and depression in this sample, receiving disorder-specific ICBT for panic disorder, depression and social anxiety disorder, was 14, 25 and 15% respectively. The alternative, comparator treatment sample, was collated from individually tailored ICBT given at the same clinic, but within the context of a trial.Reference Hallgren, Kraepelien, Ojehagen, Lindefors, Zeebari and Kaldo7 Comorbid anxiety and depression was 67% in this sample. Details on the sample, trial methodology and clinical effects are found in previously reported results from the REGASSA study.Reference Hallgren, Kraepelien, Ojehagen, Lindefors, Zeebari and Kaldo7 In that study TAIL was superior to treatment as usual in primary care in reducing depression. Details on ethical registration, treatment content, inclusion criteria, diagnostic procedures and the study flow chart of the current study can be found in the supplementary Data 1 and supplementary Fig. 1 available at

The measures, employed at pre-treatment and post-treatment 12 weeks later in both treatment samples, include well-established disorder-specific self-report scales: the Montgomery–Åsberg Depression Rating Scale – Self rated (MADRS-S),Reference Svanborg and Asberg8 the Panic Disorder Severity Scale – Self report (PDSS-SR)Reference Houck, Spiegel, Shear and Rucci9 and the Liebowitz Social Anxiety Scale – Self-Rated (LSAS-SR).Reference Baker, Heinrichs, Kim and Hofmann10 Measurements were collected weekly in the disorder-specific treatment. In TAIL only MADRS-S was collected weekly whereas PDSS-SR and LSAS-SR were collected before and after treatment. Demographics and data on treatment use was collected from both TAIL and disorder-specific ICBT. Self-rated data from TAIL has not been previously published.

Clinical change within each group was estimated through longitudinal generalised estimating equations models, with each of the symptom outcomes modelled separately. These models tested for both differences in the symptom scores between groups at baseline, as well as for group differences in the rate of symptom reduction (group × time interaction) as an additional exploratory analysis.

In addition, in line with methodological recommendations for clinical measurement and reporting,Reference Ioannidis, Greenland, Hlatky, Khoury, Macleod and Moher11 demographical variables such as age, employment and relationship status variables were used to adjust the longitudinal models, and the test between groups. Specifically, those variables that illustrated a significant statistical association to either the overall symptom score, or the rate of improvement, were retained in the models that test for differences between groups. Additional information about the confounding variables, adjusted and unadjusted estimates of change is presented in supplementary data 1.

The estimated marginal means from the resulting longitudinal models were also used to determine within-group effect sizes (Cohen's d) and percentage score estimates that conveyed the clinical change through both treatment conditions. In addition, the frequency of individuals who demonstrated a clinical response, defined as a 50% reduction in symptoms, was tested for between groups through a logistic regression.

The main analysis of this paper employed a benchmarking strategy proposed by Minami and colleagues.Reference Minami, Serlin, Wampold, Kircher and Brown12 This procedure does not rely on the traditional statistical tests for differences described above. Instead, the within-group effect size of the treatment would be considered non-inferior, or ‘clinically equivalent’ using Minami and colleagues’ terminology, to the effect size of the benchmark treatment if the lower end of the 95% CI round the treatment mean effect size is at least as large as the benchmark mean effect size minus the minimally clinically relevant effect (Δ). This so called ‘good enough principle’, originally proposed by Serlin & Lapsey,Reference Serlin and Lapsley13 requires a value of Δ to be chosen to reflect the best available knowledge in the field. We chose to use Δ = 0.24 as the minimal clinically relevant effect following the empirically derived proposal for depression treatment by Cuijpers and colleagues.Reference Cuijpers, Turner, Koole, van Dijke and Smit14 Thus, if the effect of disorder-specific treatment minus 0.24, is lower than the lower end of the 95% CI of the effect of the individually tailored treatment, the treatment is considered non-inferior to the benchmark.


Demographical information and information about use of treatment is available in supplementary Table 1. Comparisons between the main groups of TAIL and disorder-specific treatment for depression demonstrated slight differences of age, employment rate and sent messages with no other significant differences between the groups.

Table 1 reports estimated symptom levels and statistical tests. Initial severity scores did not differ significantly (P = 0.28–0.68 as described in the similarities and differences section in the supplementary material). Supplementary Table 2 reports the adjusted estimates. Since there were no changes to the unadjusted test results, the unadjusted model is used as the main analysis.

Table 1 Comparison of disorder-specific and individually tailored treatments

DS, disorder-specific treatment; TAIL, individually tailored treatment; G × T, interaction effect of group and time from generalised estimating equations; Wχ 2, Wald chi-squared; d, Cohen's d; d – Δ, non-inferiority margin; MADRS-S, Montgomery–Åsberg Depression Rating Scale – Self-rated; PDSS-SR, Panic Disorder Severity Scale – Self report; LSAS-SR, Liebowitz Social Anxiety Scale – Self-Rated.

For the TAIL-depression group, the within-group effect size of d = 1.33 (95% CI 1.14–1.52) was above the non-inferiority margin of the DS-depression group (d = 1.35, d − Δ = 1.11). The effect size of the TAIL-panic group d = 1.11 (95% CI 0.66–1.56) was slightly lower than the effect of the DS-panic group and not above the non-inferiority margin (d = 1.47, d − Δ = 1.23). The effect size of the TAIL-social anxiety group was 0.74 (95% CI 0.27–1.21) and similar to the effect of the DS-social anxiety group, but the lower end of the confidence interval was not above the non-inferiority margin (d = 0.81, d − Δ = 0.57).

In the additional testing for differences in symptom reduction, no TAIL intervention was significantly different from any of the disorder-specific treatments, although TAIL-panic showed a close to significantly lower decrease of panic symptoms.


When benchmarking individually tailored ICBT to well-established disorder-specific ICBT programmes in routine care, we demonstrated that the effect of tailored treatment for patients with depression was non-inferior to disorder-specific treatment on depression. The effect on comorbid panic was at least of moderate size but non-inferiority could not be established. In addition, the effect size point estimate was 0.36 lower than the effect of the disorder-specific panic treatment, and the exploratory test for difference between TAIL-panic and DS-panic was near statistical significance (P = 0.06). The effects on comorbid social anxiety had a similar point estimate as the effects of disorder-specific social anxiety treatment but non-inferiority could not be established.

One limitation was the low power in the benchmark analyses for participants with panic or social anxiety, calling for future studies with larger samples. Also, some minor differences in sociodemographic factors and interaction during treatment were found, and the diagnostic procedures and the context (randomised controlled trial versus routine care) differed. However, it is not obvious which treatment would benefit overall from these differences and many important conditions were very similar, for example the use of the same technical platform, length of treatment, level and mode of support, therapist's competence and initial severity. The result was also robust to controlling for demographical variables. Another limitation was that the benchmarks were routine care treatments, whereas the context of the comparator intervention was that of a randomised controlled trial. However, these disorder-specific treatments were run in a context not very different from a clinical trial and in many ways similar to how the REGASSA study was carried out (see the similarities and differences-section in the supplementary material for comparison).

Individually tailored ICBT for patients with depression was identified as non-inferior to disorder-specific ICBT for depression. For the subgroup of patients with social anxiety disorder, the reductions in anxiety were comparable with those found in disorder-specific ICBT for social anxiety, although the low statistical power does not admit conclusions about non-inferiority. For panic disorder the disorder-specific ICBT reduces anxiety almost significantly more than individually tailored ICBT, and the difference in effect size point estimates is greater than the non-inferiority margin, giving a non-significant but suggestive notion that there could be a clinically relevant advantage of selecting disorder-specific ICBT when the patient has panic disorder. Therapist-time was also significantly greater in TAIL for patients with social anxiety and panic disorder, which suggests lower cost-effectiveness with regards to the effects on the patient's main problem.

The empirical findings in this study do not provide any suggestions on why depression might be more suitable to treat with individually tailored ICBT than anxiety, but in theory it is in line with the finding that a wide range of interventions seems to be equally effective (or ineffective) for depression,Reference Cuijpers, Andersson, Donker and van Straten15 indicating less need for specific treatment components, whereas this might not be the case for anxiety where, for example, exposure seems to be a technique of specific importance. If this is the case, and if tailored treatments might not focus enough on exposure exercises, this could be one of several explanations for our observed difference.

In conclusion, although more, highly powered, research on individually tailored ICBT for social anxiety and panic is needed, our findings support the use of individually tailored ICBT as an alternative for patients presenting with depression as the primary condition of concern. The increased use of individually tailored ICBT, where a patient is allowed to have more than one key treatment target, could lead to less need for highly precise diagnostic procedures or having to undertake several disorder-specific treatments.

Supplementary material

Supplementary material is available online at


1Hedman, E, Ljótsson, B, Lindefors, N. Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert Rev Pharmacoecon Outcomes Res 2012; 12: 745–64.CrossRefGoogle ScholarPubMed
2Hedman, E, Ljótsson, B, Rück, C, Bergström, J, Andersson, G, Kaldo, V, et al. Effectiveness of internet-based cognitive behaviour therapy for panic disorder in routine psychiatric care. Acta Psychiatr Scand 2013; 128: 457–67.CrossRefGoogle ScholarPubMed
3Hedman, E, Ljótsson, B, Kaldo, V, Hesser, H, El Alaoui, S, Kraepelien, M, et al. Effectiveness of internet-based cognitive behaviour therapy for depression in routine psychiatric care. J Affect Disord 2014; 155: 4958.CrossRefGoogle ScholarPubMed
4El Alaoui, S, Hedman, E, Kaldo, V, Hesser, H, Kraepelien, M, Andersson, E, et al. Effectiveness of internet-based cognitive-behavior therapy for social anxiety disorder in clinical psychiatry. J Consult Clin Psychol 2015; 83: 902–14.CrossRefGoogle ScholarPubMed
5Sundquist, J, Ohlsson, H, Sundquist, K, Kendler, KS. Common adult psychiatric disorders in Swedish primary care where most mental health patients are treated. BMC Psychiatry 2017; 17: 235.CrossRefGoogle ScholarPubMed
6Johansson, R, Sjöberg, E, Sjögren, M, Johnsson, E, Carlbring, P, Andersson, T, et al. Tailored vs. standardized internet-based cognitive behavior therapy for depression and comorbid symptoms: a randomized controlled trial. PLoS ONE 2012; 7: e36905.CrossRefGoogle ScholarPubMed
7Hallgren, M, Kraepelien, M, Ojehagen, A, Lindefors, N, Zeebari, Z, Kaldo, V, et al. Physical exercise and internet-based cognitive-behavioural therapy in the treatment of depression: randomised controlled trial. Br J Psychiatry 2015; 207: 227–34.CrossRefGoogle ScholarPubMed
8Svanborg, P, Asberg, M. A new self-rating scale for depression and anxiety states based on the Comprehensive Psychopathological Rating Scale. Acta Psychiatr Scand 1994; 89: 21–8.CrossRefGoogle ScholarPubMed
9Houck, PR, Spiegel, DA, Shear, MK, Rucci, P. Reliability of the self-report version of the Panic Disorder Severity Scale. Depress Anxiety 2002; 15: 183–5.CrossRefGoogle ScholarPubMed
10Baker, SL, Heinrichs, N, Kim, H-J, Hofmann, SG. The Liebowitz Social Anxiety Scale as a self-report instrument: a preliminary psychometric analysis. Behav Res Ther 2002; 40: 701–15.CrossRefGoogle ScholarPubMed
11Ioannidis, JPA, Greenland, S, Hlatky, MA, Khoury, MJ, Macleod, MR, Moher, D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet 2014; 383: 166–75.CrossRefGoogle ScholarPubMed
12Minami, T, Serlin, RC, Wampold, BE, Kircher, JC, Brown, GS. Using clinical trials to benchmark effects produced in clinical practice. Qual Quant 2008; 42: 513–25.CrossRefGoogle Scholar
13Serlin, RC, Lapsley, DK. Rationality in psychological research: the good-enough principle. Am Psychol 1985; 40: 7383.CrossRefGoogle Scholar
14Cuijpers, P, Turner, EH, Koole, SL, van Dijke, A, Smit, F. What is the threshold for a clinically relevant effect? The case of major depressive disorders. Depress Anxiety 2014; 31: 374–8.CrossRefGoogle ScholarPubMed
15Cuijpers, P, Andersson, G, Donker, T, van Straten, A. Psychological treatment of depression: results of a series of meta-analyses. Nord J Psychiatry 2011; 65: 354–64.CrossRefGoogle ScholarPubMed
Figure 0

Table 1 Comparison of disorder-specific and individually tailored treatments

Supplementary material: File

Kraepelien et al. supplementary material

Kraepelien et al. supplementary material 1

Download Kraepelien et al. supplementary material(File)
File 90 KB
Submit a response


No eLetters have been published for this article.