Hostname: page-component-7c8c6479df-p566r Total loading time: 0 Render date: 2024-03-19T04:30:31.229Z Has data issue: false hasContentIssue false

Does Internet-based guided-self-help for depression cause harm? An individual participant data meta-analysis on deterioration rates and its moderators in randomized controlled trials

Published online by Cambridge University Press:  20 September 2016

D. D. Ebert*
Affiliation:
Friedrich Alexander University Erlangen-Nuremberg, Erlangen, Germany
L. Donkin
Affiliation:
The Brain and Mind Research Institute, University of Sydney, NSW, Australia Department of Clinical, Neuro and Developmental Psychology, VU University Amsterdam, the Netherlands
G. Andersson
Affiliation:
Department of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden Department of Clinical Neuroscience, Karolinska Institute, Stockholm, Sweden
G. Andrews
Affiliation:
Clinical Research Unit for Anxiety and Depression, School of Psychiatry, University of New South Wales at St Vincent's Hospital, Darlinghurst, NSW, Australia
T. Berger
Affiliation:
Department of Clinical Psychology and Psychotherapy, University of Bern, Bern, Switzerland
P. Carlbring
Affiliation:
Department of Psychology, Stockholm University, Sweden
A. Rozenthal
Affiliation:
Department of Psychology, Stockholm University, Sweden
I. Choi
Affiliation:
Brain and Mind Centre, University of Sydney, Sydney, Australia
J. A. C. Laferton
Affiliation:
Psychologische Hochschule Berlin
R. Johansson
Affiliation:
Department of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden Department of Clinical Neuroscience, Karolinska Institute, Stockholm, Sweden
A. Kleiboer
Affiliation:
Department of Clinical, Neuro and Developmental Psychology, VU University Amsterdam, the Netherlands
A. Lange
Affiliation:
Department of Clinical Psychology, University of Amsterdam, Amsterdam, The Netherlands
D. Lehr
Affiliation:
Leuphana University Lüneburg, Germany
J. A. Reins
Affiliation:
Leuphana University Lüneburg, Germany
B. Funk
Affiliation:
Leuphana University Lüneburg, Germany
J. Newby
Affiliation:
Clinical Research Unit for Anxiety and Depression, School of Psychiatry, University of New South Wales at St Vincent's Hospital, Darlinghurst, NSW, Australia
S. Perini
Affiliation:
Centre for Emotional Health, Macquarie University, Australia
H. Riper
Affiliation:
Department of Clinical, Neuro and Developmental Psychology, VU University Amsterdam, the Netherlands
J. Ruwaard
Affiliation:
Department of Clinical, Neuro and Developmental Psychology, VU University Amsterdam, the Netherlands
L. Sheeber
Affiliation:
Oregon Research Institute, Eugene, Oregon, USA
F. J. Snoek
Affiliation:
Department of Medical Psychology, VU University Medical Center Amsterdam, The Netherlands Academic Medical Center/University of Amsterdam, The Netherlands
N. Titov
Affiliation:
eCentreClinic and MindSpot Clinic, Department of Psychology, Macquarie University, Australia
B. Ünlü Ince
Affiliation:
Department of Clinical, Neuro and Developmental Psychology, VU University Amsterdam, the Netherlands
K. van Bastelaar
Affiliation:
Department of Medical Psychology, VU University Medical Center Amsterdam, The Netherlands
K. Vernmark
Affiliation:
Department of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden Psykologpartners, Private Practice, Linköping, Sweden
A. van Straten
Affiliation:
Department of Clinical, Neuro and Developmental Psychology, VU University Amsterdam, the Netherlands
L. Warmerdam
Affiliation:
Department of Clinical, Neuro and Developmental Psychology, VU University Amsterdam, the Netherlands
N. Salsman
Affiliation:
School of Psychology, Xavier University, Cincinnati, USA
P. Cuijpers
Affiliation:
Department of Clinical, Neuro and Developmental Psychology, VU University Amsterdam, the Netherlands
*
*Address for correspondence: D. D. Ebert, Ph.D., Friedrich Alexander University Erlangen-Nuremberg, Erlangen, Germany. (Email: david.ebert@fau.de)
Rights & Permissions [Opens in a new window]

Abstract

Background

Almost nothing is known about the potential negative effects of Internet-based psychological treatments for depression. This study aims at investigating deterioration and its moderators within randomized trials on Internet-based guided self-help for adult depression, using an individual patient data meta-analyses (IPDMA) approach.

Method

Studies were identified through systematic searches (PubMed, PsycINFO, EMBASE, Cochrane Library). Deterioration in participants was defined as a significant symptom increase according to the reliable change index (i.e. 7.68 points in the CES-D; 7.63 points in the BDI). Two-step IPDMA procedures, with a random-effects model were used to pool data.

Results

A total of 18 studies (21 comparisons, 2079 participants) contributed data to the analysis. The risk for a reliable deterioration from baseline to post-treatment was significantly lower in the intervention v. control conditions (3.36 v. 7.60; relative risk 0.47, 95% confidence interval 0.29–0.75). Education moderated effects on deterioration, with patients with low education displaying a higher risk for deterioration than patients with higher education. Deterioration rates for patients with low education did not differ statistically significantly between intervention and control groups. The benefit–risk ratio for patients with low education indicated that 9.38 patients achieve a treatment response for each patient experiencing a symptom deterioration.

Conclusions

Internet-based guided self-help is associated with a mean reduced risk for a symptom deterioration compared to controls. Treatment and symptom progress of patients with low education should be closely monitored, as some patients might face an increased risk for symptom deterioration. Future studies should examine predictors of deterioration in patients with low education.

Type
Review Article
Copyright
Copyright © Cambridge University Press 2016 

Introduction

Major depressive disorder (MDD) is not only highly prevalent (Alonso et al. Reference Alonso, Angermeyer, Bernert, Bruffaerts, Brugha, Bryson, de Girolamo, Graaf, Demyttenaere, Gasquet, Haro, Katz, Kessler, Kovess, Lépine, Ormel, Polidori, Russo, Vilagut, Almansa, Arbabzadeh-Bouchez, Autonell, Bernal, Buist-Bouwman, Codony, Domingo-Salvany, Ferrer, Joo, Martínez-Alonso, Matschinger, Mazzi, Morgan, Morosini, Palacín, Romera, Taub and Vollebergh2004; Waraich et al. Reference Waraich, Goldner, Somers and Hsu2004; Kessler et al. Reference Kessler, Chiu, Demler, Merikangas and Walters2005; Wittchen et al. Reference Wittchen, Jacobi, Rehm, Gustavsson, Svensson, Jönsson, Olesen, Allgulander, Alonso, Faravelli, Fratiglioni, Jennum, Lieb, Maercker, van Os, Preisig, Salvador-Carulla, Simon and Steinhausen2011; Rozental et al. Reference Rozental, Andersson, Boettcher, Ebert, Cuijpers, Knaevelsrud, Ljótsson, Kaldo, Titov and Carlbring2014) but also associated with substantial impairment (Ustün et al. Reference Ustün, Ayuso-Mateos, Chatterji, Mathers and Murray2004; Saarni et al. Reference Saarni, Suvisaari, Sintonen, Pirkola, Koskinen, Aromaa and Lönnqvist2007) and economic costs (Berto et al. Reference Berto, D'Ilario, Ruffo, Di Virgilio and Rizzo2000; Greenberg & Birnbaum, Reference Greenberg and Birnbaum2005; Smit et al. Reference Smit, Cuijpers, Oostenbrink, Batelaan, de Graaf and Beekman2006).

Psychological treatments have been shown to be effective in the treatment of depression (Cuijpers et al. Reference Cuijpers, van Straten, Andersson and van Oppen2008a , Reference Cuijpers, Karyotaki, Weitz, Andersson, Hollon and van Straten2014). However, not all benefit from these treatments and many affected individuals remain untreated (Kohn et al. Reference Kohn, Saxena, Levav and Saraceno2004; Wittchen et al. Reference Wittchen, Jacobi, Rehm, Gustavsson, Svensson, Jönsson, Olesen, Allgulander, Alonso, Faravelli, Fratiglioni, Jennum, Lieb, Maercker, van Os, Preisig, Salvador-Carulla, Simon and Steinhausen2011).

Internet-based guided self-help interventions might be an acceptable (Cavanagh et al. Reference Cavanagh, Seccombe and Lidbetter2011), effective (Johansson & Andersson, Reference Johansson and Andersson2012; Richards & Richardson, Reference Richards and Richardson2012), and cost-effective (Hedman et al. Reference Hedman, Ljótsson and Lindefors2012) treatment alternative, that could provide treatment to individuals not reached so far (Ebert et al. Reference Ebert, Berking, Heber, Riper, Laferton, Cuijpers and Lehr2015a ). While researchers have consistently demonstrated positive effects of Internet-based guided self-help for depression both, for adults (Richards & Richardson, Reference Richards and Richardson2012) and youths (Ebert et al. Reference Ebert, Zarski, Christensen, Stikkelbroek, Cuijpers, Berking and Riper2015b ), little is known about potential negative effects of Internet-based psychological treatments for depression (Boettcher et al. Reference Boettcher, Rozental, Andersson and Carlbring2014; Ebert et al. Reference Ebert, Lehr, Baumeister, Boß, Riper, Cuijpers, Reins, Buntrock and Berking2014a ; Rozental et al. Reference Rozental, Andersson, Boettcher, Ebert, Cuijpers, Knaevelsrud, Ljótsson, Kaldo, Titov and Carlbring2014). This is not unique for Internet treatments, as limited data are also available regarding negative effects for psychotherapy in general (Barlow, Reference Barlow2010; Emmelkamp et al. Reference Emmelkamp, David, Beckers, Muris, Cuijpers, Lutz, Andersson, Araya, Banos Rivera, Barkham, Berking, Berger, Botella, Carlbring, Colom, Essau, Hermans, Hofmann, Knappe, Ollendick, Raes, Rief, Riper, Van Der Oord and Vervliet2014).

While in pharmacological outcome research the standard is to always evaluate both risks and benefits of an intervention (Willan et al. Reference Willan, O'Brien and Cook1997; Curtin & Schulz, Reference Curtin and Schulz2011) psychotherapy outcome research has so far mostly focused on treatment benefits only (Lilienfeld, Reference Lilienfeld2007; Dimidjian & Hollon, Reference Dimidjian and Hollon2010).

Among different potential negative effects of psychotherapy one particularly unfavorable outcome is deterioration of symptoms as a consequence of treatment. Evidence from uncontrolled psychotherapy outcome studies indicates that a substantial number of patients experience a symptom deterioration while being in psychotherapy. The proportion of patients with symptom deterioration in these uncontrolled studies range from 3% to 14% (Smith & Glass, Reference Smith and Glass1977; Mohr, Reference Mohr1995; Hansen et al. Reference Hansen, Lambert and Forman2006; Lambert et al. Reference Lambert, Whipple, Hawkins, Vermeersch, Nielsen and Smart2006). This phenomenon of ‘the deterioration effect’ has been noted even in the early years of psychotherapy research (Bergin, Reference Bergin1966; Garfield et al. Reference Garfield, Prager and Bergin1971).

With regard to Internet-based self-help treatments, it could be argued that such interventions may be associated with an even greater risk for symptom deterioration than face-to-face approaches. For example, for some individuals a self-help approach might not be intense enough (Kiluk et al. Reference Kiluk, Sugarman, Nich, Gibbons, Martino, Rounsaville and Carroll2011). Further, individuals might be overstrained by trying to apply psychotherapeutic self-help strategies. Some therapeutic techniques could be inappropriately implemented by participants without direct guidance from a therapist. These problems could result in a further aggravation of hopelessness in severely affected individuals. It could also be argued that in face-to-face treatments it is much easier to observe and react to early signs of deterioration than it is via the Internet. Another potential negative effect could be that self-help treatments could lead to a delayed help-seeking, which could result in a further deterioration of symptoms, if the initial low-intensity self-help treatment should be not sufficient.

Despite the fact that the topic of potential negative effects of both Internet-based treatments (Kiluk et al. Reference Kiluk, Sugarman, Nich, Gibbons, Martino, Rounsaville and Carroll2011; Boettcher et al. Reference Boettcher, Rozental, Andersson and Carlbring2014; Rozental et al. Reference Rozental, Andersson, Boettcher, Ebert, Cuijpers, Knaevelsrud, Ljótsson, Kaldo, Titov and Carlbring2014, Reference Rozental, Boettcher, Andersson, Schmidt and Carlbring2015; Bengtsson et al. Reference Bengtsson, Nordin and Carlbring2015) and face-to-face psychotherapy (Lilienfeld, Reference Lilienfeld2007; Barlow, Reference Barlow2010; Dimidjian & Hollon, Reference Dimidjian and Hollon2010; Linden, Reference Linden2013; Ladwig et al. Reference Ladwig, Rief and Nestoriuc2014) have recently gained attention in the literature, empirical evidence on potential negative effects drawn from randomized controlled trials (RCTs) is still almost absent (Lilienfeld, Reference Lilienfeld2007).

RCTs are the most reasonable approach to determine whether a treatment is differentially associated with a deterioration in functioning or an increase in symptomatology (Dimidjian & Hollon, Reference Dimidjian and Hollon2010). If a randomized trial shows that participants in the active condition show greater deterioration in functioning than those in the non-treatment control condition, one can confidently conclude that the deterioration was a consequence of therapy (Lilienfeld, Reference Lilienfeld2007). However, given that the number of people deteriorating during treatment is expected to be small, randomized trials are mostly underpowered to examine this research question adequately.

Moreover, RCTs evaluating psychological treatments seldom report the number of patients who deteriorated during treatment, thus it is not possible to investigate mean deterioration effects found in randomized trials and its moderators using traditional meta-analytical approaches. Consequently, to the best of our knowledge, there is no meta-analytical review on symptom deterioration and its moderators within RCTs evaluating Internet-based treatments or psychological interventions for depression in general. Given the increasing popularity of Internet-based treatments in healthcare systems worldwide (Andersson et al. Reference Andersson, Cuijpers, Carlbring, Riper and Hedman2014), there is a pressing need to evaluate potential deterioration effects of Internet-based treatments.

Individual participant data meta-analysis (IPDMA) can overcome some of the limitations of conventional meta-analysis at the study level (Clarke, Reference Clarke2005; Riley et al. Reference Riley, Lambert and Abo-Zaid2010). By collecting and pooling the primary data of individual trials, analyses can be conducted, which have not been reported in the original studies. Furthermore, trials designed to detect overall treatment effects have limited power to detect treatment ×subgroup interactions (Brookes et al. Reference Brookes, Whitely, Egger, Smith, Mulheran and Peters2004). By combining the primary data of multiple trials using an IPDMA approach, it is possible to obtain a large sample size with sufficient power to examine effects in relevant subgroups and identify moderators of outcome (Cooper & Patall, Reference Cooper and Patall2009).

Hence, the present study aims to investigate deterioration rates and moderators of deterioration within randomized trials on Internet-based guided self-help interventions for adult depression, using IPDMA. We also evaluated deterioration rates in a number of subgroups of interest.

Method

Identification and selection of studies

In this study, we included randomized trials in which the effects of an Internet-based guided self-help treatment were compared with a control or comparison group (waiting list, care-as-usual, other) in adults (aged ⩾18 years) with depression (established by diagnostic interview or elevated levels of depressive symptoms based on self-report measures). Studies were excluded if study participants were not currently in a depressive episode (e.g. if they were in remission), if the interventions were provided without guidance (i.e. without support from a therapist or other healthcare professional) in order to increase internal validity and to reduce potential heterogeneity, if the interventions were delivered to the individual via a group format or were delivered at a location that required the individual to travel to use the programme (e.g. a clinic). Co-morbid general medical or psychiatric disorders were not used as a study exclusion criterion. No language restrictions were applied. Fig. 1 shows the selection process for included studies.

Fig. 1. Flowchart inclusion of studies.

For the identification of potential studies for inclusion, we used a database of 1476 papers on the psychological treatment of depression that has been described in detail elsewhere (Cuijpers et al. Reference Cuijpers, van Straten, Warmerdam and Andersson2008b ). These searches covered papers published until January 2014 and in these searches we examined 14 164 abstracts in Pubmed (3638 abstracts), PsycInfo (2824), EMBASE (4682), and the Cochrane Central Register of Controlled Trials (3020). These abstracts were identified by combining terms indicative of psychological treatment and depression (both MeSH terms and text words). Further, the primary studies from 42 meta-analyses of psychological treatment for depression were checked to ensure that no published studies were missed. From the 14 164 abstracts (10 474 after removal of duplicates) 1476 full-text papers were retrieved for possible inclusion in the database.

Data collection, characteristics of included studies and participants

Corresponding authors were contacted for each of the identified papers and asked to provide raw data from their study. Of the 15 published studies identified from the database search, primary data was obtained from 14 (Andersson et al. Reference Andersson, Bergström, Holländare, Carlbring, Kaldo and Ekselius2005; van Straten et al. Reference van Straten, Cuijpers and Smits2008; Warmerdam et al. Reference Warmerdam, van Straten, Twisk, Riper and Cuijpers2008; Perini et al. Reference Perini, Titov and Andrews2009; Ruwaard et al. Reference Ruwaard, Schrieken, Schrijver, Broeksteeg, Dekker, Vermeulen and Lange2009; Vernmark et al. Reference Vernmark, Lenndin, Bjärehed, Carlsson, Karlsson, Oberg, Carlbring, Eriksson and Andersson2010; Titov et al. Reference Titov, Andrews, Davies, McIntyre, Robinson and Solley2010; Berger et al. Reference Berger, Hämmerli, Gubser, Andersson and Caspar2011; van Bastelaar et al. Reference van Bastelaar, Pouwer, Cuijpers, Riper and Snoek2011; Choi et al. Reference Choi, Zou, Titov, Dear, Li, Johnston, Andrews and Hunt2012; Johansson et al. Reference Johansson, Ekbladh, Hebert, Lindström, Möller, Petitt, Poysti, Larsson, Rousseau, Carlbring, Cuijpers and Andersson2012a , Reference Johansson, Sjöberg, Sjögren, Johnsson, Carlbring, Andersson, Rousseau and Andersson b ; Sheeber et al. Reference Sheeber, Seeley, Feil, Davis, Sorensen, Kosty and Lewinsohn2012; Ünlü Ince et al. Reference Ünlü Ince, Cuijpers, van ‘t Hof, van Ballegooijen, Christensen and Riper2013). Data for one study (Titov et al. Reference Titov, Dear, Schwencke, Andrews, Johnston, Craske and McEvoy2011) could not be obtained, as the dataset was no longer available to the Titov research team. The study that was not included did not differ from the other studies in terms of design, participants, intervention, or quality. We also asked all authors whether they were aware of other recently completed RCTs that met our inclusion criteria, but were not yet published. Four more studies were identified by this method, and the authors were all willing to contribute their primary data to this project (Carlbring et al. Reference Carlbring, Hägglund, Luthström, Dahlin, Kadowaki, Vernmark and Andersson2013; Newby et al. Reference Newby, Mackenzie, Williams, McIntyre, Watts, Wong and Andrews2013; Ebert et al. Reference Ebert, Lehr, Boß, Riper, Cuijpers, Andersson, Thiart, Heber and Berking2014c ; Kleiboer et al. Reference Kleiboer, Donker, Seekles, van Straten, Riper and Cuijpers2015). This process resulted in a dataset with the primary data from 18 RCTs including 2079 cases. These 18 randomized controlled studies included 21 comparisons between an Internet-based guided self-help group v. control condition from baseline to post-test, five comparisons in addition from baseline to follow-up I (1–4 months, mean = 2.44, s.d. = 1.09, range 1–4, n = 737 participants) and four comparisons from baseline to follow-up 2 (⩾6 months, mean = 6.96, s.d. = 1.7, range 6–10, n = 594 participants). If a study had three conditions there would be two comparisons (i.e. the active treatment condition with each of the two control conditions). Only one study provided data for both follow-up time points (Ebert et al. 2014b). Characteristics of each included study are described in Table 1. Detailed information on sociodemographic and clinical characteristics of study participants can be found in Table 2.

Table 1. Selected characteristics of randomized controlled studies examining the effects of Internet-based psychotherapies for depression in adults

Recr, Recruitment population; Comm, Community sample; Clin, Clinical Sample; Depression, confirmation of depression; CBT, cognitive behaviour therapy; ACT, acceptance and commitment therapy; PD, psychodynamic therapy; PST, problem-solving therapy; N mod, Number of modules in the intervention; WL, waiting list control; BDI, Beck Depression Inventory; CES-D, Centre for Epidemiology Studies Depression Scale; Qual, risk of bias Score; Publ, publication of result (0 = unpublished, 1 = published); SWE, Sweden; SWZ, Switzerland; GER, Germany; NL, The Netherlands; AU, Australia; USA, United States of America.

a In this column a positive or negative sign is given for four quality criteria, respectively: allocation sequence; concealment of allocation to conditions; blinding of assessors; and intention-to-treat analyses.

Table 2. Demographic and clinical characteristics

BDI, Beck Depression Inventory; CES-D, Centre for Epidemiological Studies Depression Scale.

a Percentages refer to those participants of studies who reported data.

b Medication: BDI and CES-D data refer to the imputed values.

c Studies that used the BDI as primary outcome did not assess follow-up I.

Risk of bias assessment

The validity of included studies was assessed using four criteria of the ‘Risk of Bias’ assessment tool, developed by the Cochrane Collaboration (Higgins et al. Reference Higgins, Altman, Sterne, Higgins and Green2011). This tool assesses possible sources of bias in randomized trials, including the adequate generation of allocation sequence; the concealment of allocation to conditions; the prevention of knowledge of the allocated intervention (masking of assessors); and dealing with incomplete outcome data (this was assessed as positive when intention-to-treat (ITT) analyses were conducted, meaning that all randomized participants were included in the analyses). Assessment of the quality was conducted independently by two assessors. Overall risk of bias was low. All studies reported an adequate sequence generation, and allocation to conditions by an independent (third) party. Sixteen studies reported blinding of outcome assessors or used only self-report outcomes, whereas five did not report blinding. All studies were coded as having handled missing data adequately, as ITT analyses were applied and missing data were imputed for all studies using multiple imputation. Sixteen studies met all four quality criteria, the remaining five studies met three of four criteria. Agreement between independent raters (P.C., L.D.) on the risk of bias was 95% across studies.

Missing data

Analyses were conducted according to the ITT principle. Missing data in the raw datasets were handled using multiple imputations (Schafer & Graham, Reference Schafer and Graham2002) with a Markov Chain Monte Carlo multivariate imputation algorithm (Missing data module in SPSS v. 20; IBM Corp., USA) and 100 estimations per missing value. For the imputation of the primary outcome depression severity, we used all complete participant and study characteristics (study identifier, intervention group, baseline depression score, age, sex, recruitment population, confirmation of depression diagnosis method, intervention type, country of study, bias score – and post-intervention depression score when imputing follow-up). We did not impute baseline predictors.

Calculating deterioration rates

All studies used either the Centre for Epidemiological Studies – Depression Scale (CES-D; Radloff, Reference Radloff1977), or the Beck Depression Inventory (BDI; Beck et al. Reference Beck, Ward, Mendelson, Mock and Erbaugh1961) as outcome measures. Where multiple depression measures were present, the BDI was coded as the primary outcome measure given that it was the most frequently used outcome measure across studies. For both measures we calculated deterioration and response rates according to the widely used reliable change index (RCI; Jacobson & Truax, Reference Jacobson and Truax1991). Participants whose scores from pre-treatment to post-treatment had RCIs below the cut point of −1.96 were considered to have experienced deterioration. A RCI of −1.96 is equivalent to increases of depression of 7.68 points on the CES-D; and 7.63 points on the BDI.

Analyses

Effects of Internet-based treatments on deterioration rates were calculated using the standard two-step IPDMA approach (Riley et al. Reference Riley, Lambert and Abo-Zaid2010). Thus, after calculating whether or not a participant deteriorated (yes/no) we calculated event rates for each study separately on the basis of the imputed data. Following this, pooled event rates across studies were calculated according to a random-effects model as implemented in the Comprehensive Meta-analysis software package version 2.2.021 (https://www.meta-analysis.com), accounting for clustering of both participants’ within-study and between-study heterogeneity (Abo-Zaid et al. Reference Abo-Zaid, Guo, Deeks, Debray, Steyerberg, Moons and Riley2013). We proceeded by calculating the relative risks for each study, and pooled the results across the studies using a random-effects DerSimonian–Laird model (DerSimonian & Laird, Reference DerSimonian and Laird1986). For all analyses we chose a random-effects model, as we expected considerable heterogeneity among the studies. If there were significant differences between the groups with regard to deterioration, response, and remission rates, we also calculated the number needed to harm (NNH) and/or the number needed to treat (NNT) and the associated 95% confidence intervals (CIs), compared to the control group. The NNH indicates the number of participants treated in the experimental condition for one extra person to demonstrate symptom deterioration as compared to the control group. We also calculated a benefit–risk ratio (Willan et al. Reference Willan, O'Brien and Cook1997), by dividing the NNH for one extra symptom deterioration through the NNT to achieve one response [response was also defined using the reliable change criteria, such that participants with a reliable positive change (+1.96 on the RCI) were considered responders]. This procedure is usually used within drug treatment research (Curtin & Schulz, Reference Curtin and Schulz2011), and quantifies the numbers of favourable outcomes achieved for each additional unfavorable outcome event incurred. Benefit–risk ratios were only calculated if there is a higher risk of deterioration in the intervention group as compared to the control group (Willan et al. Reference Willan, O'Brien and Cook1997).

Sensitivity analyses

To test the robustness of our findings, we also conducted a sensitivity analysis applying an alternative criterion for deterioration. We defined the alternative criterion for deterioration such that individuals whose depression scores at baseline increased by ⩾50% at follow-up were categorized as having experienced deterioration. This criterion refers to a relative change instead of to an absolute change in symptoms.

Multiple treatments within one study

There were three studies in which two treatments were compared with a single control group (Warmerdam et al. Reference Warmerdam, van Straten, Twisk, Riper and Cuijpers2008; Titov et al. Reference Titov, Andrews, Davies, McIntyre, Robinson and Solley2010; Johansson et al. Reference Johansson, Sjöberg, Sjögren, Johnsson, Carlbring, Andersson, Rousseau and Andersson2012b ). In these cases, we treated each comparison as a separate study, and we avoided double counting of controls by randomly assigning half the control participants to each comparison.

Heterogeneity

As a test of homogeneity of effect sizes, we calculated the I 2 statistic as an indicator of heterogeneity in percentages (Ioannidis et al. Reference Ioannidis, Patsopoulos and Evangelou2007). A value of 0% indicates no observed heterogeneity, and larger values indicate increasing heterogeneity, with 25% as low, 50% as moderate, and 75% as high heterogeneity. We calculated 95% CIs around relative risks (RRs), using the non-central χ²-based approach within the heterogi module for Stata (Orsini et al. Reference Orsini, Higgins, Bottai and Buchan2013). We also calculated the Q statistic, but only report whether this was significant.

Publication bias

Publication bias was tested by inspecting the funnel plot and by Egger's test (Egger et al. Reference Egger, Davey Smith, Schneider and Minder1997). We also applied Duval & Tweedie's trim-and-fill procedure (Duval & Tweedie, Reference Duval and Tweedie2000), which yields an estimate of the effect size after the publication bias has been taken into account (Borenstein et al. Reference Borenstein, Hedges, Higgins and Rothstein2009).

Subgroup analyses

We conducted a series of subgroup analyses. Pooling of the results was conducted according to the mixed-effects model. In this model, studies within subgroups are pooled with the random-effects model, while tests for significant differences between subgroups are conducted with the fixed-effects model. Subgroup analyses were only conducted for post-treatment data and not for follow-up data, as the sample sizes of follow-up datasets were not large enough to test for significant differences between subgroups. The following subgroups were investigated: Participant characteristics: sex (male/female); age group [adults (18–59 years), older adults (⩾60 years)]; education [low (up to high school), medium to high (high school degree or further education after high school)]; co-morbid anxiety disorder (yes/no); depression severity at baseline [mild to moderate (BDI < 29); severe (BDI ⩾29)]; depression severity at baseline subgroup analyses was only calculated for participants of studies using the BDI, as the CES-D does not have an established cut-off score for depression severity. Study characteristics: MDD confirmed using an established diagnostic interview (yes/no); recruitment (community, clinical setting); risk of bias score [low (4); some risk (<4)]; type of control group (non-active/active). Intervention characteristics: theoretical model of the intervention (CBT, other); number of modules (4–5, 6–7, 8–11).

Ethical standards

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

Results

Deterioration rates

Overall pooled reliable deterioration rates across measurements are summarised in Table 3. The risk for a reliable deterioration from baseline to post-treatment was significantly lower in the intervention v. control conditions (RR 0.47, 95% CI 0.29–0.75) and the NNT to avoid one additional deterioration was 43.21 (95% CI 25.83–132.10). Heterogeneity was zero. The risk of a deterioration from baseline to follow-up I (1–4 months) appeared to demonstrate a trend towards being lower in the intervention groups compared to the control groups (RR 0.47, 95% CI 0.20–1.42), although the difference did not reach statistical significance (p = 0.097). There were no significant differences between the groups in the relative risk of a deterioration from baseline to follow-up II (p = 0.72). Heterogeneity was zero at both follow-ups.

Table 3. Deterioration rates in Internet-based treatments of depression, sensitivity, and subgroup analyses

N co, Number of comparisons; ER, event rate (number of patients with reliable deterioration); IG, intervention group; CG, control group; RR, relative risk for a deterioration; RCI, Reliable Change Index; BDI, Beck Depression Inventory; CES-D, Centre for Epidemiological Studies Depression Scale; MDD, major depressive disorder; CBT, cognitive behavioural therapy.

a A negative number in the column NNT/NNH indicates numbers needed to harm. A positive number indicates numbers needed to treat.

b The p value indicates whether differences between subgroups are significant.

c 95% CI around I 2 cannot be calculated as there must be at least three studies with patients experiencing a reliable deterioration to calculate 95% CIs.

* p < 0.05, ** p < 0.01, *** p < 0.001, † p < 0.1.

Sensitivity and subgroup analyses are presented in Table 3. Applying the alternative deterioration criteria (50% symptom increase) resulted in slightly lower deterioration rates, but demonstrated a similar overall pattern.

Effects of Internet-based treatments compared to control conditions on deterioration rates at post-treatment were non-significant (p > 0.05) in 11 out of 25 subgroups tested and showed significantly better outcomes for those in Internet-based treatment in 14 out of 25 subgroups tested. RR was higher among those in Internet-based treatments than those in control conditions in only one subgroup (i.e. those with low education), although this difference was not significant (RR 1.72, NNH 30.3, 95% CI −26.32 to 9.62, p = 0.37).

Moderator of deterioration effects

Education level was also the only significant moderator of treatment effects on deterioration, such that there was significantly higher risk for deterioration for participants with lower levels of educational attainment compared to those with more education. All other differences between subgroups on deterioration rates were non-significant (p > 0.10).

Benefit–risk ratio

Participating in Internet-based treatments for depression was not associated with an increased risk for deterioration when compared to a control group. There was only one subgroup analysis (i.e. participants with low levels of education) in which the relative risk for a deterioration was higher in the intervention group compared to the controls, although this difference was non-significant. Nonetheless, analyses of response rates at post treatment for the subgroup of participants with low education showed that their responses to treatment were significantly higher in the intervention group compared to controls, with a relative risk of 1.91 and a NNT to achieve one additional response of 3.23) (Ebert et al. unpublished data). Dividing the NNH in order that one symptom deterioration occurs through the NNT to achieve one treatment response, results in a benefit–risk ratio of 9.38, indicating that 9.38 participants with low education achieved a treatment response compared to the control group for each participant experiencing a deterioration in symptoms.

Publication bias

Inspection of the funnel plot and Egger's test indicated some possible publication biases. However, adjustment for publication bias using Duval & Tweedie's trim-and-fill procedure did not result in substantial changes. After adjustment for missing studies (five imputed studies) the RR for deterioration by post-test was 0.58 (95% CI 0.38–0.90), NNT were 43.21 (95% CI 25.83–132.10). Results at both follow-ups stayed the same.

Discussion

This IPDMA evaluated deterioration rates of Internet-based guided self-help interventions for depression compared to control conditions in randomized trials. In addition, deterioration rates were evaluated in subgroups of interest and potential moderating effects were examined.

Results showed that overall deterioration rates were low and the risk of deterioration was significantly lower for participants in Internet-based guided self-help conditions compared to controls. Education significantly moderated the risk for deterioration such that participants with lower educational attainment displayed a higher risk of deterioration compared to participants with more education. Nonetheless, a risk–benefit ratio analysis indicated that also in the subgroup of participants with low education the likelihood of benefits of positive response to Internet-based treatment clearly outweigh the possible risk for deterioration.

To the best of our knowledge, the present study is the first meta-analysis that evaluated deterioration effects and its moderators in RCTs evaluating a psychological treatment. Observed deterioration effects are in the lower range (3.6%) of those found in observational studies of face-to-face psychotherapy (3–14%) (Strupp et al. Reference Strupp, Hadley and Gomes-Schwartz1977; Mohr, Reference Mohr1995; Hansen et al. Reference Hansen, Lambert and Forman2006; Lambert et al. Reference Lambert, Whipple, Hawkins, Vermeersch, Nielsen and Smart2006). In contrast to early indications of possible adverse effects of psychological treatments (Bergin, Reference Bergin1966; Garfield et al. Reference Garfield, Prager and Bergin1971), we did not find a deterioration effect as a consequence of therapy. Instead results indicate that participating in Internet-based guided self-help programmes is associated with a lower risk of deterioration (RR 0.47) relative to controls. This effect held for the overall group and most subgroups. However, education level was identified as a significant moderator, with low educated participants at a greater risk for a deterioration than highly educated participants. This finding corresponds to results from some randomized trials that found that lower educational attainment was associated with worse treatment outcomes compared to higher educated participants in Internet-based self-help interventions (Spek et al. Reference Spek, Nyklícek, Cuijpers and Pop2008; Warmerdam et al. Reference Warmerdam, Van Straten, Twisk and Cuijpers2013). An explanation for such findings may be, that some patients with a lower educational level experience difficulties in terms of understanding the treatment modules, as most self-help manuals require a quite advanced reading comprehension. That may, in turn, decrease their self-efficacy and create feelings of hopelessness. Although all trials involved some form of guidance, this kind of support might not be sufficient for some individuals to overcome the barrier of low education. A more intensive treatment modality, as seeing a therapist face-to-face instead, could potentially help these patients understand the treatment rationale, and, thus, result in (hypothetically) less deterioration (Martinez et al. Reference Martinez, Whitfield, Dafters and Williams2007). However, given that the topic of predictors of deterioration in psychotherapy has so far not been addressed in face-to-face psychotherapy, future studies should examine whether participants with a high risk for deterioration in Internet-interventions would be better suited for face-to-face psychotherapy. Another explanation may be that people with low education may also have other confounds (e.g. low income, poor physical health status, physical comorbidities, lower social support, less access to health services, etc.) which may either contribute to increased severity or lower ability to engage with the content/practice of skills from these programmes.

It should be noted, however, that deterioration rates of participants with low education (10%) were still in the range of those found in observational studies on face-to-face psychotherapy (3–14%, see above). Further, the benefit–risk ratio indicates that, in comparison to the control group, 9.38 patients with low education achieve a treatment response compared to the control group for each participant experiencing a deterioration in symptoms. It is also of note that with regard to response v. deterioration, a previously reported study using the same dataset did not find that education was a moderator of treatment response. Patients with low levels of education profited significantly and to almost the same extent (NNT = 3.23 for response) as patients with more education (NNT = 3.25 for response; Ebert et al. unpublished data). Thus, it is clear that most with low education experience response rather deterioration in Internet-based treatment. Therefore, low education alone should not be used to identify someone as high risk for deterioration and further research is needed to more specifically identify those who may be at high risk for deterioration.

When interpreting results from this study, several limitations need to be considered. First, the only negative effect evaluated was depression symptom deterioration. Other adverse effects may also occur and should be examined alongside RCTs in the context of Internet-based guided self-help interventions in the future. For example, providing less intensive treatment than necessary through a self-help intervention might lead to lower treatment expectation in participants who fail to achieve a treatment response (Ebert et al. Reference Ebert, Lehr, Baumeister, Boß, Riper, Cuijpers, Reins, Buntrock and Berking2014a ). Hence, although the present study did not find indications for harm of Internet-based guided self-help interventions, the study design does not allow to conclude an absence of harm. For a complete discussion on negative effects in Internet-based psychotherapy see Rozental et al. (Reference Rozental, Andersson, Boettcher, Ebert, Cuijpers, Knaevelsrud, Ljótsson, Kaldo, Titov and Carlbring2014) and for psychological treatments in general see Linden (Reference Linden2013). Future studies should examine other potential harmful effects of Internet-based treatments alongside RCTs. Second, given the limited number of studies that included a follow-up assessment, both for the intervention and the control condition, the analyses were underpowered to adequately conduct subgroup and moderator analyses at follow-up. Third, although the total number of participants was very high (2079), the low number of participants per subgroup did not allow for an examination of the association between the intervention and participant-characteristics within subgroups. For example, given the result for education as a moderator, future studies should investigate predictors of deterioration in the subgroup of participants with low levels of education in order to differentiate between those participants with low education with high chances for treatment success and participants with low education at high risk for failure. Moreover, all programmes were only examined in one randomized trial. Hence we were not able to investigate potential negative effects of specific programmes, which should be done in future studies. Fourth, the present study only investigated guided treatments for depression, hence we can not conclude anything about potential negative effects of unguided self-help treatments. Fifth, given the nature of IPDMA, the examined moderators of outcome were limited to those assessed in the original randomized trials. There might be other relevant moderators that have been not assessed.

The present study has relevant implications for both clinical practice and research. First, the differential results for moderators of effects on deterioration and treatment response indicate that the chances and risks for positive and negative change in psychological treatments might be two distinct constructs. Thus future studies on the differential effectiveness of psychological treatments should investigate moderators of both outcomes separately, instead of only evaluating the effects on mean change scores as is commonly done in psychotherapy research. Second, while many healthcare systems hesitate to implement Internet-based guided self-help approaches, the present study indicates, that such interventions are not associated with an increased risk for deterioration, but instead reduces the risk for a further aggravation of symptoms. Taken together with findings showing that such interventions have substantial positive effects on mean symptom improvement and on treatment response and remission (Johansson & Andersson, Reference Johansson and Andersson2012; Ebert et al. Reference Ebert, Zarski, Christensen, Stikkelbroek, Cuijpers, Berking and Riper2015b ) this further supports the need for dissemination of such treatments in routine mental health care. Nevertheless the moderator result for education indicates that a monitoring of participants with low education seems warranted, as they face an increased risk to deteriorate compared to participants with high education. Given that most participants with low education nevertheless achieve treatment response, and the mean effects on treatment response are comparable to those of participants with higher education, low education should not be used as an exclusion criteria in clinical practice. Instead, therapists should closely monitor the treatment and symptom progress in order to detect and react to early signs of a deterioration, e.g. by referral to more intensive treatment modalities. Given that the present study is the first study on deterioration rates in RCTs, one cannot conclude yet, whether these results are specific for Internet-based guided self-help intervention or whether such findings refer to psychological treatments for depression in general. Thus, future studies should evaluate deterioration rates and their moderators for face-to-face psychotherapy and should also compare overall and differential deterioration effects of internet-based and face-to-face psychotherapy.

The present study did not show any evidence for harm of Internet-based guided self-help interventions and indicates that such interventions reduce the risk for a symptom deterioration.

Acknowledgements

The European Union funded this study [EU EFRE: ZW6-67280119999, CCI 2007DE161PR001 & FP7 E-Compared HEALTH.2013.3.1-1: Comparative Effectiveness Research CER in health systems and health services interventions (603098)].

Declaration of Interest

None.

References

Abo-Zaid, G, Guo, B, Deeks, JJ, Debray, TPA, Steyerberg, EW, Moons, KGM, Riley, RD (2013). Individual participant data meta-analyses should not ignore clustering. Journal of Clinical Epidemiology 66, 865873.CrossRefGoogle Scholar
Alonso, J, Angermeyer, MC, Bernert, S, Bruffaerts, R, Brugha, TS, Bryson, H, de Girolamo, G, Graaf, R, Demyttenaere, K, Gasquet, I, Haro, JM, Katz, SJ, Kessler, RC, Kovess, V, Lépine, JP, Ormel, J, Polidori, G, Russo, LJ, Vilagut, G, Almansa, J, Arbabzadeh-Bouchez, S, Autonell, J, Bernal, M, Buist-Bouwman, MA, Codony, M, Domingo-Salvany, A, Ferrer, M, Joo, SS, Martínez-Alonso, M, Matschinger, H, Mazzi, F, Morgan, Z, Morosini, P, Palacín, C, Romera, B, Taub, N, Vollebergh, WAM (2004). Prevalence of mental disorders in Europe: results from the European Study of the Epidemiology of Mental Disorders (ESEMeD) project. Acta Psychiatrica Scandinavica (Suppl.) , 420, 2127.Google Scholar
Andersson, G, Bergström, J, Holländare, F, Carlbring, P, Kaldo, V, Ekselius, L (2005). Internet-based self-help for depression: randomised controlled trial. British Journal of Psychiatry 187, 456461.CrossRefGoogle ScholarPubMed
Andersson, G, Cuijpers, P, Carlbring, P, Riper, H, Hedman, E (2014). Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry 13, 288295.Google Scholar
Barlow, DH (2010). Negative effects from psychological treatments: a perspective. American Psychologist 65, 1320.Google Scholar
Beck, AT, Ward, CH, Mendelson, M, Mock, J, Erbaugh, J (1961). An inventory for measuring depression. Archives of GeneralPpsychiatry 4, 561571.Google Scholar
Bengtsson, J, Nordin, S, Carlbring, P (2015). Therapists’ experiences of conducting cognitive behavioural therapy online vis-à-vis face-to-face. Cognitive Behaviour Therapy 44, 470479.CrossRefGoogle ScholarPubMed
Berger, T, Hämmerli, K, Gubser, N, Andersson, G, Caspar, F (2011). Internet-based treatment of depression: a randomized controlled trial comparing guided with unguided self-help. Cognitive Behaviour Therapy 40, 251266.CrossRefGoogle ScholarPubMed
Bergin, AE (1966). Some implications of psychotherapy research for therapeutic practice. Journal of Abnormal Psychology 71, 235246.Google Scholar
Berto, P, D'Ilario, D, Ruffo, P, Di Virgilio, R, Rizzo, F (2000). Depression: cost-of-illness studies in the international literature, a review. Journal of Mental Health Policy and Economics 3, 310.3.0.CO;2-H>CrossRefGoogle ScholarPubMed
Boettcher, J, Rozental, A, Andersson, G, Carlbring, P (2014). Side effects in internet-based interventions for social anxiety disorder. Internet Interventions 1, 311.CrossRefGoogle Scholar
Borenstein, M, Hedges, LV, Higgins, JPT, Rothstein, HR (2009). Introduction to Meta-Analysis. Wiley: Chichester, UK.Google Scholar
Brookes, ST, Whitely, E, Egger, M, Smith, GD, Mulheran, PA, Peters, TJ (2004). Subgroup analyses in randomized trials: risks of subgroup-specific analyses; power and sample size for the interaction test. Journal of Clinical Epidemiology 57, 229236.CrossRefGoogle ScholarPubMed
Carlbring, P, Hägglund, M, Luthström, A, Dahlin, M, Kadowaki, Å, Vernmark, K, Andersson, G (2013). Internet-based behavioral activation and acceptance-based treatment for depression: a randomized controlled trial. Journal of Affective Disorders 148, 331337.Google Scholar
Cavanagh, K, Seccombe, N, Lidbetter, N (2011). The implementation of computerized cognitive behavioural therapies in a service user-led, third sector self-help clinic. Behavioural and Cognitive Psychotherapy 39, 427442.CrossRefGoogle Scholar
Choi, I, Zou, J, Titov, N, Dear, BF, Li, S, Johnston, L, Andrews, G, Hunt, C (2012). Culturally attuned Internet treatment for depression amongst Chinese Australians: a randomised controlled trial. Journal of Affective Disorders 136, 459468.CrossRefGoogle ScholarPubMed
Clarke, MJ (2005). Individual patient data meta-analyses. Best Practice and Research. Clinical Obstetrics and Gynaecology 19, 4755.CrossRefGoogle ScholarPubMed
Cooper, H, Patall, EA (2009). The relative benefits of meta-analysis conducted with individual participant data versus aggregated data. Psychological Methods 14, 165176.CrossRefGoogle ScholarPubMed
Cuijpers, P, Karyotaki, E, Weitz, E, Andersson, G, Hollon, SD, van Straten, A (2014). The effects of psychotherapies for major depression in adults on remission, recovery and improvement: a meta-analysis. Journal of Affective Disorders 159, 118126.Google Scholar
Cuijpers, P, van Straten, A, Andersson, G, van Oppen, P (2008 a). Psychotherapy for depression in adults: a meta-analysis of comparative outcome studies. Journal of Consulting and Clinical Psychology 76, 909922.Google Scholar
Cuijpers, P, van Straten, A, Warmerdam, L, Andersson, G (2008 b). Psychological treatment of depression: a meta-analytic database of randomized studies. BMC Psychiatry 8, 36.Google Scholar
Curtin, F, Schulz, P (2011). Assessing the benefit: risk ratio of a drug–randomized and naturalistic evidence. Dialogues in Clinical Neuroscience 13, 183190.Google Scholar
DerSimonian, R, Laird, N (1986). Meta-analysis in clinical trials. Controlled Clinical Trials 7, 177188.Google Scholar
Dimidjian, S, Hollon, SD (2010). How would we know if psychotherapy were harmful? American Psychologist 1, 2133.CrossRefGoogle Scholar
Duval, S, Tweedie, R (2000). Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56, 455463.Google Scholar
Ebert, DD, Berking, M, Heber, E, Riper, H, Laferton, J, Cuijpers, P, Lehr, D (2015 a). Restoring depleted resources: efficacy and mechanisms of change of an Internet-based unguided recovery training for better sleep and psychological detachment from work. Health Psychology 34 (Suppl.), 12401251.Google Scholar
Ebert, DD, Lehr, D, Baumeister, H, Boß, L, Riper, H, Cuijpers, P, Reins, JA, Buntrock, C, Berking, M (2014 a). GET.ON Mood Enhancer: efficacy of Internet-based guided self-help compared to psychoeducation for depression: an investigator-blinded randomised controlled trial. Trials 15, 39.CrossRefGoogle ScholarPubMed
Ebert, DD, Lehr, D, Boß, L, Riper, H, Cuijpers, P, Andersson, G, Thiart, H, Heber, E, Berking, M (2014 c). Efficacy of an internet-based problem-solving training for teachers: results of a randomized controlled trial. Scandinavian Journal of Work, Environment and Health 40, 582596.Google Scholar
Ebert, DD, Zarski, A-C, Christensen, H, Stikkelbroek, Y, Cuijpers, P, Berking, M, Riper, H (2015 b). Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS ONE 10, e0119895.Google Scholar
Egger, M, Davey Smith, G, Schneider, M, Minder, C (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal (Clinical Research Edition) 315, 629634.Google Scholar
Emmelkamp, PMG, David, D, Beckers, T, Muris, P, Cuijpers, P, Lutz, W, Andersson, G, Araya, R, Banos Rivera, RM, Barkham, M, Berking, M, Berger, T, Botella, C, Carlbring, P, Colom, F, Essau, C, Hermans, D, Hofmann, SG, Knappe, S, Ollendick, TH, Raes, F, Rief, W, Riper, H, Van Der Oord, S, Vervliet, B (2014). Advancing psychotherapy and evidence-based psychological interventions. International Journal of Methods in Psychiatric Research 23 (Suppl. 1), 5891.Google Scholar
Garfield, SL, Prager, RA, Bergin, AE (1971). Evaluating outcome in psychotherapy: a hardy perennial. Journal of Consulting and Clinical Psychology 37, 320322.Google Scholar
Greenberg, PE, Birnbaum, HG (2005). The economic burden of depression in the US: societal and patient perspectives. Expert Opinion on Pharmacotherapy 6, 369376.Google Scholar
Hansen, NB, Lambert, MJ, Forman, EM (2006). The psychotherapy dose-response effect and its implications for treatment delivery services. Clinical Psychology: Science and Practice 9, 329343.Google Scholar
Hedman, E, Ljótsson, B, Lindefors, N (2012). Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert Review of Pharmacoeconomics and Outcomes Research 12, 745764.CrossRefGoogle ScholarPubMed
Higgins, JPT, Altman, DG, Sterne, JAC (eds) (2011). Chapter 8: Assessing risk of bias in included studies. In: Cochrane Handbook for Systematic Reviews of Interventions, version 5.1.0 (updated March 2011), (updated March 2011) (ed. Higgins, J. P. T. & Green, S.). The Cochrane Collaboration, 2011 (www.cochrane-handbook.org). The Atrium, Southern Gate, Chichester.Google Scholar
Ioannidis, JPA, Patsopoulos, NA, Evangelou, E (2007). Uncertainty in heterogeneity estimates in meta-analyses. British Medical Journal (Clinical Research Edition) 335, 914916.Google Scholar
Jacobson, NS, Truax, P (1991). Clinical significance: a statistical approach to denning meaningful change in psychotherapy research. Psychology 59, 1219.Google Scholar
Johansson, R, Andersson, G (2012). Internet-based psychological treatments for depression. Expert Review of Neurotherapeutics 12, 861869.Google Scholar
Johansson, R, Ekbladh, S, Hebert, A, Lindström, M, Möller, S, Petitt, E, Poysti, S, Larsson, MH, Rousseau, A, Carlbring, P, Cuijpers, P, Andersson, G (2012 a). Psychodynamic guided self-help for adult depression through the internet: a randomised controlled trial. PLoS ONE 7, e38021.CrossRefGoogle ScholarPubMed
Johansson, R, Sjöberg, E, Sjögren, M, Johnsson, E, Carlbring, P, Andersson, T, Rousseau, A, Andersson, G (2012 b). Tailored vs. standardized internet-based cognitive behavior therapy for depression and comorbid symptoms: a randomized controlled trial. PLoS ONE 7, e36905.Google Scholar
Kessler, RC, Chiu, WT, Demler, O, Merikangas, KR, Walters, EE (2005). Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry 62, 617627.Google Scholar
Kiluk, BD, Sugarman, DE, Nich, C, Gibbons, CJ, Martino, S, Rounsaville, BJ, Carroll, KM (2011). A methodological analysis of randomized clinical trials of computer-assisted therapies for psychiatric disorders: toward improved standards for an emerging field. American Journal of Psychiatry 168, 790799.CrossRefGoogle ScholarPubMed
Kleiboer, A, Donker, T, Seekles, W, van Straten, A, Riper, H, Cuijpers, P (2015). A randomized controlled trial on the role of support in internet-based problem solving therapy for depression and anxiety. Behaviour Research and Therapy 72, 6371.Google Scholar
Kohn, R, Saxena, S, Levav, I, Saraceno, B (2004). The treatment gap in mental health care. Bulletin of the World Health Organization 82, 858866.Google Scholar
Ladwig, I, Rief, W, Nestoriuc, Y (2014). Welche Risiken und Nebenwirkungen hat Psychotherapie? – Entwicklung des Inventars zur Erfassung Negativer Effekte von Psychotherapie (INEP). Verhaltenstherapie 24, 252263.Google Scholar
Lambert, MJ, Whipple, JL, Hawkins, EJ, Vermeersch, DA, Nielsen, SL, Smart, DW (2006). Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clinical Psychology: Science and Practice 10, 288301.Google Scholar
Lilienfeld, SO (2007). Psychological treatments that cause harm. Perspectives on Psychological Science 2, 5370.CrossRefGoogle ScholarPubMed
Linden, M (2013). How to define, find and classify side effects in psychotherapy: from unwanted events to adverse treatment reactions. Clinical Psychology and Psychotherapy 20, 286296.Google Scholar
Martinez, R, Whitfield, G, Dafters, R, Williams, C (2007). Can people read self-help manuals for depression? a challenge for the stepped care model and book prescription schemes. Behavioural and Cognitive Psychotherapy 36, 8997.Google Scholar
Mohr, DC (1995). Negative outcome in psychotherapy: a critical review. Clinical Psychology: Science and Practice 2, 127.Google Scholar
Newby, JM, Mackenzie, A, Williams, AD, McIntyre, K, Watts, S, Wong, N, Andrews, G (2013). Internet cognitive behavioural therapy for mixed anxiety and depression: a randomized controlled trial and evidence of effectiveness in primary care. Psychological Medicine 43, 26352648.Google Scholar
Orsini, N, Higgins, J, Bottai, M, Buchan, I (2013). Heterogi: Stata module to quantify heterogeneity in a meta-analysis.Google Scholar
Perini, S, Titov, N, Andrews, G (2009). Clinician-assisted Internet-based treatment is effective for depression: randomized controlled trial. Australian and New Zealand Journal of Psychiatry 43, 571578.Google Scholar
Radloff, LS (1977). The CES-D scale: a self-report depression scale for research in the general population. Applied Psychological Measurement 1, 385401.Google Scholar
Richards, D, Richardson, T (2012). Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clinical Psychology Review 32, 329342.Google Scholar
Riley, RD, Lambert, PC, Abo-Zaid, G (2010). Meta-analysis of individual participant data: rationale, conduct, and reporting. British Medical Journal (Clinical Research Edition) 340, c221.CrossRefGoogle ScholarPubMed
Rozental, A, Andersson, G, Boettcher, J, Ebert, DD, Cuijpers, P, Knaevelsrud, C, Ljótsson, B, Kaldo, V, Titov, N, Carlbring, P (2014). Consensus statement on defining and measuring negative effects of Internet interventions. Internet Interventions 1, 1219.Google Scholar
Rozental, A, Boettcher, J, Andersson, G, Schmidt, B, Carlbring, P (2015). Negative effects of internet interventions: a qualitative content analysis of patients’ experiences with treatments delivered online. Cognitive Behaviour Therapy 44, 223236.Google Scholar
Ruwaard, J, Schrieken, B, Schrijver, M, Broeksteeg, J, Dekker, J, Vermeulen, H, Lange, A (2009). Standardized web-based cognitive behavioural therapy of mild to moderate depression: a randomized controlled trial with a long-term follow-up. Cognitive Behaviour Therapy 38, 206221.Google Scholar
Saarni, SI, Suvisaari, J, Sintonen, H, Pirkola, S, Koskinen, S, Aromaa, A, Lönnqvist, J (2007). Impact of psychiatric disorders on health-related quality of life: general population survey. British Journal of Psychiatry 190, 326332.CrossRefGoogle ScholarPubMed
Schafer, JL, Graham, JW (2002). Missing data: our view of the state of the art. Psychological Methods 7, 147177.Google Scholar
Sheeber, LB, Seeley, JR, Feil, EG, Davis, B, Sorensen, E, Kosty, DB, Lewinsohn, PM (2012). Development and pilot evaluation of an Internet-facilitated cognitive-behavioral intervention for maternal depression. Journal of Consulting and Clinical Psychology 80, 739749.Google Scholar
Smit, F, Cuijpers, P, Oostenbrink, J, Batelaan, N, de Graaf, R, Beekman, A (2006). Costs of nine common mental disorders: implications for curative and preventive psychiatry. Journal of Mental Health Policy and Economics 9, 193200.Google Scholar
Smith, ML, Glass, GV (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist 32, 752760.Google Scholar
Spek, V, Nyklícek, I, Cuijpers, P, Pop, V (2008). Predictors of outcome of group and internet-based cognitive behavior therapy. Journal of Affective Disorders 105, 137145.Google Scholar
Strupp, HH, Hadley, SW, Gomes-Schwartz, B (1977). Psychotherapy for Better or Worse: the Problem of Negative Effects. Jason Aronson Inc. Publishers. New York.Google Scholar
Titov, N, Andrews, G, Davies, M, McIntyre, K, Robinson, E, Solley, K (2010). Internet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance. PLoS ONE 5, e10939.Google Scholar
Titov, N, Dear, BF, Schwencke, G, Andrews, G, Johnston, L, Craske, MG, McEvoy, P (2011). Transdiagnostic internet treatment for anxiety and depression: a randomised controlled trial. Behaviour Research and Therapy 49, 441452.Google Scholar
Ünlü Ince, B, Cuijpers, P, van ‘t Hof, E, van Ballegooijen, W, Christensen, H, Riper, H (2013). Internet-based, culturally sensitive, problem-solving therapy for Turkish migrants with depression: randomized controlled trial. Journal of Medical Internet Research 15, e227.Google Scholar
Ustün, TB, Ayuso-Mateos, JL, Chatterji, S, Mathers, C, Murray, CJL (2004). Global burden of depressive disorders in the year 2000. British Journal of Psychiatry: the Journal of Mental Science 184, 386392.Google Scholar
van Bastelaar, KMP, Pouwer, F, Cuijpers, P, Riper, H, Snoek, FJ (2011). Web-based depression treatment for type 1 and type 2 diabetic patients: a randomized, controlled trial. Diabetes Care 34, 320325.Google Scholar
van Straten, A, Cuijpers, P, Smits, N (2008). Effectiveness of a web-based self-help intervention for symptoms of depression, anxiety, and stress: randomized controlled trial. Journal of Medical Internet Research 10, e7.Google Scholar
Vernmark, K, Lenndin, J, Bjärehed, J, Carlsson, M, Karlsson, J, Oberg, J, Carlbring, P, Eriksson, T, Andersson, G (2010). Internet administered guided self-help versus individualized e-mail therapy: a randomized trial of two versions of CBT for major depression. Behaviour Research and Therapy 48, 368376.Google Scholar
Waraich, P, Goldner, EM, Somers, JM, Hsu, L (2004). Prevalence and incidence studies of mood disorders: a systematic review of the literature. Canadian Journal of Psychiatry 49, 124138.Google Scholar
Warmerdam, L, Van Straten, A, Twisk, J, Cuijpers, P (2013). Predicting outcome of Internet-based treatment for depressive symptoms. Psychotherapy Research: Journal of the Society for Psychotherapy Research 23, 559567.Google Scholar
Warmerdam, L, van Straten, A, Twisk, J, Riper, H, Cuijpers, P (2008). Internet-based treatment for adults with depressive symptoms: randomized controlled trial. Journal of Medical Internet Research 10, e44.Google Scholar
Willan, AR, O'Brien, BJ, Cook, DJ (1997). Benefit-risk ratios in the assessment of the clinical evidence of a new therapy. Controlled Clinical Trials 18, 121130.Google Scholar
Wittchen, H-U, Jacobi, F, Rehm, J, Gustavsson, A, Svensson, M, Jönsson, B, Olesen, J, Allgulander, C, Alonso, J, Faravelli, C, Fratiglioni, L, Jennum, P, Lieb, R, Maercker, A, van Os, J, Preisig, M, Salvador-Carulla, L, Simon, R, Steinhausen, H-C (2011). The size and burden of mental disorders and other disorders of the brain in Europe 2010. European Neuropsychopharmacology 21, 655679.Google Scholar
Figure 0

Fig. 1. Flowchart inclusion of studies.

Figure 1

Table 1. Selected characteristics of randomized controlled studies examining the effects of Internet-based psychotherapies for depression in adults

Figure 2

Table 2. Demographic and clinical characteristics

Figure 3

Table 3. Deterioration rates in Internet-based treatments of depression, sensitivity, and subgroup analyses