Introduction
Major depressive disorder (MDD) is a highly prevalent disorder (Gutiérrez-Rojas, Porras-Segovia, Dunne, Andrade-González, & Cervilla, Reference Gutiérrez-Rojas, Porras-Segovia, Dunne, Andrade-González and Cervilla2020) associated with great individual (Ferrari et al., Reference Ferrari, Charlson, Norman, Patten, Freedman, Murray and Whiteford2013) and societal burden (World Health Organization, 2022). Psychological treatments such as cognitive behavioral therapy (CBT) are the first-line treatments for depression (National Institute for Health and Care Excellence, Reference National Institute for Health and Care Excellence2022) and have the potential to prevent MDD onset (Cuijpers et al., Reference Cuijpers, Pineda, Quero, Karyotaki, Struijs, Figueroa and Muñoz2021b). However, the uptake of psychological interventions remains low, even in high-income countries, where only 28% of individuals in need of treatment receiving it (Chisholm et al., Reference Chisholm, Sweeny, Sheehan, Rasmussen, Smit, Cuijpers and Saxena2016) and less than 1% use indicated preventive interventions (Cuijpers, van Straten, Warmerdam, & van Rooy, Reference Cuijpers, van Straten, Warmerdam and van Rooy2010). New approaches to increase the uptake of mental health interventions are required to reduce the overall depression burden (Cuijpers, Reference Cuijpers2021).
Structural barriers to healthcare access can be addressed using the internet (Ebert et al., Reference Ebert, Van Daele, Nordgreen, Karekla, Compare, Zarbo and Baumeister2018), but especially in high-income countries, attitudinal barriers, including a common preference to solve one's own problems and the perceived stigma of mental illness, are important barriers for treatment uptake (Andrade et al., Reference Andrade, Alonso, Mneimneh, Wells, Al-Hamzawi, Borges and Kessler2014; Clement et al., Reference Clement, Schauman, Graham, Maggioni, Evans-Lacko, Bezborodovs and Thornicroft2015). An indirect approach to depression prevention and treatment (Cuijpers, Reference Cuijpers2021) using (guided) online self-help interventions may be a promising alternative to overcome these barriers. Instead of focusing on depression, the idea of an indirect approach is to target mental health problems contributing to depression or that are frequently comorbid, such as low self-esteem, procrastination (Cuijpers et al., Reference Cuijpers, Smit, Aalten, Batelaan, Klein, Salemink and Karyotaki2021c), and stress (Harrer et al., Reference Harrer, Apolinário-Hagen, Fritsche, Salewski, Zarski, Lehr and Ebert2021; Weisel et al., Reference Weisel, Lehr, Heber, Zarski, Berking, Riper and Ebert2018), or that are less stigmatizing, such as insomnia (van der Zweerde, van Straten, Effting, Kyle, & Lancee, Reference van der Zweerde, van Straten, Effting, Kyle and Lancee2019). By addressing these problems, depressive symptom severity (DSS) may be reduced but interventions might face more acceptance and match participants' perceived needs better. Insomnia is a particularly promising target as it is an impairing and burdensome disorder even in the absence of depressive symptoms (Morin et al., Reference Morin, Drake, Harvey, Krystal, Manber, Riemann and Spiegelhalder2015; Roach et al., Reference Roach, Juday, Tuly, Chou, Jena and Doghramji2021; Wade, Reference Wade2010) and can be effectively treated with specialized CBT for insomnia in person or via the internet (henceforth termed iCBT-I) (Feng, Han, Li, Geng, & Miao, Reference Feng, Han, Li, Geng and Miao2020; Simon et al., Reference Simon, Steinmetz, Feige, Benz, Spiegelhalder and Baumeister2023; Ye et al., Reference Ye, Chen, Chen, Liu, Lin, Liu and Jiang2016). Insomnia is also a predictor of new and recurrent depressive episodes (Baglioni et al., Reference Baglioni, Battagliese, Feige, Spiegelhalder, Nissen, Voderholzer and Riemann2011; Li, Wu, Gan, Qu, & Lu, Reference Li, Wu, Gan, Qu and Lu2016), is comorbid in 38%– 83% of depression cases (Bjorvatn, Olufsen, & Sørensen, Reference Bjorvatn, Olufsen and Sørensen2019; Staner, Reference Staner2010; Stewart et al., Reference Stewart, Besset, Bebbington, Brugha, Lindesay, Jenkins and Meltzer2006), and often persists after depression treatment (Vargas & Perlis, Reference Vargas and Perlis2020).
Emerging evidence shows that iCBT-I can effectively reduce DSS in insomnia patients with subthreshold depression (Batterham et al., Reference Batterham, Christensen, Mackinnon, Gosling, Thorndike, Ritterband and Griffiths2017; Cheng et al., Reference Cheng, Luik, Fellman-Couture, Peterson, Joseph, Tallent and Drake2019a; Christensen et al., Reference Christensen, Batterham, Gosling, Ritterband, Griffiths, Thorndike and Mackinnon2016; van der Zweerde et al., Reference van der Zweerde, van Straten, Effting, Kyle and Lancee2019) and in cases with comorbid clinical depression (Blom et al., Reference Blom, Jernelöv, Kraepelien, Bergdahl, Jungmarker, Ankartjärn and Kaldo2015; Blom, Jernelöv, Rück, Lindefors, & Kaldo, Reference Blom, Jernelöv, Rück, Lindefors and Kaldo2017). However, evidence that iCBT-I can prevent onset of new depressive episodes is ambiguous. One study found a preventive effect of iCBT-I on self-reported depression onset after 12 months compared to sleep education (Cheng et al., Reference Cheng, Kalmbach, Kalmbach, Tallent, Joseph, Espie and Drake2019a), but another did not find a preventive effect at 6-month follow-up compared to an active control group (internet-based placebo control program) using diagnostic interviews (Christensen et al., Reference Christensen, Batterham, Gosling, Ritterband, Griffiths, Thorndike and Mackinnon2016). Only one study investigated treatment moderators and identified baseline depression severity but no sociodemographic variables as moderators (Cheng et al., Reference Cheng, Luik, Fellman-Couture, Peterson, Joseph, Tallent and Drake2019b).
More insight of the potential of this indirect approach in prevention and treatment can be gained from focusing on one specific intervention (with well-known components) (Riley, Lambert, & Abo-Zaid, Reference Riley, Lambert and Abo-Zaid2010) used in different populations to allow for a greater precision and guide recommendations for researchers and clinicians by revealing who might profit most from it by reveling subgroups based on participant characteristics. Therefore, we performed an individual-participant-data (IPD) meta-analysis that focuses on the web-based insomnia intervention GET.ON Recovery, which is based on classic CBT-I components (e.g., sleep hygiene and sleep restriction) and enhanced by behavioral activation and a variety of methods to reduce hyperarousal. This program emphasizes detachment from work-related thoughts by including techniques to counter worry and rumination (Thiart et al., Reference Thiart, Lehr, Ebert, Sieland, Berking and Riper2013). It was originally developed and evaluated in teachers (Ebert et al., Reference Ebert, Thiart, Laferton, Berking, Riper, Cuijpers and Lehr2015; Thiart, Lehr, Ebert, Berking, & Riper, Reference Thiart, Lehr, Ebert, Berking and Riper2015; Thiart et al., Reference Thiart, Lehr, Ebert, Sieland, Berking and Riper2013) but has since been adapted and evaluated in the general employee population (Behrendt, Ebert, Spiegelhalder, & Lehr, Reference Behrendt, Ebert, Spiegelhalder and Lehr2020, Brückner et al., Reference Brückner, Wallot, Horvath, Ebert and Lehr2024), where it has been shown to reduce insomnia complaints (Behrendt et al., Reference Behrendt, Ebert, Spiegelhalder and Lehr2020; Ebert et al., Reference Ebert, Thiart, Laferton, Berking, Riper, Cuijpers and Lehr2015; Thiart et al., Reference Thiart, Lehr, Ebert, Berking and Riper2015). Further adaptations and (pilot) tests have been conducted among farmers (Braun et al., Reference Braun, Titzler, Ebert, Buntrock, Terhorst, Freund and Baumeister2019), international students (Spanhel et al., Reference Spanhel, Burdach, Pfeiffer, Lehr, Spiegelhalder, Ebert and Sander2021), and refugees (Spanhel et al., Reference Spanhel, Burdach, Pfeiffer, Lehr, Spiegelhalder, Ebert and Sander2021). All of these groups might profit from an indirect treatment approach since stigma of mental health problems is associated with different fears depending on the context, such as assumed workplace difficulties among employees (Brohan & Thornicroft, Reference Brohan and Thornicroft2010), loss of community support among refugees (Satinsky, Fuhr, Woodward, Sondorp, & Roberts, Reference Satinsky, Fuhr, Woodward, Sondorp and Roberts2019; Shannon, Wieling, Simmelink-McCleary, & Becher, Reference Shannon, Wieling, Simmelink-McCleary and Becher2015), and academic performance, finances, and career anxiety among college students (Cooper, Gin, & Brownell, Reference Cooper, Gin and Brownell2020; Ebert et al., Reference Ebert, Mortier, Kaehlke, Bruffaerts, Baumeister and Auerbach2019).
The aim of the current analysis is to (1) evaluate the efficacy of GET.ON Recovery on DSS reduction in individuals with subclinical or clinical depressive symptoms across different populations as well as potential MDD onset compared to a waiting-list control group (WLC) and to (2) identify possible moderating effects of various participant, clinical and intervention-related characteristics, and combinations thereof.
Methods
This study was designed as an IPD meta-analysis to investigate the efficacy of GET.ON Recovery training or an adapted version thereof on depressive symptom outcomes. The intervention is described in detail in the original study's protocol (Thiart et al., Reference Thiart, Lehr, Ebert, Sieland, Berking and Riper2013). The study was preregistered using the OSF (https://osf.io/xcus5) and follows the Preferred Reporting Items for Systematic Review and Meta-Analyses of IPD (PRISMA-IPD, see online Supplement 1) statement (Stewart et al., Reference Stewart, Clarke, Rovers, Riley, Simmonds, Stewart and Tierney2015) where applicable. For details and rationale for all deviations from the registration, see online Supplement 2.
Identification and selection of studies
Randomized controlled trials investigating a version of the GET.ON Recovery training (intervention group, IG) in comparison to any kind of control group (CG) among adult populations, which assessed DSS at post-treatment and/or follow-up were eligible for inclusion. Studies were identified through the scientific advisors at GET.ON institute (DDE) and by searching the German Clinical Trial Registry (DRKS) using the keyword ‘GET.ON Recovery’ in November 2021. The authors of the eligible studies were contacted and invited to provide IPD.
Risk-of-bias assessment
The revised version of the Cochrane risk-of-bias tool for randomized trials (RoB2; Sterne et al., Reference Sterne, Savović, Page, Elbers, Blencowe, Boutron and Higgins2019) and the related excel tool (Higgins, Savović, Page, & Sterne, Reference Higgins, Savović, Page and Sterne2019) were used to assess the quality of included studies, focusing on the intention-to-treat data available for DSS at post-treatment and/or follow-up. The RoB2 assesses possible bias in five domains: ‘randomization process,’ ‘deviations from interventions,’ ‘missing data,’ ‘outcome measurement,’ and ‘selective reporting.’ Each domain is rated as either ‘low risk,’ ‘some concern,’ or ‘high risk.’ We followed the proposed algorithm to reach an overall judgment, which reflected at least the lowest assessment of an individual domain. Published papers and/or the clinical trial registrations were used for the assessments which were conducted independently by two researchers (PK & JT) who were not involved in the original studies. Disagreements were resolved by discussion.
Depressive outcomes
All depressive symptom outcomes were based on the German version of the Center for Epidemiological Studies Depression Scale (CES-D, Hautzinger, Bailer, Hofmeister, and Keller, Reference Hautzinger, Bailer, Hofmeister and Keller2012). This self-reporting scale consists of 20 items, each rated 0–3, yielding a total score from 0 to 60 with higher scores indicating more severe depressive symptoms. Psychometric properties of the CES-D are well established with a Cronbach's α = 0.89 (Hautzinger et al., Reference Hautzinger, Bailer, Hofmeister and Keller2012). As a primary objective, we focused on DSS at post-treatment and follow-up. Additionally, we examined the following secondary outcomes at post-treatment and follow-up assessments: (1) reliable improvement and deterioration according to the reliable change index (RCI) by Jacobson and Truax (Reference Jacobson and Truax1991), (2) anchor-based clinically relevant change reflecting a 33% change in CES-D score as recommended by the German guideline for treating depression (Bundesärztekammer, Kassenärztliche Bundesvereinigung, & Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften, Reference Bundesärztekammer2022), (3) close-to-symptom-free status defined as a CES-D score <16, and (4) onset of potential MDD (CES-D ⩾16) based on participant self-report(in individuals below cut-off at baseline).
Potential moderators of the intervention effect
Given the limited knowledge of potential moderators, we included a wide range of variables in multivariable analyses. For sociodemographic variables sex, age, relationship status, ethnicity, children, education, and employment, were sought from the original studies. For clinical characteristics, baseline DSS (CES-D), insomnia severity (Insomnia Severity Index, ISI, range: 0–28; Dieck, Morin, and Backhaus, Reference Dieck, Morin and Backhaus2018), and previous experience with psychotherapy and/or health training were selected. After obtaining IPD and inventorying available measures, the following work-related variables were available for all studies and were included as potential moderators in the exploratory analysis: the Effort-Reward Imbalance Scale – Short form (ERI-S; Siegrist, Wege, Pühlhofer, & Wahrendorf, Reference Siegrist, Wege, Pühlhofer and Wahrendorf2009) with subscales effort (range 3–15) and reward (range 7–35) used to calculate an effort-reward ratio (>0.715 indicating imbalance, Lehr, Koch, & Hillert, Reference Lehr, Koch and Hillert2010) and work engagement (Utrecht Work Engagement Scale, UWES; Schaufeli & Bakker, Reference Schaufeli and Bakker2004) with subscales vigor, dedication, and absorption (score range for each: 0–6).
Statistical analyses
The obtained IPD were harmonized by trained personnel according to established coding guidelines (Harrer & Ebert, Reference Harrer and Ebert2023). For all analyses, the significance level was set to α = 0.05 (two-sided) and adjusted for 10 multiple comparisons using the Bonferroni method (Emerson, Reference Emerson2020).
Missing data
This study followed an intention-to-treat approach. Missing post-treatment and follow-up data were estimated separately using the mice package (van Buuren & Groothuis-Oudshoorn, Reference van Buuren and Groothuis-Oudshoorn2011) for multivariate imputation by chained equations in R (R Core Team, 2022) under a missing at random assumption. This assumption provides a plausible starting point for RCTs (van Buuren, Reference van Buuren and van Buuren2018) Stratification by treatment was implemented using the bygroup function in miceadds (Robitzsch & Grund, Reference Robitzsch and Grund2022), which generates imputations separately for intervention and control conditions. Two-level predictive mean matching (2 l.pmm) from the miceadds package was used to account for data clustering. Trial means of baseline DSS were used in the prediction of post-treatment and follow-up symptom severity outcomes (online Supplement 3). A total of 50 imputed datasets were created for each time point. Parameters of interest were estimated from each corresponding imputed data set and were combined using Rubin's rules (Little & Rubin, Reference Little and Rubin2002; Rubin, Reference Rubin1987).
Depression efficacy
To evaluate depression outcomes, we used a one-step IPD approach to better account for the small number of participants/events in the included studies (Riley et al., Reference Riley, Debray, Fisher, Hattle, Marlin, Hoogland and Ensor2020). Separate linear mixed models (LMMs) were specified for each outcome. All models included a random intercept for trial and random slope for the treatment effect and were adjusted for baseline DSS. To calculate the between-study heterogeneity variance in intercept and treatment effect, τ 2 was used. Fixed model parameters are reported with 95% confidence intervals (CIs). LMMs predicting continuous outcomes are reported with model-based Cohen's d values directly estimated by standardizing the outcome using the pooled standard deviation of each group. Parameters for categorical outcomes were estimated from the imputed data using generalized linear mixed models (GLMMs) with a binomial logit link to calculate Odds Ratios (ORs) as the effect size measure of the treatment effect. Numbers-needed-to-treat (NNTs) were calculated as the inverted absolute risk difference and absolute numbers are given for all dichotomous depressive outcomes. Since not all original studies provided relevant variables to calculate the treatment effect on DSS, effects in the individual studies were estimated by separate models in the appropriate imputed data subsets.
Moderation analysis
A multistep approach was used to investigate potential univariable and multivariable moderating effects. First, potential moderators were separately included in the ‘unconditional’ GLMMs. Following the recommendations of Riley et al. (Reference Riley, Debray, Fisher, Hattle, Marlin, Hoogland and Ensor2020), we centered all moderators by their trial-specific mean and included the mean as a level-two predictor to avoid amalgamation of within- and across-trial information. Second, we investigated multivariable treatment-by-moderator interactions. Putative moderators were first ranked for their relevance by calculating the variable permutation importance using the model-based random-forest method (Garge, Bobashev, & Eggleston, Reference Garge, Bobashev and Eggleston2013) in an aggregated dataset as Rubin's rules are not directly applicable for non-parametric approaches. With this method, DSS at post-treatment and follow-up times were regressed on the treatment indicator using 300 bootstrapped samples. All potential moderators were introduced as partitioning variables (i.e. variables to define a subset) on their raw scale using the mobforest package (Garge et al., Reference Garge, Bobashev and Eggleston2013). Variables were ranked by relative importance according to the frequency with which they served as a splitting variable in the trees (termed the ‘permutation accuracy method’).
In the final model-based tree analysis to evaluate possible multivariable moderation, all variables with significant interaction with the treatment effect in univariable models and/or that yielded variable importance values >0 in the random-forest model were included as partitioning variables. Model-based recursive partitioning allows incorporation of machine learning approaches, specifically recursive partitioning, into a parametric model. The result is an easy to interpret decision tree describing subgroups based on distinct values of algorithmically selected variables which have differential treatment effects. The tree was operationalized using the R package glmertree (Zeileis, Hothorn, & Hornik, Reference Zeileis, Hothorn and Hornik2008). In the models for post-treatment and follow-up, DSS was regressed on the treatment indicator in the aggregated data. The nested structure of the patients within the studies was accounted for by specifying a random trial intercept and random treatment effect. Treatment effects in the subgroups were estimated separately to receive model-based Cohen's d as described above. Since the model-based tree analysis approach is prone to overfitting, we reported an optimism-adjusted R 2 obtained by bootstrap bias correction (Harrell, Lee, & Mark, Reference Harrell, Lee and Mark1996; Smith, Seaman, Wood, Royston, & White, Reference Smith, Seaman, Wood, Royston and White2014).
Quasi-Bayesian approach
Between-trial heterogeneity was deemed highly plausible, but the small number of included studies led to an increased risk of improperly estimated heterogeneity variances of zero (singular fits). Therefore, a ‘quasi-Bayesian’ approach was applied using the functionality of the blme package (Chung, Rabe-Hesketh, Dorie, Gelman, & Liu, Reference Chung, Rabe-Hesketh, Dorie, Gelman and Liu2013) throughout imputation, one-stage IPD analyses, and decision-tree building. A weakly informative Wishart prior with df = 4 and a scale matrix multiplied by 0.05 (adapted to 0.01 or 0.075 in case of convergence problems) was used. The prior helped to avoid boundary-fit issues while remaining largely uninformative itself.
Sensitivity analysis
Analysis was repeated as pre-registered in a complete case subsample and additionally in the total sample while excluding the sleep item from the CES-D scores to evaluate the robustness of our results. We decided not to conduct moderation analysis using the complete case sample due to the reduced sample size and power, which would increase the chance for spurious effects in multiple testing.
Results
Study selection and IPD obtained
A total of eight studies evaluating GET.ON Recovery (or a modified version) was identified, of which seven were deemed eligible and the authors of six were asked to contribute IPD. One study was deemed ineligible because it did not assess DSS (unpublished, trial registration: DRKS00017737), while the authors of one eligible study were not asked to contribute IPD because the online training was used only in a small subsample (15/150) of participants who could choose from a portfolio of online training programs (Braun et al., Reference Braun, Titzler, Terhorst, Freund, Thielecke, Ebert and Baumeister2021a, Reference Braun, Titzler, Terhorst, Freund, Thielecke, Ebert and Baumeister2021b). The IPD from two studies (Spanhel et al., Reference Spanhel, Burdach, Pfeiffer, Lehr, Spiegelhalder, Ebert and Sander2021, Reference Spanhel, Hovestadt, Lehr, Spiegelhalder, Baumeister, Bengel and Sander2022) were unavailable because no formal data sharing agreement could be reached. Ultimately, the IPD from four studies was included in the meta-analysis, and no integrity concerns were raised based on inspecting randomization, pre-registered outcomes, data consistency, and completeness, (Fig. 1). Since the study by Ebert et al. (Reference Ebert, Thiart, Laferton, Berking, Riper, Cuijpers and Lehr2015) did not assess follow-up data in the control group due to a shorter waiting-list time, this study was excluded from analysis at follow-up. In total, data from 561 participants in four trials were analyzed for DSS at post-treatment (8 weeks post-randomization) and 433 participants from three trials were analyzed at follow-up (24 weeks post-randomization).
Figure 1. Flowchart of study selection and inclusion.
Study and participant characteristics
All four included studies assessed the effects of GET.ON Recovery training on DSS among employees with high work-related rumination or without clear separation of work and private life and used a waiting-list control group (Table 1). Most participants were female (68%, n = 381/561), in a relationship (70%, n = 395/561), and had achieved more than high school education (76%, n = 429/561). Mean age was 47 years (s.d. = 9.73). The majority had clinically relevant insomnia (ISI⩾15, 76%, n = 425/563) and 78% (n = 440/563) reported clinically relevant levels of depression (CES-D⩾16). Work engagement was considered average (subscale means 3.02–3.38), and the effort-reward ratio suggested an imbalance (M = 1.41, s.d. = 0.40). For more details, see online Supplement 4.
Table 1. Characteristics of the included studies investigating the efficacy of GET.ON recovery on insomnia severity
IG, intervention group; CG, control group; WLC, waiting-list control; IS-CI, Cognitive Irritation Scale (Mohr, Rigotti, & Müller, Reference Mohr, Rigotti and Müller2005); ISI, Insomnia Severity Index (Dieck et al. Reference Dieck, Morin and Backhaus2018); T0, baseline assessment; T1, post-treatment assessment (weeks); T2, follow-up assessment (weeks); segmentation supplies: Subscale from the workplace segmentation preferences and supplies (Kreiner, Reference Kreiner2006); ‘ * ’in IG only; Risk-of-bias assessment: ‘ + ’ low–risk of bias, ‘−’ some concern, ‘x’ high risk, ‘?’ not enough information to assess; Rand, randomization process; Int Dev, deviations from interventions; Out Mis, missing data; Out Meas, outcome measurement; Sel Rep, selective reporting.
Risk-of-bias assessment
All included studies were conservatively judged to have a high risk-of-bias, mainly due to unblinded participants reporting on the outcome by self-report.
Effects on depressive symptom outcomes
All depression-related outcomes are presented in Table 2. Sensitivity analyses confirmed the robustness of the results (online Supplements 5 and 6).
Table 2. Overview of depression outcomes at post-treatment (8 weeks post-randomization) and at follow-up (24 weeks post-randomization) based on multiple imputed data
CES-D, Center for Epidemiological Studies Depression Scale; MDD, major depressive disorder; RCI, reliable change index; IG, intervention group; CG, control group; NNT, number-needed-to-treat; n T0, case number at baseline assessment; n T0, case number with outcome post-treatment or at follow-up; τ int, intercept variance; τ group, slope variance for the treatment effect.
Note: Analysis based on multiple imputation.
a Subgroup exceeding the cut-off for clinically relevant depressive symptoms at baseline (CES-D⩾16).
b Subgroup considered close-to-symptom-free at baseline (CES-D < 16).
DSS
Depression symptom severity was significantly reduced in the intervention group (post-treatment: M = 13.86, s.d. = 7.12; follow-up: M = 14.23, s.d. = 6.73) compared to the control group (post-treatment: M = 19.70, s.d. = 8.19; follow-up: M = 21.13, s.d. = 8.47) both at post-treatment (ß = −5.99 [95% CI: −7.68 to −4.29], T(419.9) = −6.93, p adjusted<0.0001) and at follow-up (ß = −7.28 [95% CI −9.61 to 4.95], T(229.2) = − 6.93, p adjusted<0.0001). The standardized effects estimated in the individual studies and the overall pooled average effects on DSS at post-treatment (d = −0.71 [95% CI −0.92 to −0.51]) and at follow-up (d = −0.84 [95% CI −1.11 to −0.57]) are presented in Fig. 2. The effects were only slightly lower when excluding the sleep item before the analysis (see online Supplement 5), both at post-treatment (M IG = 12.68, s.d.IG = 6.87 v. M CG = 17.81, s.d.CG = 7.89; (ß = −5.28 [95% CI −6.96 to −3.59], T(353.5) = −6.14, p adjusted<0.0001, τ 2 = 1.07, R 2 = 0.35) and follow-up (M IG = 12.98, s.d.IG = 6.48 v. M CG = 19.08, s.d.CG = 8.28; (ß = −6.46 [95% CI −8.59 to −4.33], T(257.5) = −5.97, p adjusted<0.0001, τ 2 = 1.27, R 2 = 0.32).
Figure 2. Forest plot summarizing the estimated effects estimated in individual studies (based on multiple imputation) and the average pooled effect from a one-stage IPD analysis.
Reliable change index
A statistically significant greater proportion of participants in IG than in CG exhibited reliable symptom improvement at post-treatment (159/280, 56.8% v. 65/281, 23.1%; OR 0.19 [95% CI 0.12–0.29], T(411) = −7.39, p adjusted<0.001;) and at follow-up (121/216, 56.0% v. 34/217, 15.7%; OR 0.11 [95% CI 0.05–0.22], T(175.6) = −6.09, p adjusted<0.001;). Additionally, fewer participants in the IG than the control group demonstrated reliable deterioration at post-treatment (7/280, 2.5% v. 21/281, 7.5%; OR 2.83 [95% CI 1.15–6.95], T(308.6) = 2.27, p adjusted=0.284) and at follow-up (7/216, 3.2% v. 24/217, 11.1%; OR 2.95 [95% CI 1.15–7.57], T(194.3) = 2.26, p adjusted = 0.297) but without statistical significance.
Anchor-based clinically relevant change
A 33% reduction in CES-D score was associated with an average point decrease of 7.30 (s.d. = 2.64). A statistically significant greater proportion of participants in IG than in CG reported anchor-based clinically relevant improvement at post-treatment (180/280, 64.3% v. 56/281, 19.9%; OR 0.17 [95% CI 0.11–0.27], T(396.1) = −7.54, p adjusted<0.001) and at follow-up (139/216, 64.4% v. 30/217, 13.8%; OR 0.13 [95% CI 0.07–0.23], T(196.6) = −6.51, p adjusted<0.001).
Close-to-symptom-free status
A statistically significant greater proportion of participants in IG than in CG attained close-to-symptom-free status at post-assessment (152/224, 67.9% v. 41/216, 19.0%; OR 0.16 [95% CI 0.09–0.28], T(306.1) = −6.72, p adjusted<0.001;) and at follow-up (118/173, 68.2% v. 21/163, 12.9%; OR 0.13 [95% CI 0.06–0.25], T(158.6) = −5.86, p adjusted<0.001).
Potential onset of depression
Within the subsample without clinically relevant depressive symptoms at baseline (n = 121), a lower proportion of participants in IG (7/56, 12.5%) than in CG (20/65, 30.8%) exhibited potential onset of MDD after 8 weeks, but the difference did not reach statistical significance (OR 2.13 [95%-CI 0.72–6.32], T(80.0) = 1.38, p = 0.17, p adjusted = 1.00). Similarly, a smaller proportion of participants in IG than in CG exhibited potential MDD onset after 24 weeks (n = 8/43, 18.6% v. n = 22/54, 40.47%), but the difference did not reach statistical significance (OR 2.00 [95% CI 0.68–5.85], T(67.9) = 1.28, p = 0.20, p adjusted = 1.00; 4.5 [95% CI 2.5–21.7]).
Moderation of the treatment effect
Based on the available IPD, ethnicity, employment, and intervention-level variables were excluded as potential moderators due to a lack of variance, while effort-reward imbalance and work engagement were included in addition to sociodemographic and clinical variables. Univariable moderation analysis (online Supplement 7) identified baseline depressive symptom severity as the only significant moderator of follow-up symptom severity (ß = −0.30 [−0.56 to −0.03], p = 0.02). Additionally, based on the model-based random-forest analysis (online Supplement 8), the following variables were included as partitioning variables in the final tree-models at both post-treatment and follow-up: baseline symptoms of depression and insomnia, previous psychotherapy, vigor, dedication, and reward. Relationship status and effort were included in the post-treatment model and age, absorption, and effort-reward ratio were included in the model for follow-up.
In the final tree-based models, only baseline DSS predicted heterogeneous treatment responses. For post-treatment, the first split divided the sample at 21 points with a second split occurring at 13 and 28 points in the two branches, respectively (Fig. 3a). Optimism-corrected R 2 was reduced by 0.11 to R 2adjusted = 0.30. Statistically significant treatment effects were observed in three of the four terminal nodes with differences in the effect magnitude between subgroups based on partitioning the dataset by baseline CES-D scores of ⩽13, >13 but ⩽21, >21 but ⩽28, and >28. Effects were highest for participants with a baseline score >28 (d = −0.87 [95% CI −1.25 to −0.48], n = 122) and smallest without statistical significance in the small group of participants with baseline scores ⩽13 (d = −0.48 [95% CI −0.97, 0.01], n = 77)
Figure 3. Tree model for depressive symptoms (CES-D) post-treatment (a) and at follow-up (b) derived from model-based recursive partitioning in aggregated data.
Similarly, in the follow-up model, only baseline DSS explained the heterogeneity in treatment effect (Fig. 3b). Two splits were identified, the first split at 24 points on the CES-D and the second at 19 points in the subgroup with baseline CES-D scores ⩽24. Optimism-corrected R 2 was reduced by 0.11 to R 2adjusted = 0.27. Treatment effects were significant in all terminal node models, with the biggest effect size in participants with baseline CES-D scores >19 but ⩽24 (d = −0.99 [95% CI −1.33 to −0.64]) and lowest in participants with baseline scores >24 (d = −0.66 [95% CI −0.95 to −0.36]).
Discussion
The aim of the study was to investigate the efficacy of internet-based CBT for insomnia (iCBT-I) as an indirect approach in the prevention and treatment of MDD. The analyses confirmed the superiority of iCBT-I compared to WLC in reducing depressive symptom severity (DSS) with an average pooled intervention effect of iCBT-I of d = −0.71 [95% CI −0.92 to −0.51] at post-treatment and d = −0.84 [95% CI −1.11 to −0.57] at follow-up. These effects were robust in sensitivity analyses with study completers (post-treatment: d = −0.70 [95% CI −0.51 to −0.89]; follow-up: d = −0.80 [95% CI −0.58 to −1.03] and when excluding the sleep item of the CES-D (post-treatment: d = −0.65 [95% CI −0.44 to −0.86], follow-up: d = −0.78 [95% CI −0.52 to −1.03]). Regarding different measures of clinically meaningful improvement, NNTs ranged from two to four individuals. In contrast, iCBT-I demonstrated no significant effect on possible depression onset after 8 weeks and 24 weeks respectively among the subsample (n = 121) without possible depression at baseline. The model-based decision tree revealed four (three) groups defined by their baseline DSS with differential treatment effects at post-treatment (follow-up) reaching from d = −0.48 to d = −0.87 (d = −0.66 to d = −0.99) with no other incremental sociodemographic, clinical, or work-related characteristic moderating effects.
The observed effects were comparable to what would be expected from online CBT directly intended for mild-to-moderate depression which are reported by a recent meta-analysis (Sztein, Koransky, Fegan, & Himelhoch, Reference Sztein, Koransky, Fegan and Himelhoch2018) as d = −0.74 [95% CI −0.62 to −0.86]) at post-treatment and as = −0.83 [95% CI −0.69 to −0.99] at 3–6 months follow-up (d).
The effects found for DSS at post-treatment were also comparable to what has been reported from previous studies using an indirect approach with iCBT-I in mostly subthreshold depression cases and comparing unguided iCBT-I to an active control group (Cohen's d between 0.60 and 0.64) (Batterham et al., Reference Batterham, Christensen, Mackinnon, Gosling, Thorndike, Ritterband and Griffiths2017; Cheng et al., Reference Cheng, Kalmbach, Kalmbach, Tallent, Joseph, Espie and Drake2019a; Christensen et al., Reference Christensen, Batterham, Gosling, Ritterband, Griffiths, Thorndike and Mackinnon2016). Few iCBT-I studies have reported longer follow-up times but those that did reported smaller effects at 6 months (d = 0.40; Batterham et al., Reference Batterham, Christensen, Mackinnon, Gosling, Thorndike, Ritterband and Griffiths2017; d = 0.48; Christensen et al., Reference Christensen, Batterham, Gosling, Ritterband, Griffiths, Thorndike and Mackinnon2016), while we report a slightly larger pooled effect (d = 0.81) at 6-month follow-up. Effects at post-treatment reported here were smaller compared to a study using guided iCBT-I and an active control for individuals with at least mild depressive symptoms (d = 1.05 based on the Patient Health Questionnaire [PHQ]) (van der Zweerde et al., Reference van der Zweerde, van Straten, Effting, Kyle and Lancee2019). This difference was, however, reduced when analyses without the sleep items were compared (van der Zweerde et al., Reference van der Zweerde, van Straten, Effting, Kyle and Lancee2019: d = 0.76 v. current study: d = 0.65). However, all studies in our analysis included more severely depressed individuals, compared the effect to a WLC and two studies were guided. All these factors may have impacted the reported effect sizes and made strong comparisons difficult (Furukawa et al., Reference Furukawa, Noma, Caldwell, Honyashiki, Shinohara, Imai and Churchill2014; Werntz, Amado, Jasman, Ervin, & Rhodes, Reference Werntz, Amado, Jasman, Ervin and Rhodes2023).
Comparison regarding MDD onset is limited due to the small subgroup of depression-free participants at baseline, therefore reduced power, and the relatively short observation period of 6 months. Similarly to our non-finding of a preventive effect on MDD, Christensen et al. (Reference Christensen, Batterham, Gosling, Ritterband, Griffiths, Thorndike and Mackinnon2016) found no differences in depression onset at 6 months after iCBT-I using diagnostic interviews for both study inclusion and evaluation. In contrast, Cheng et al. (Reference Cheng, Kalmbach, Kalmbach, Tallent, Joseph, Espie and Drake2019a, Reference Cheng, Luik, Fellman-Couture, Peterson, Joseph, Tallent and Drake2019b) reported that the risk for depression onset was halved in the intervention group compared to the control group (relative risk ratio = 0.51 [95% CI 0.26–0.81]) in one-year post-intervention based on cut-off scores for self-reported measurements.
Additionally, the effects are in line with a small-scale study directly comparing iCBT-I to online CBT for depression in participants with clinical insomnia (Blom et al., Reference Blom, Jernelöv, Kraepelien, Bergdahl, Jungmarker, Ankartjärn and Kaldo2015). While iCBT-I was superior for reducing sleep problems, both iCBT-I and depression CBT were equally effective in reducing depressive symptoms up to 3 years post-treatment (Blom et al., Reference Blom, Jernelöv, Rück, Lindefors and Kaldo2017).
Model-based recursive partitioning analyses suggested differential efficacy of iCBT-I among subgroups of participants that differed depending on initial depressive symptom severity. However, both likely non-clinical cases with elevated depressive symptoms (CES-D score 14–21) (Bundesärztekammer et al., Reference Bundesärztekammer2022; Radloff, Reference Radloff1977; Vilagut, Forero, Barbaglia, & Alonso, Reference Vilagut, Forero, Barbaglia and Alonso2016) and probable MDD cases (CES-D > 28) (Bundesärztekammer et al., Reference Bundesärztekammer2022) seem to benefit from iCBT-I to a similar degree.
The largest effect sizes were observed in the group of individuals scoring around the cut-off for highly likely diagnosable depression (CES-D > 28; Bundesärztekammer et al., Reference Bundesärztekammer2022). The results are in line with Cheng et al. (Reference Cheng, Luik, Fellman-Couture, Peterson, Joseph, Tallent and Drake2019b), who also identified baseline DSS as the sole moderator, with participants in the upper tertile of a mild-moderate depressed sample showing the greatest improvements.
Implications for research and practice
Our analyses add to the evidence that iCBT-I can effectively reduce depressive symptom severity and the multivariable moderation analyses additionally suggested its efficacy across different levels of baseline depressive symptom severity. Consequently, severely depressed individuals without suicidal ideation would not need to be excluded from an indirect treatment approach. The rates of reliable deterioration in IG (n = 7/280, 2.5% at post-treatment; n = 7/216, 3.24% at follow-up) are comparable to those meta-analytically reported for (in-person and online) psychotherapy for depression with an estimated RCI deterioration rate of 5% at post-treatment (Cuijpers et al., Reference Cuijpers, Karyotaki, Ciharova, Miguel, Noma and Furukawa2021a). While the studies included in this IPD meta-analysis did not include suicidal participants, other studies suggest that this indirect approach (targeting insomnia) may also be an opportunity for suicide prevention with an appropriate safety protocol (Christensen et al., Reference Christensen, Batterham, Gosling, Ritterband, Griffiths, Thorndike and Mackinnon2016; Kalmbach et al., Reference Kalmbach, Cheng, Ahmedani, Peterson, Reffi, Sagong and Drake2022; Torok et al., Reference Torok, Han, Baker, Werner-Seidler, Wong, Larsen and Christensen2020).
Given the growing evidence for the efficacy of CBT-I in the treatment of depressive symptoms (Asarnow & Manber, Reference Asarnow and Manber2019) and that the efficacy of iCBT-I on DSS did not vary across demographics in this IPD meta-analysis, iCBT-I should be more highlighted in practice and included in the associated treatment guidelines as earlier demanded by Morin et al. (Reference Morin, Bertisch, Pelayo, Watson, Winkelman, Zee and Krystal2023). Implementation science should focus on how to integrate an indirect treatment approach into routine care. However, it is still important to test the differences in the uptake of insomnia and depression interventions in a more naturalistic setting and in a sample less confounded by high work stress, which is associated with depression risk independent of sleep pathology (Siegrist, Reference Siegrist2008). In addition, patients' attitudes towards an indirect treatment approach and their naïve perception of how their symptoms relate to each other should be considered in future research as Kraepelien et al. (Reference Kraepelien, Forsell and Blom2022) revealed that depressed patients with elevated insomnia symptoms actively sought depression-focused CBT instead of iCBT-I.
Studies directly comparing depression to insomnia treatment (as in Blom et al., Reference Blom, Jernelöv, Kraepelien, Bergdahl, Jungmarker, Ankartjärn and Kaldo2015) in a preventive setting with subthreshold insomnia and depression are still warranted to inform further personalization of preventive offers. Enhancing personalization and levering effectiveness of preventive interventions mighty derive from considering individual symptoms instead of general symptom severity at baseline, given that multiple studies have identified especially problems initiating sleep as a predictor of later depression onset (Bjorøy, Jørgensen, Pallesen, & Bjorvatn, Reference Bjorøy, Jørgensen, Pallesen and Bjorvatn2020; Blanken, Borsboom, Penninx, & Van Someren, Reference Blanken, Borsboom, Penninx and Van Someren2020; Leerssen et al., Reference Leerssen, Lakbila-Kamal, Dekkers, Ikelaar, Albers, Blanken and Van Someren2021).
With regard to maintaining treatment effects and relapse prevention it would be positive if treatment experience encourage future help-seeking intentions if needed, but it is unclear to what extent an indirect approach can support this. Studies thus far have mainly assessed help-seeking intention as a means to estimate further need after the intervention (Blom et al., Reference Blom, Jernelöv, Kraepelien, Bergdahl, Jungmarker, Ankartjärn and Kaldo2015; Christensen et al., Reference Christensen, Batterham, Gosling, Ritterband, Griffiths, Thorndike and Mackinnon2016). Moreover, promotion of help-seeking and de-stigmatization of mental health problems should not stop at an individual level but should be seen as a societal effort (Clement et al., Reference Clement, Schauman, Graham, Maggioni, Evans-Lacko, Bezborodovs and Thornicroft2015).
Finally, Asarnow and Manber (Reference Asarnow and Manber2019) found inconclusive evidence for the greater efficacy of combined over sequential insomnia and depression treatment and also suggested that comorbidity may influence adherence and dropout. Internet interventions could serve as an ideal testing ground to address these questions (Domhardt, Cuijpers, Ebert, & Baumeister, Reference Domhardt, Cuijpers, Ebert and Baumeister2021). Module-based online training could be used to explore if the order of components, for example, behavioral activations and sleep restriction, interact with each other and the treatment outcomes over time or if individual modules targeting potential transdiagnostic factors like rumination (Behrendt et al., Reference Behrendt, Ebert, Spiegelhalder and Lehr2020; Cheng, Kalmbach, Castelan, Murugan, & Drake, Reference Cheng, Kalmbach, Castelan, Murugan and Drake2020) are especially crucial for combined treatment. Internet interventions combining aspects of insomnia and depression treatment as a predefined module, on demand additional modules chosen by the user, or recommendations by the program/guiding coach could be used to adapt the intervention to the individual's needs and preferences.
Limitations
The current results should be interpreted considering several limitations. First, depending on the obtained IPD, our analysis included studies from a very homogenous group and a single (adapted) intervention. For instance, all participants came from a high-income country, were employed, predominantly female and highly educated. The intervention itself focused on work-related rumination in addition to the classic CBT-I components, such as sleep hygiene and sleep restriction. While this focus on the work context supports the ecological validity and highlights the potential of an indirect approach for occupational health, it limits transferability to other contexts. Especially the transfer to unemployed individuals, who are more prone to develop a depression than employed individuals (Van Der Noordt, IJzelenberg, Droomers, & Proper, Reference Van Der Noordt, IJzelenberg, Droomers and Proper2014), is not possible and is in need of dedicated studies. Second, we based our analysis on the German version of the CES-D, which does not have one uniform cut-off for depression onset or different categorical levels of DSS, limiting comparability with other studies in the field (Vilagut et al., Reference Vilagut, Forero, Barbaglia and Alonso2016). We retained the pre-specified cut-off of ⩾16 but included other cut-offs suggested from more recent depression guidelines, but future research should also consider clinical assessments. This limits generalizability and comparability with other standardized measures for depression such as the PHQ-9. Third, our results on subthreshold depression and potential depression onset should be interpreted with special caution because we (a) used a self-reported cut-off to identify potential MDD cases and (b) had a very reduced sample size and power to examine the effects on onset in a population with insomnia/depressive symptom comorbidity. Fourth, the reported effect sizes may be higher than would be expected in routine care given the WLC and high risk of bias in outcome measures. The reported OR were controlled for baseline CES-D to report conditional effects but given the underlying logistic distribution, the marginal effect on population level might be smaller (Groenwold, Moons, Peelen, Knol, & Hoes, Reference Groenwold, Moons, Peelen, Knol and Hoes2011). The effect sizes could also vary in relation to the amount of guidance provided, which was not studied due to the small number of included studies, but which should be focused on in future studies on (aggregated) data. Finally, the univariable and multivariable moderation analyses were the first in the field of indirect prevention and treatment of depression, and due to the relatively small sample size, should be considered exploratory and in need of validation across different samples. The random-forest method was only feasible in an aggregated dataset and did not account for the multilevel structure; thus, the analysis did not consider imputation insecurity or heterogeneity among included studies. Similarly, the model-based recursive partitioning was also run in an aggregated dataset but could consider the multilevel structure. Further, for ease of interpretation, we did not center variables in the tree, which could introduce ecological bias. Finally, due to the relatively small sample size, the tree analysis was prone to overfitting, so we adjusted the R 2.
Conclusion
The findings of the current study provide evidence that iCBT-I can probably effectively reduce depressive symptom severity in working adults experiencing sleep problems and high work stress. Multivariable moderation analyses suggested that the effect size magnitude of iCBT-I varied according to baseline symptom severity, but that iCBT-I is a promising intervention approach for treatment of comorbid insomniac and depressive symptoms. Dedicated studies are needed to conclude whether or not this approach is also applicable to the preventive setting.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291724000527
Data availability statement
The data are not publicly available. The corresponding author is a data processor, not a data owner and thus cannot provide data access upon request. Access to the data must be sought from authors of original studies and might depend on to be specified data security and data exchange regulation agreements. The analyses scripts can be assessed via OSF: https://osf.io/fg46t/
Acknowledgements
Authors would like to thank enago Academy for proofreading the manuscript and TUM Graduate School for covering the costs for this service as part of JT's PhD program.
Author contributions
Janika Thielecke: Conceptualization, Methodology, Formal Analysis, Data Curation, Visualization, Project administration, Writing – Original Draft, Writing – Review & Editing. Paula Kuper: Conzeptionalization, Methodology, Formal Analysis, Data Curation, Visualization, Writing – Review & Editing. Dirk Lehr: Conzeptionalization, Resources, Writing – Review & Editing. Lea Schuurmans: Conzeptionalization, Methodology, Formal Analysis, Data Curation, Writing – Review & Editing. Mathias Harrer: Conzeptionalization, Methodology, Data Curation, Formal Analysis, Writing – Review & Editing. David D. Ebert: Conzeptionalization, Resources, Supervision, Writing – Review & Editing. Dörte Behrendt: Resources, Writing – Review & Editing. Hanna Brückner: Resources, Writing – Review & Editing. Hanne Horvath: Resources, Writing – Review & Editing. Heleen Riper: Resources, Writing – Review & Editing. Pim Cuijpers: Resources, Writing – Review & Editing. Claudia Buntrock: Conzeptionalization, Methodology, Project administration, Supervision, Writing – Review & Editing.
Funding statement
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Competing interests
MH is a part-time employee at the Institute for Health Trainings Online (GET.ON), which aims to implement scientific findings related to digital health interventions into routine care.
DDE reports to have received consultancy fees or served on the scientific advisory board of several companies such as Novartis, Sanofi, Lantern, Schön Kliniken, Minddistrict, and German health insurance companies (BARMER, Techniker Krankenkasse). DDE is a stakeholder of GET.ON. HH is Founder and Chief Commercial Officer at GET.ON. JT, PK, LS, DL, HB, DB, HR, PC, and CB report no competing interests.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. All included studies individually underwent approval by the ethics committee of the responsible institutions as described in the original articles.
Open access
