Skip to main content Accessibility help
Hostname: page-component-cf9d5c678-8r4lv Total loading time: 0.307 Render date: 2021-07-28T07:09:36.067Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": true, "newCiteModal": false, "newCitedByModal": true, "newEcommerce": true, "newUsageEvents": true }

A rejoinder from Gerger and colleagues

Published online by Cambridge University Press:  05 September 2014

H. Gerger
Department of Psychology, Division of Clinical Psychology and Psychotherapy, University of Basel, Switzerland
T. Munder
Psychologische Hochschule Berlin, Germany
J. Barth
Institute of Complementary and Integrative Medicine, University Hospital of Zurich, Switzerland
E-mail address:
Rights & Permissions[Opens in a new window]


Copyright © Cambridge University Press 2014 

Systematic syntheses of individual trials have been described as the ‘gold standard’ for the evaluation of interventions (Sackett et al. Reference Sackett, Rosenberg, Gray, Haynes and Richardson1996, p. 72). As pointed out by de Jonghe et al., systematic reviews and meta-analyses play an increasing role in the decision making of clinicians, researchers and policy makers. However, meta-analyses are, of course, not immune from bias. In their letter, de Jonghe et al. criticize our recent network meta-analysis of psychological interventions for post-traumatic stress disorder (PTSD) (Gerger et al. Reference Gerger, Munder, Gemperli, Nüesch, Trelle, Jüni and Barth2014 b) for severe methodological shortcomings and question the relevance of our study. Many of the issues raised by de Jonghe et al. have already been considered in our paper. However, we would like to use the opportunity of this rejoinder to further clarify some issues.

de Jonghe et al. seem to be unsatisfied with our finding of equivalent effectiveness of specific psychological interventions. They argue that we did not identify superiority of any intervention ‘due to the considerable between-trial heterogeneity’, which they attribute to our inclusion of heterogeneous samples. The first part of this statement lacks scientific evidence, however. As described in the Introduction of our paper, the large majority of meta-analyses in the field of PTSD interventions conclude equivalent effectiveness of specific interventions (e.g. Bisson & Andrew, Reference Bisson and Andrew2007; Watts et al. Reference Watts, Schnurr, Mayo, Young-Xu, Weeks and Friedman2013) and none of the interventions has consistently been shown to outperform the others; not even in meta-analyses with less between-trial heterogeneity (e.g. Benish et al. Reference Benish, Imel and Wampold2008). However, as stated in our Limitations we admit that we did not control for possibly moderating effects of clinical patient characteristics, which have previously been shown to affect relative effect size estimates (Gerger et al. Reference Gerger, Munder and Barth2014 a). We have, however, conducted a moderator analysis including the status of a full PTSD diagnosis (versus subclinical PTSD symptoms) to explain heterogeneity, which de Jonghe and colleagues may have overseen in our paper.

A further point of critique is that we distinguished trials with small to moderate samples from trials with larger samples. We elaborated extensively on the rationale for the cut-offs chosen in our analyses in the Method. We are therefore not clear about the actual critique here. Our cut-offs conform to those proposed by Schnurr (Reference Schnurr2007), which also rely on power considerations. Given the vast literature on the risk of bias that is typically associated with small samples (Egger et al. Reference Egger, Smith, Schneider and Minder1997; Sterne et al. Reference Sterne, Gavaghan and Egger2000; Cuijpers et al. Reference Cuijpers, Smit, Bohlmeijer, Hollon and Andersson2010a , Reference Cuijpers, van Straten, Bohlmeijer, Hollon and Andersson b ; Nüesch et al. Reference Nüesch, Trelle, Reichenbach, Rutjes, Tschannen, Altman, Egger and Jüni2010; Barth et al. Reference Barth, Munder, Gerger, Nüesch, Trelle, Znoj, Jüni and Cuijpers2013; Watts et al. Reference Watts, Schnurr, Mayo, Young-Xu, Weeks and Friedman2013), we do not believe that the authors aimed at fundamentally questioning the relevance of sample size as a moderator variable.

Furthermore, our conclusion of eye movement desensitization and reprocessing (EMDR) as ‘promising’, which is not negative in principle, seems to contradict de Jonghe et al.'s expectations. The authors argue that, in the presence of more than 20 trials on EMDR, our conclusion ‘lacks scientific merit’. However, from our point of view and based on the extensive empirical literature on small sample bias, we feel very confident in repeating the conclusion regarding the lack of robust evidence for EMDR. We were unable to identify a single trial on the efficacy of EMDR that was adequately sized to detect relative intervention effects of moderate to small size. We therefore strongly argue for the need for collaborative research projects (such as the Social Phobia Psychotherapy Network by Leichsenring et al. Reference Leichsenring, Hoyer, Beutel, Herpertz, Hiller, Irle, Joraschky, Konig, de Liz, Nolting, Pohlmann, Salzer, Schauenburg, Stangier, Strauss, Subic-Wrana, Vormfelde, Weniger, Willutzki, Wiltink and Leibing2009) that aim at maximizing the number of patients included in a comparative trial and at minimizing the potential for bias from researchers’ preferences (the so-called allegiance bias; see Munder et al. Reference Munder, Gerger, Trelle and Barth2011, Reference Munder, Flückiger, Gerger, Wampold and Barth2012) at the same time. Our evaluation of the evidence for EMDR also mirrors the appraisal of the Society of Clinical Psychology (APA Division 12) that summarizes the evidence for EMDR as ‘strong research support/controversial’. We admit, however, that more recent publications may reveal additional evidence from larger trials on the effectiveness of EMDR.

Referring to the last statement, de Jonghe and colleagues criticize our study for ending our search in 2010 and thereby missing ‘a number of studies’. We agree that a more recent search of trials would be preferable. However, we learned from two previous updates of our meta-analysis that effect estimates and their precision changed only slightly. The main results remained constant across the updates. The reliability of the major findings regarding the equivalent effectiveness of individual types of intervention may also be regarded as confirmed by several previous meta-analyses on PTSD interventions (e.g. Bradley et al. Reference Bradley, Greene, Russ, Dutra and Westen2005; Bisson & Andrew, Reference Bisson and Andrew2007) and also by the most recent meta-analysis by Watts et al. (Reference Watts, Schnurr, Mayo, Young-Xu, Weeks and Friedman2013). The conclusions of these meta-analyses are very similar to ours. We summarized these results in the Introduction of our paper.

We agree with de Jonghe and colleagues that our meta-analysis does not provide radically new insights with regard to the classic horse race question. However, the major advantage of our meta-analysis compared to other meta-analyses in the field of PTSD interventions is that we applied a novel approach of analysis, namely network meta-analysis. By summarizing all available evidence from studies with multiple comparisons and diverse control groups in a single analysis, network meta-analysis allowed us to address the question of relative intervention effects more elegantly and with more statistical power. Thus, our network meta-analysis allowed us to go beyond the classic horse race question and explore sources of heterogeneity and alternative ways of clustering interventions, and in particular allowed us to conduct analyses with a relatively small number of trials and a small number of direct comparisons. On closer examination of earlier meta-analyses, however, the relatively small number of seven trials, that were included in our analysis of large-scaled trials, seems to be the rule rather than the exception. Thus, in the influential systematic review by Bisson & Andrew (Reference Bisson and Andrew2007), for instance, out of more than 80 effect size estimates only three were based on more than seven trials. The meta-analysis by Watts et al. (Reference Watts, Schnurr, Mayo, Young-Xu, Weeks and Friedman2013) shows a similar picture, with a large number of meta-analyses that are based on only few included trials. By combining all available evidence in one analysis we found that the so-called non-specific psychological interventions may be beneficial PTSD treatments; a finding that partly contradicts previous conclusions (e.g. Bisson & Andrew, Reference Bisson and Andrew2007) but conforms to a very recent meta-analysis on present-centered therapy (Frost et al. Reference Frost, Laska and Wampold2014). In the latter paper, present-centered therapy, which has been designed as a control for non-specific intervention effects by Schnurr et al. (Reference Schnurr, Friedman, Foy, Shea, Hsieh, Lavori, Glynn, Wattenberg and Bernardy2003), has been proven to be as effective as trauma-focused and evidence-based interventions.

We therefore do not share the evaluation of de Jonghe et al. of our meta-analysis as being ‘marginally’ relevant ‘at best’. If our findings were confirmed by high-standard trials, they may indeed be of great value for decision making in clinical practice. In this sense, as discussed in our initial report, the findings from our study, which need to be confirmed in adequately designed experimental trials, should be regarded as hypothesis generating. This, however, is true for any meta-analytic finding, as meta-analysis is observational by nature.


We thank Prof. Dr. J. Gaab for comments on a previous version of this letter.

Declaration of Interest



Barth, J, Munder, T, Gerger, H, Nüesch, E, Trelle, S, Znoj, H, Jüni, P, Cuijpers, P (2013). Comparative efficacy of seven psychotherapeutic interventions for depressed patients: results of a network meta-analysis. PLoS Medicine 10, e1001454.CrossRefGoogle Scholar
Benish, SG, Imel, ZE, Wampold, BE (2008). The relative efficacy of bona fide psychotherapies for treating post-traumatic stress disorder: a meta-analysis of direct comparisons. Clinical Psychology Review 28, 746758.CrossRefGoogle ScholarPubMed
Bisson, JI, Andrew, M (2007). Psychological treatment of post-traumatic stress disorder (PTSD). Cochrane Database of Systematic Reviews. Issue 3, Art. No.: CD003388.CrossRefGoogle Scholar
Bradley, R, Greene, J, Russ, E, Dutra, L, Westen, D (2005). A multidimensional meta-analysis of psychotherapy for PTSD. American Journal of Psychiatry 162, 214227.CrossRefGoogle ScholarPubMed
Cuijpers, P, Smit, F, Bohlmeijer, E, Hollon, SD, Andersson, G (2010 a). Efficacy of cognitive-behavioural therapy and other psychological treatments for adult depression: meta-analytic study of publication bias. British Journal of Psychiatry 196, 173178.CrossRefGoogle ScholarPubMed
Cuijpers, P, van Straten, A, Bohlmeijer, E, Hollon, SD, Andersson, G (2010 b). The effects of psychotherapy for adult depression are overestimated: a meta-analysis of study quality and effect size. Psychological Medicine 40, 211223.CrossRefGoogle ScholarPubMed
Egger, M, Smith, GD, Schneider, M, Minder, C (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal 315, 629634.CrossRefGoogle ScholarPubMed
Frost, ND, Laska, KM, Wampold, BE (2014). The evidence for present-centered therapy as a treatment for posttraumatic stress disorder. Journal of Traumatic Stress 27, 18.CrossRefGoogle ScholarPubMed
Gerger, H, Munder, T, Barth, J (2014 a). Specific and nonspecific psychological interventions for PTSD symptoms: a meta-analysis with problem complexity as a moderator. Journal of Clinical Psychology 70, 601615.CrossRefGoogle ScholarPubMed
Gerger, H, Munder, T, Gemperli, A, Nüesch, E, Trelle, S, Jüni, P, Barth, J (2014 b). Integrating fragmented evidence by network meta-analysis: relative effectiveness of psychological interventions for adults with post-traumatic stress disorder. Psychological Medicine. Published online: 16 April 2014 . doi:10.1017/S0033291714000853.CrossRefGoogle ScholarPubMed
Leichsenring, F, Hoyer, J, Beutel, M, Herpertz, S, Hiller, W, Irle, E, Joraschky, P, Konig, HH, de Liz, TM, Nolting, B, Pohlmann, K, Salzer, S, Schauenburg, H, Stangier, U, Strauss, B, Subic-Wrana, C, Vormfelde, S, Weniger, G, Willutzki, U, Wiltink, J, Leibing, E (2009). The social phobia psychotherapy research network. The first multicenter randomized controlled trial of psychotherapy for social phobia: rationale, methods and patient characteristics Psychotherapy and Psychosomatics 78, 3541.CrossRefGoogle Scholar
Munder, T, Flückiger, C, Gerger, H, Wampold, BE, Barth, J (2012). Is the allegiance effect an epiphenomenon of true efficacy differences between treatments? A meta-analysis. Journal of Counseling Psychology 59, 631637.CrossRefGoogle ScholarPubMed
Munder, T, Gerger, H, Trelle, S, Barth, J (2011). Testing the allegiance bias hypothesis: a meta-analysis. Psychotherapy Research 21, 670684.CrossRefGoogle ScholarPubMed
Nüesch, E, Trelle, S, Reichenbach, S, Rutjes, AWS, Tschannen, B, Altman, DG, Egger, M, Jüni, P (2010). Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study. British Medical Journal 341, c3515.CrossRefGoogle Scholar
Sackett, DL, Rosenberg, W, Gray, JA, Haynes, RB, Richardson, WS (1996). Evidence based medicine: what it is and what it isn't. British Medical Journal 312, 7172.CrossRefGoogle ScholarPubMed
Schnurr, PP (2007). The rocks and hard places in psychotherapy outcome research. Journal of Traumatic Stress 20, 779792.CrossRefGoogle Scholar
Schnurr, PP, Friedman, MJ, Foy, DW, Shea, MT, Hsieh, FY, Lavori, PW, Glynn, SM, Wattenberg, M, Bernardy, NC (2003). Randomized trial of trauma-focused group therapy for posttraumatic stress disorder: results from a Department of Veterans Affairs Cooperative Study. Archives of General Psychiatry 60, 481489.CrossRefGoogle Scholar
Sterne, JAC, Gavaghan, D, Egger, M (2000). Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. Journal of Clinical Epidemiology 53, 11191129.CrossRefGoogle ScholarPubMed
Watts, BV, Schnurr, PP, Mayo, L, Young-Xu, Y, Weeks, WB, Friedman, MJ (2013). Meta-analysis of the efficacy of treatments for posttraumatic stress disorder. Journal of Clinical Psychiatry 74, e541–e550.Google ScholarPubMed
You have Access

Send article to Kindle

To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

A rejoinder from Gerger and colleagues
Available formats

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

A rejoinder from Gerger and colleagues
Available formats

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

A rejoinder from Gerger and colleagues
Available formats

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *