Hostname: page-component-cd9895bd7-jkksz Total loading time: 0 Render date: 2024-12-27T01:35:45.889Z Has data issue: false hasContentIssue false
  • English
  • Français

Citation bias and selective focus on positive findings in the literature on the serotonin transporter gene (5-HTTLPR), life stress and depression

Published online by Cambridge University Press:  12 August 2016

Y. A. de Vries ,
A. M. Roest ,
M. Franzen ,
M. R. Munafò  and
J. A. Bastiaansen
Y. A. de Vries*
Affiliation:
Department of Psychiatry, Interdisciplinary Center Psychopathology and Emotion Regulation, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands
A. M. Roest
Affiliation:
Department of Psychiatry, Interdisciplinary Center Psychopathology and Emotion Regulation, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands
M. Franzen
Affiliation:
Department of Psychology, University of Groningen, Groningen, The Netherlands
M. R. Munafò
Affiliation:
MRC Integrative Epidemiology Unit (IEU) at the University of Bristol, Bristol, UK UK Centre for Tobacco and Alcohol Studies, School of Experimental Psychology, University of Bristol, Bristol, UK
J. A. Bastiaansen
Affiliation:
Department of Psychiatry, Interdisciplinary Center Psychopathology and Emotion Regulation, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands
*
*Address for correspondence: Y. A. de Vries, M.Sc., Department of Psychiatry, University Medical Center Groningen, Hanzeplein 1, 9713 GZ Groningen, The Netherlands. (Email: y.a.de.vries@umcg.nl)
Get access Rights & Permissions [Opens in a new window]

Abstract

Background

Caspi et al.'s 2003 report that 5-HTTLPR genotype moderates the influence of life stress on depression has been highly influential but remains contentious. We examined whether the evidence base for the 5-HTTLPR–stress interaction has been distorted by citation bias and a selective focus on positive findings.

Method

A total of 73 primary studies were coded for study outcomes and focus on positive findings in the abstract. Citation rates were compared between studies with positive and negative results, both within this network of primary studies and in Web of Science. In addition, the impact of focus on citation rates was examined.

Results

In all, 24 (33%) studies were coded as positive, but these received 48% of within-network and 68% of Web of Science citations. The 38 (52%) negative studies received 42 and 23% of citations, respectively, while the 11 (15%) unclear studies received 10 and 9%. Of the negative studies, the 16 studies without a positive focus (42%) received 47% of within-network citations and 32% of Web of Science citations, while the 13 (34%) studies with a positive focus received 39 and 51%, respectively, and the nine (24%) studies with a partially positive focus received 14 and 17%.

Conclusions

Negative studies received fewer citations than positive studies. Furthermore, over half of the negative studies had a (partially) positive focus, and Web of Science citation rates were higher for these studies. Thus, discussion of the 5-HTTLPR–stress interaction is more positive than warranted. This study exemplifies how evidence-base-distorting mechanisms undermine the authenticity of research findings.

Keywords

Citation biasdepression5-HTTLPRstress
Type
Original Articles
Information
Psychological Medicine , Volume 46 , Issue 14 , October 2016 , pp. 2971 - 2979
Copyright
Copyright © Cambridge University Press 2016 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bastiaansen, JA, de Vries, YA, Munafò, MR (2015). Citation distortions in the literature on the serotonin-transporter-linked polymorphic region and amygdala activation. Biological Psychiatry 78, E35–E36.CrossRefGoogle ScholarPubMed
Bastiaansen, JA, Servaas, MN, Marsman, JBC, Ormel, J, Nolte, IM, Riese, H, Aleman, A (2014). Filling the gap: relationship between the serotonin-transporter-linked polymorphic region and amygdala activation. Psychological Science 25, 2058–2066.CrossRefGoogle Scholar
Boutron, I, Dutton, S, Ravaud, P, Altman, DG (2010). Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA 303, 2058–2064.CrossRefGoogle ScholarPubMed
Caspi, A, Hariri, AR, Holmes, A, Uher, R, Moffitt, TE (2010). Genetic sensitivity to the environment: the case of the serotonin transporter gene and its implications for studying complex diseases and traits. American Journal of Psychiatry 167, 509–527.CrossRefGoogle Scholar
Caspi, A, Sugden, K, Moffitt, TE, Taylor, A, Craig, IW, Harrington, HL, McClay, J, Mill, J, Martin, J, Braithwaite, AW, Poulton, R (2003). Influence of life stress on depression: moderation by a polymorphism in the 5-HTT gene. Science 301, 386–389.CrossRefGoogle ScholarPubMed
Culverhouse, RC, Bowes, L, Breslau, N, Nurnberger, JI, Burmeister, M, Fergusson, DM, Munafò, MR, Saccone, NL, Bierut, LJ (2013). Protocol for a collaborative meta-analysis of 5-HTTLPR, stress, and depression. BMC Psychiatry 13, 304.CrossRefGoogle ScholarPubMed
De Jonge, P, Conradi, HJ, Thombs, BD, Rosmalen, JGM, Burger, H, Ormel, J (2011). Prevention of false positive findings in observational studies: registration will not work but replication might. Journal of Epidemiology and Community Health 65, 95–96.CrossRefGoogle ScholarPubMed
Duncan, L, Keller, M (2011). A critical review of the first 10 years of candidate gene-by-environment interaction research in psychiatry. American Journal of Psychiatry 168, 1041–1049.CrossRefGoogle ScholarPubMed
Etter, J-F, Stapleton, J (2009). Citations to trials of nicotine replacement therapy were biased toward positive results and high-impact-factor journals. Journal of Clinical Epidemiology 62, 831–837.CrossRefGoogle ScholarPubMed
Greenberg, SA (2009). How citation distortions create unfounded authority: analysis of a citation network. BMJ 339, b2680.CrossRefGoogle ScholarPubMed
Hammen, C (2005). Stress and depression. Annual Review of Clinical Psychology 1, 293–319.CrossRefGoogle ScholarPubMed
Heininga, VE, Oldehinkel, AJ, Veenstra, R, Nederhof, E (2015). I just ran a thousand analyses: benefits of multiple testing in understanding equivocal evidence on gene–environment interactions. PLOS ONE 10, e0125383.CrossRefGoogle Scholar
Hu, X, Oroszi, G, Chun, J, Smith, TL, Goldman, D, Schuckit, MA (2005). An expanded evaluation of the relationship of four alleles to the level of response to alcohol and the alcoholism risk. Alcoholism: Clinical and Experimental Research 29, 8–16.CrossRefGoogle Scholar
Jannot, A-S, Agoritsas, T, Gayet-Ageron, A, Perneger, TV (2013). Citation bias favoring statistically significant studies was present in medical research. Journal of Clinical Epidemiology 66, 296–301.CrossRefGoogle ScholarPubMed
Kapur, S, Phillips, AG, Insel, TR (2012). Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it? Molecular Psychiatry 17, 1174–1179.CrossRefGoogle Scholar
Karg, K, Burmeister, M, Shedden, K, Sen, S (2011). The serotonin transporter promoter variant (5-HTTLPR), stress, and depression meta-analysis revisited: evidence of genetic moderation. Archives of General Psychiatry 68, 444–454.CrossRefGoogle ScholarPubMed
Kjaergard, LL, Gluud, C (2002). Citation bias of hepato-biliary randomized clinical trials. Journal of Clinical Epidemiology 55, 407–410.CrossRefGoogle ScholarPubMed
Martin, J, Cleak, J, Willis-Owen, SAG, Flint, J, Shifman, S (2007). Mapping regulatory variants for the serotonin transporter gene based on allelic expression imbalance. Molecular Psychiatry 12, 421–422.CrossRefGoogle ScholarPubMed
Munafò, MR, Durrant, C, Lewis, G, Flint, J (2009). Gene x environment interactions at the serotonin transporter locus. Biological Psychiatry 65, 211–219.CrossRefGoogle Scholar
Murphy, SE, Norbury, R, Godlewska, BR, Cowen, PJ, Mannie, ZM, Harmer, CJ, Munafò, MR (2013). The effect of the serotonin transporter polymorphism (5-HTTLPR) on amygdala function: a meta-analysis. Molecular Psychiatry 18, 512–520.CrossRefGoogle ScholarPubMed
Nieminen, P, Rucker, G, Miettunen, J, Carpenter, J, Schumacher, M (2007). Statistically significant papers in psychiatry were cited more often than others. Journal of Clinical Epidemiology 60, 939–946.CrossRefGoogle ScholarPubMed
Nosek, BA, Spies, JR, Motyl, M (2012). Scientific Utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science 7, 615–631.CrossRefGoogle ScholarPubMed
Park, I-U, Peacey, MW, Munafò, MR (2014). Modelling the effects of subjective and objective decision making in scientific peer review. Nature 506, 93–96.CrossRefGoogle ScholarPubMed
Risch, N, Herrell, R, Lehner, T, Liang, K, Eaves, L, Hoh, J, Griem, A, Kovacs, M, Ott, J, Merikangas, KR (2009). Interaction between the serotonin transporter gene (5-HTTLPR), stressful life events, and risk of depression. JAMA 301, 2462–2471.CrossRefGoogle ScholarPubMed
Roest, AM, de Jonge, P, Williams, CD, de Vries, YA, Schoevers, RA, Turner, EH (2015). Reporting bias in clinical trials investigating the efficacy of second-generation antidepressants in the treatment of anxiety disorders: a report of 2 meta-analyses. JAMA Psychiatry 72, 500–510.CrossRefGoogle ScholarPubMed
Sharpley, CF, Palanisamy, SKA, Glyde, NS, Dillingham, PW, Agnew, LL (2014). An update on the interaction between the serotonin transporter promoter variant (5-HTTLPR), stress and depression, plus an exploration of non-confirming findings. Behavioural Brain Research 273, 89–105.CrossRefGoogle ScholarPubMed
Simmons, JP, Nelson, LD, Simonsohn, U (2011). False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 22, 1359–1366.CrossRefGoogle ScholarPubMed
Song, F, Parekh-Bhurke, S, Hooper, L, Loke, YK, Ryder, JJ, Sutton, AJ, Hing, CB, Harvey, I (2009). Extent of publication bias in different categories of research cohorts: a meta-analysis of empirical studies. BMC Medical Research Methodology 9, 79.CrossRefGoogle ScholarPubMed
Sullivan, PF (2007). Spurious genetic associations. Biological Psychiatry 61, 1121–1126.CrossRefGoogle ScholarPubMed
Sullivan, PF, Neale, MC, Kendler, KS (2000). Genetic epidemiology of major depression: review and meta-analysis. American Journal of Psychiatry 157, 1552–1562.CrossRefGoogle ScholarPubMed
Wendland, JR, Martin, BJ, Kruse, MR, Lesch, K-P, Murphy, DL (2006). Simultaneous genotyping of four functional loci of human SLC6A4, with a reappraisal of 5-HTTLPR and rs25531. Molecular Psychiatry 11, 224–226.CrossRefGoogle ScholarPubMed
Whiteford, HA, Degenhardt, L, Rehm, J, Baxter, AJ, Ferrari, AJ, Erskine, HE, Charlson, FJ, Norman, RE, Flaxman, AD, Johns, N, Burstein, R, Murray, CJL, Vos, T (2013). Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet 382, 1575–1586.CrossRefGoogle ScholarPubMed
Zammit, S, Owen, MJ, Lewis, G (2010). Misconceptions about gene–environment interactions in psychiatry. Evidence-Based Mental Health 13, 65–68.CrossRefGoogle ScholarPubMed
Supplementary material: Image

de Vries supplementary material

Figure image

Download de Vries supplementary material(Image)
Image 2.6 MB
Supplementary material: Image

de Vries supplementary material

Figure image

Download de Vries supplementary material(Image)
Image 2.4 MB
Supplementary material: PDF

de Vries supplementary material

Flow chart

Download de Vries supplementary material(PDF)
PDF 44.7 KB
Supplementary material: File

de Vries supplementary material

Table

Download de Vries supplementary material(File)
File 144.9 KB
Supplementary material: File

de Vries supplementary material

Tables S1-S4

Download de Vries supplementary material(File)
File 29.3 KB