Hostname: page-component-8448b6f56d-tj2md Total loading time: 0 Render date: 2024-04-23T08:42:23.373Z Has data issue: false hasContentIssue false
  • English
  • Français

Adjustment of publication bias using a cumulative meta-analytic framework

Part of: Special Collection - Methods

Published online by Cambridge University Press:  28 June 2022

W. J. Canestaro Open the ORCID record for W. J. Canestaro [Opens in a new window] ,
E. B. Devine ,
A. Bansal ,
S. D. Sullivan  and
J. J. Carlson
W. J. Canestaro*
Affiliation:
School of Pharmacy, University of Washington, Seattle, WA, USA Washington Research Foundation, Seattle, WA, USA
E. B. Devine
Affiliation:
School of Pharmacy, University of Washington, Seattle, WA, USA
A. Bansal
Affiliation:
School of Pharmacy, University of Washington, Seattle, WA, USA
S. D. Sullivan
Affiliation:
School of Pharmacy, University of Washington, Seattle, WA, USA
J. J. Carlson
Affiliation:
School of Pharmacy, University of Washington, Seattle, WA, USA
*
* Author for correspondence: W. J. Canestaro, E-mail: canestaro@gmail.com
Get access Rights & Permissions [Opens in a new window]

Abstract

Objectives

Publication bias has the potential to adversely impact clinical decision making and patient health if alternative decisions would have been made had there been complete publication of evidence.

Methods

The objective of our analysis was to determine if earlier publication of the complete evidence on rosiglitazone’s risk of myocardial infarction (MI) would have changed clinical decision making at an earlier point in time. We tested several methods for adjustment of publication bias to assess the impact of potential time delays to identifying the MI effect. We then performed a cumulative meta-analysis (CMA) for both published studies (published-only data set) and all studies performed (comprehensive data set). We then created an adjusted data set using existing methods of adjustment for publication bias (Harbord regression, Peter’s regression, and the nonparametric trim and fill method) applied to the limited data set. Finally, we compared the time to the decision threshold for each data set using CMA.

Results

Although published-only and comprehensive data sets did not provide notably different final summary estimates [OR = 1.4 (95 percent confidence interval [CI]: .95–2.05) and 1.42 (95 percent CI: 1.03–1.97)], the comprehensive data set reached the decision threshold 36 months earlier than the published-only data set. All three adjustment methods tested did not show a differential time to decision threshold versus the published-only data set.

Conclusions

Complete access to studies capturing MI risk for rosiglitazone would have led to the evidence reaching a clinically meaningful decision threshold 3 years earlier.

Keywords

Publication biasCumulative meta-analysisStatistical adjustment
Type
Method
Information
International Journal of Technology Assessment in Health Care , Volume 38 , Issue 1 , 2022 , e61
Copyright
© The Author(s), 2022. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Sackett, DL, Rosenberg, WM, Gray, JA, Haynes, RB, Richardson, WS (1996) Evidence based medicine: What it is and what it isn’t. BMJ 312, 71–72.CrossRefGoogle ScholarPubMed
Sackett, DL, Rosenberg, WM (1995) The need for evidence-based medicine. J R Soc Med 88, 620–624.CrossRefGoogle ScholarPubMed
Bornmann, L, Mutz, R (2015) Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references. J Assoc Inf Sci Technol 66, 2215–2222.CrossRefGoogle Scholar
Medicine USNLo (2017) Detailed indexing statistics: 1965–2016. Available at: https://www.nlm.nih.gov/bsd/index_stats_comp.html. Accessed 2017.Google Scholar
Ioannidis, JP (1998) Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 279, 281–286.CrossRefGoogle ScholarPubMed
Hopewell, S, Loudon, K, Clarke, MJ, Oxman, AD, Dickersin, K (2009) Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev 2009, MR000006.Google Scholar
Hopewell, S, Clarke, M, Stewart, L, Tierney, J (2007) Time to publication for results of clinical trials. Cochrane Database Syst Rev 2007, MR000011.Google Scholar
Blumle, A, Antes, G, Schumacher, M, Just, H, von Elm, E (2008) Clinical research projects at a German medical faculty: Follow-up from ethical approval to publication and citation by others. J Med Ethics 34, e20.CrossRefGoogle Scholar
von Elm, E, Rollin, A, Blumle, A, et al (2008) Publication and non-publication of clinical trials: Longitudinal study of applications submitted to a research ethics committee. Swiss Med Wkly 138, 197–203.Google ScholarPubMed
Stern, JM, Simes, RJ (1997) Publication bias: Evidence of delayed publication in a cohort study of clinical research projects. BMJ (Clin Res Ed) 315, 640–645.CrossRefGoogle Scholar
Pich, J, Carne, X, Arnaiz, JA, et al (2003) Role of a research ethics committee in follow-up and publication of results. Lancet 361, 1015–1016.CrossRefGoogle ScholarPubMed
Decullier, E, Lheritier, V, Chapuis, F (2005) Fate of biomedical research protocols and publication bias in France: Retrospective cohort study. BMJ (Clin Res Ed) 331, 19.CrossRefGoogle ScholarPubMed
Easterbrook, PJ, Berlin, JA, Gopalan, R, Matthews, DR (1991) Publication bias in clinical research. Lancet 337, 867–872.CrossRefGoogle ScholarPubMed
Dickersin, K, Min, YI, Meinert, CL (1992) Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA 267, 374–378.CrossRefGoogle ScholarPubMed
Lee, K, Bacchetti, P, Sim, I (2008) Publication of clinical trials supporting successful new drug applications: A literature analysis. PLoS Med 5, e191.CrossRefGoogle ScholarPubMed
Lundh, A, Sismondo, S, Lexchin, J, Busuioc, OA, Bero, L (2012) Industry sponsorship and research outcome. Cochrane Database Syst Rev 12, MR000033.Google ScholarPubMed
Moreno, SG, Sutton, AJ, Turner, EH, et al (2009) Novel methods to deal with publication biases: Secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications. BMJ (Clin Res Ed) 339, b2981.CrossRefGoogle ScholarPubMed
Mark, TL, Swait, J (2004) Using stated preference and revealed preference modeling to evaluate prescribing decisions. Health Econ 13, 563–573.CrossRefGoogle ScholarPubMed
Wells, G, Beaton, D, Shea, B, et al (2001) Minimal clinically important differences: Review of methods. J Rheumatol 28, 406–412.Google ScholarPubMed
Copay, AG, Subach, BR, Glassman, SD, Polly, DW, Schuler, TC (2007) Understanding the minimum clinically important difference: A review of concepts and methods. Spine J 7, 541–546.CrossRefGoogle ScholarPubMed
Nissen, SE, Wolski, K (2007) Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med 356, 2457–2471.CrossRefGoogle ScholarPubMed
Morrish, NJ, Wang, SL, Stevens, LK, Fuller, JH, Keen, H (2001) Mortality and causes of death in the WHO multinational study of vascular disease in diabetes. Diabetologia 44, S14–S21.CrossRefGoogle ScholarPubMed
Starner, CI, Schafer, JA, Heaton, AH, Gleason, PP (2008) Rosiglitazone and pioglitazone utilization from January 2007 through May 2008 associated with five risk-warning events. J Manag Care Pharm 14, 523–531.CrossRefGoogle ScholarPubMed
Morrow, RL, Carney, G, Wright, JM, et al (2010) Impact of rosiglitazone meta-analysis on use of glucose-lowering medications. Open Med 4, e50–e59.Google ScholarPubMed
Home, PD, Pocock, SJ, Beck-Nielsen, H, et al (2009) Rosiglitazone evaluated for cardiovascular outcomes in oral agent combination therapy for type 2 diabetes (RECORD): A multicentre, randomised, open-label trial. Lancet 373, 2125–2135.CrossRefGoogle ScholarPubMed
Nissen, SE, Wolski, K (2007) Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med 356, 2457–2471.CrossRefGoogle ScholarPubMed
Yusuf, S, Peto, R, Lewis, J, Collins, R, Sleight, P (1985) Beta blockade during and after myocardial infarction: An overview of the randomized trials. Prog Cardiovasc Dis 27, 335–371.CrossRefGoogle ScholarPubMed
Bradburn, MJ, Deeks, JJ, Berlin, JA, Russell Localio, A (2007) Much ado about nothing: A comparison of the performance of meta-analytical methods with rare events. Stat Med 26, 53–77.CrossRefGoogle ScholarPubMed
Higgins, JP, Green, S (2011) Cochrane handbook for systematic reviews of interventions, vol. 4. Chichester, UK: Wiley.Google Scholar
Higgins, J, Thompson, SG (2002) Quantifying heterogeneity in a meta-analysis. Stat Med 21, 1539–1558.CrossRefGoogle ScholarPubMed
Sterne, JA, Egger, M (2001) Funnel plots for detecting bias in meta-analysis: Guidelines on choice of axis. J Clin Epidemiol 54, 1046–1055.CrossRefGoogle ScholarPubMed
Song, F, Khan, KS, Dinnes, J, Sutton, AJ (2002) Asymmetric funnel plots and publication bias in meta-analyses of diagnostic accuracy. Int J Epidemiol 31, 88–95.CrossRefGoogle ScholarPubMed
Moreno, SG, Sutton, AJ, Ades, AE, et al (2009) Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study. BMC Med Res Methodol 9, 2.CrossRefGoogle ScholarPubMed
Emerson, SS, Fleming, TR (1990) Interim analyses in clinical trials. Oncology 4, 126–133; Discussion 134, 136.Google ScholarPubMed
Supplementary material: File

Canestaro et al. supplementary material

Canestaro et al. supplementary material

Download Canestaro et al. supplementary material(File)
File 158.1 KB