Skip to main content Accessibility help


  • Pamela Royle (a1) and Ruairidh Milne (a1)


Objectives: To analyze sources searched in Cochrane reviews, to determine the proportion of trials included in reviews that are indexed in major databases, and to compare the quality of these trials with those from other sources.

Methods: All new systematic reviews in the Cochrane Library, Issue1 2001, that were restricted to randomized controlled trials (RCTs) or quasi-RCTs were selected. The sources searched in the reviews were recorded, and the trials included were checked to see whether they were indexed in four major databases. Trials not indexed were checked to determine how they could be identified. The quality of trials found in major databases was compared with those found from other sources.

Results: The range in the number of databases searched per review ranged between one and twenty-seven. The proportion of the trials in the four databases were Cochrane Controlled Trials Register=78.5%, MEDLINE=68.8%, Embase=65.0%, and Science/Social Sciences Citation Index=60.7%. Searching another twenty-six databases after Cochrane Controlled Trials Register (CCTR), MEDLINE, and Embase only found 2.4% additional trials. There was no significant difference between trials found in the CCTR, MEDLINE, and Embase compared with other trials, with respect to adequate allocation concealment or sample size.

Conclusions: There was a large variation between reviews in the exhaustiveness of the literature searches. CCTR was the single best source of RCTs. Additional database searching retrieved only a small percentage of extra trials. Contacting authors and manufacturers to find unpublished trials appeared to be a more effective method of obtaining the additional better quality trials.



Hide All
Adams CE, Power A, Frederick K, Lefebvre C. 1994 An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care. Psychol Med. 24: 741- 748.
Brazier H, Begley CM. 1996 Selecting a database for literature searches in nursing: MEDLINE or CINAHL? J Adv Nurs. 24: 868- 875.
Burnham J, Shearer B. 1993 Comparison of CINAHL, EMBASE, and MEDLINE databases for the nurse researcher. Med Ref Serv Q. 12: 45- 57.
Castro AA, Clark OA, Atallah AN. 1997 Optimal search strategy for clinical trials in the Latin American and Caribbean Health Science Literature Database (LILACS). Rev Paul Med. 115: 1423- 1426.
Clarke M, Oxman AD, eds. 2001 Assessment of study quality. Cochrane Reviewers Handbook 4.1.2 [updated March 2001] Section 6. In: The Cochrane Library. Oxford: Update Software, Issue 2.
Clarke M, Oxman AD, eds. 2001 Locating and selecting studies. Cochrane Reviewers Handbook 4.1.2 [updated March 2001] Section 5. In: The Cochrane Library. Oxford: Update Software, Issue 2.
Early Breast Cancer Trialists' Collaborative Group. 2001 Tamoxifen for early breast cancer (Cochrane Review). In: The Cochrane Library. Oxford: Update Software, Issue 1.
Egger M, Ebrahim S, Smith GD. 2002 Where now for meta-analysis? Int J Epidemiol. 31: 1- 5.
Fergusson D, Laupacis A, Salmi LR et al. 2000 What should be included in meta-analyses? An exploration of methodological issues using the ISPOT meta-analyses. Int J Technol Assess Health Care. 16: 1109- 1119.
Jadad AJ, Moher D, Browman GP et al. 2000 Systematic reviews and meta-analysis on treatment of asthma: Critical evaluation. BMJ. 320: 537- 540.
Jadad AR, Cook DJ, Jones A et al. 1998 Methodology and reports of systematic reviews and meta-analyses: A comparison of Cochrane reviews with articles published in paper-based journals. JAMA. 280: 278- 280.
Juni P, Altman DG, Egger M. 2001 Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ. 323: 42- 46.
Juni P, Holenstein F, Sterne J et al. 2002 Direction and impact of language bias in meta-analyses of controlled trials: Empirical study. Int J Epidemiol. 31: 115- 123.
Kjaergard LL, Nikolova D, Gluud C. 1999 Randomized clinical trials in hepatology: Predictors of quality. Hepatology. 30: 1134- 1138.
Kjaergard LL, Villumsen J, Gluud C. 2001 Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 135: 982- 989.
Kleijnen J, Knipschild P. 1992 The comprehensiveness of MEDLINE and EMBASE computer searches. Searches for controlled trials of homoeopathy, ascorbic acid for common cold and ginkgo biloba for cerebral insufficiency and intermittent claudication. Pharm Weekbl Sci. 14: 316- 320.
Langham J, Thompson E, Rowan K. 1999 Identification of randomized controlled trials from the emergency medicine literature: Comparison of hand searching versus MEDLINE searching. Ann Emerg Med. 34: 25- 34.
McAuley L, Pham B, Tugwell P, Moher D. 2000 Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet. 356: 1228- 1231.
McDonald S, Taylor L, Adams C. 1999 Searching the right database. A comparison of four databases for psychiatry journals. Health Libr Rev. 16: 151- 156.
Sterne JA, Egger M, Davey SG. 2001 Investigating and dealing with publication and other biases. In: Egger M, Davey SG, Altman DG, eds. Systematic reviews in health care: Meta-analysis in context. London: BMJ Books; 189- 208.
Sterne JA, Gavaghan D, Egger M. 2000 Publication and related bias in meta-analysis: Power of statistical tests and prevalence in the literature. J Clin Epidemiol. 53: 1119- 1129.
Stevens A, Colin-Jones A, Gabbay J. 1995 “Quick and clean”: Authoritative health technology assessment for local health care contracting. Health Trends. 27: 37- 42.
Suarez-Almazor ME, Belseck E, Homik J et al. 2000 Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials. 21: 476- 487.
Topfer LA, Parada A, Menon D et al. 1999 Comparison of literature searches on quality and costs for health technology assessment using the MEDLINE and EMBASE databases. Int J Technol Assess Health Care. 15: 297- 303.
Vickers AJ, Smith C. 2000 Incorporating data from dissertations in systematic reviews. Int J Technol Assess Health Care. 16: 711- 713.



Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed