Skip to main content Accessibility help


  • Julia Kreis (a1), Dimitra Panteli (a2) and Reinhard Busse (a2)


Objectives: Reporting bias potentially threatens the validity of results in health technology assessment (HTA) reports. Our study aimed to explore policies and practices of HTA agencies regarding strategies to include previously unpublished data in their assessments, focusing on requests to industry for unpublished data.

Methods: We included international HTA agencies with publicly available methods papers as well as HTA reports. From the methods papers and recent reports we extracted information on requests to industry and on searches in trial registries, regulatory authority Web sites and for conference abstracts.

Results: Eighteen HTA agencies and seventy-three reports were included. Agencies’ methods papers showed variability regarding requests to industry (requests are routinely carried out in seven cases, not mentioned in six, at the discretion of HTA authors in three, and based on manufacturer applications in two), which were reflected in the reports investigated. As reporting of requests was limited, it often remained unclear whether unpublished data had been received. Searches in trial registries, at regulatory authorities or for conference abstracts are described as a routine or optional part of the search strategy in the methods papers of 9, 11, and 8 included agencies, respectively. A total of 52 percent, 39 percent, and 16 percent of reports described searches in trial registries, at regulatory agencies, and hand searching of conference proceedings.

Conclusion: International HTA agencies currently differ considerably in their efforts to address the issue of unpublished data. Requests to industry may constitute one strategy to access and include unpublished data, while agencies can learn from each other concerning successful practice.



Hide All
1. Sterne, JAC, Egger, M, Moher, D, on behalf of the Cochrane Bias Methods Group. Addressing reporting biases. In: Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions. Chichester, UK: Cochrane Collaboration and John Wiley & Sons; 2008.
2. McGauran, N, Wieseler, B, Kreis, J, et al. Reporting bias in medical research - A narrative review. Trials. 2010;11:37.
3. Higgins, JPT, Altman, DG, on behalf of the Cochrane Statistical Methods Group and the Cochrane Bias Methods Group. Assessing risk of bias in included studies. In: Higgins, JPT, Green, S, eds. Cochrane handbook for systematic reviews of interventions. Chichester, UK: Cochrane Collaboration and John Wiley & Sons; 2008.
4. Song, F, Parekh, S, Hooper, L, et al. Dissemination and publication of research findings: An updated review of related biases. Health Technol Assess. 2010;14:iii, ix-xi, 1193.
5. Eyding, D, Lelgemann, M, Grouven, U, et al. Reboxetine for acute treatment of major depression: Systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ. 2010;341:c4737.
6. Godlee, F, Loder, E. Missing clinical trial data: Setting the record straight. BMJ. 2010;341:c5641.
7. Code of Federal Regulations. Title 21 - Food and Drugs Chapter - Food and Drug Administration of Health and Human Services. Subchapter D - Drugs for Human Use. Part 314 - Applications for FDA Approval to Market a New Drug. Subpart B–ApplicationsSec. 314.50 Content and format of an application (21CFR314.50).
8. Wieseler, B, McGauran, N, Kaiser, T. Finding studies on reboxetine: A tale of hide and seek. BMJ. 2010;341:c4942.
9. Laurance, J. Drug giants warned: Tell the truth on medicines. The Independent. 27.02.2008.
10. Depressing research. Lancet. 2004;363:1335.
11. De Angelis, C, Drazen, JM, Frizelle, FA, et al. Clinical trial registration: A statement from the International Committee of Medical Journal Editors. Lancet. 2004;364:911912.
12. New York State Attorney General. Settlement sets new standard for release of drug information [online]. (accessed December 2, 2012).
13. Wood, AJ. Progress and deficiencies in the registration of clinical trials. N Engl J Med. 2009;360:824830.
14. European Medicines Agency. About EU Clinical Trials Register [online]. (accessed August 29, 2013).
15. Turner, EH. Closing a loophole in the FDA Amendments Act. Science. 2008;322:4446.
16. Lefebvre, C, Manheimer, E, Glanville, J, on behalf of the Cochrane Information Retrieval Methods. Searching for studies. In: Higgins, JPT, Green, S, eds. Cochrane handbook for systematic reviews of interventions. Chichester, UK: Cochrane Collaboration and John Wiley & Sons; 2008.
17. Smith, R. What is publication? BMJ. 1999;318:142.
18. Yank, V, Tuohy, CV, Logan, AC, et al. Comparative effectiveness of in-hospital use of recombinant factor VIIa for off-label indications vs. usual care. Rockville (MD): Agency for Healthcare Research and Quality (US); 2010 May. Report No.: 10-EHC030-EF.
19. Fayter, D, Corbett, M, Heirs, M, Fox, D, Eastwood, A. A systematic review of photodynamic therapy in the treatment of pre-cancerous skin conditions, Barrett's oesophagus and cancers of the biliary tract, brain, head and neck, lung, oesophagus and skin. Health Technol Assess. 2010;14:1288.
20. Hopewell, S, McDonald, S, Clarke, M, Egger, M. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev. 2007;2:MR000010.
21. Smith, B, Carson, S, Fu, R, et al. Drug class review: Disease-modifying drugs for multiple sclerosis. Final update 1 report. Portland (OR): 2010. Oregon Health & Science University; 2010 Aug.
22. Wessling, A, Ramsberg, J. The review of antidepressants. Solna: TLV; 2008.
23. Metcalfe, S, Burgess, C, Laking, G, et al. Trastuzumab: Possible publication bias. Lancet. 2008;371:16461648.
24. Buchberger, B, Follman, M, Freyer, D, et al. Bedeutung von Wachstumsfaktoren für die Behandlung von chronischen Wunden am Beispiel des diabetischen Fußulcus. Köln: DAHTA; 2010. (Schriftenreihe Health Technology Assessment).
25. Coleman, CI, Baker, WL, Kluger, J, et al. Comparative effectiveness of angiotensin converting enzyme inhibitors or angiotensin II receptor blockers added to standard medical therapy for treating stable ischemic heart disease. Rockville (MD): 2009. Agency for Healthcare Research and Quality (US); 2009 Oct.
26. Neyt, M, Van den Bruel, A, Gailly, J, Thiry, N, Devriese, S. Tiotropium in the treatment of chronic obstructive pulmonary disease: Health technology assessment. KCE reports 108C. Brussels: Belgian Health Care Knowledge Centre (KCE); 2009. (Health Technology Assessment).
27. Gorenoi, V, Schönermark, MP, Hagen, A. Infektionsschutz in der Knieendoprothetik. Köln: DAHTA; 2010. (Schriftenreihe Health Technology Assessment).
28. Medical Services Advisory Committee. Cryotherapy for recurrent prostate cancer and renal cancer: Part A – Salvage cryotherapy for recurrent or persistent prostate cancer after radiotherapy. 2009. (MSAC application 1124).
29. Prayle, AP, Hurley, MN, Smyth, AR. Compliance with mandatory reporting of clinical trial results on Cross sectional study. BMJ. 2012;344:d7373.
30. Moher, D, Hopewell, S, Schulz, KF, et al. CONSORT 2010: Explanation and elaboration; Updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869.
31. Drummond, MF, Schwartz, JS, Jonsson, B, et al. Key principles for the improved conduct of health technology assessments for resource allocation decisions. Int J Technol Assess Health Care. 2008;24:244258.
32. Moher, D, Liberati, A, Tetzlaff, J, Altman, DG, Group, P. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Ann Intern Med. 2009;151:264269, W264.
33. Niederstadt, C, Droste, S. Reporting and presenting information retrieval processes: The need for optimizing common practice in health technology assessment. Int J Technol Assess Health Care. 2010;26:450457.
34. Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen. Selektive Serotonin- und Noradrenalin-Wiederaufnahmehemmer (SNRI) bei Patienten mit Depressionen: Abschlussbericht; Auftrag A05-20A; Version 1.1 [online]. 18.08.2010 (IQWiG-Berichte; Band 55).–20A_Abschlussbericht_SNRI_bei_Patienten_mit_Depressionen_V1-1.pdf (accessed August 29, 2013).
35. Jefferson, T, Jones, MA, Doshi, P, et al. Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Database Syst Rev. 2012;1:CD008965.
36. Payne, D. Tamiflu: The battle for secret drug data. BMJ. 2012;345:e7303.
37. Gesetz zur Neurordnung des Arzneimittelmarktes in der gesetzlichen Krankenversicherung (Arzneimittelmarktneuordnungsgesetz - AMNOG) vom 22. Dezember 2010. Bundesgesetzblatt Jahrgang. 2010 Teil I Nr. 67.
38. European Medicines Agency. European Medicines Agency widens public access to documents [online]. 30.11.2010. (accessed August 29, 2013).
39. European Medicines Agency. European Medicines Agency receives interim decisions of the General Court of the EU on access to clinical and non-clinical information [online]. 30.04.2013. (accessed August 29, 2013).
40. European Medicines Agency. Publication and access to clinical-trial data [online]. 24.06.2013. (accessed August 29, 2013).
41. Godlee, F. Clinical trial data for all drugs in current use. BMJ. 2012;345:e7304.


Type Description Title
Supplementary materials

Kreis Supplementary Material
Tables S1-S4

 Word (50 KB)
50 KB


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed