Skip to main content Accessibility help


  • David Tordrup (a1), Christos Chouaid (a2), Pim Cuijpers (a3), William Dab (a4), Johanna Maria van Dongen (a5), Jaime Espin (a6), Bengt Jönsson (a7), Christian Léonard (a8), David McDaid (a9), Martin McKee (a10), José Pereira Miguel (a11), Anita Patel (a12), Jean-Yves Reginster (a13), Walter Ricciardi (a14), Maureen Rutten-van Molken (a15), Valentina Prevolnik Rupel (a16), Tracey Sach (a17), Franco Sassi (a18), Norman Waugh (a19) and Roberto Bertollini (a20)...


Background: The importance of economic evaluation in decision making is growing with increasing budgetary pressures on health systems. Diverse economic evidence is available for a range of interventions across national contexts within Europe, but little attention has been given to identifying evidence gaps that, if filled, could contribute to more efficient allocation of resources. One objective of the Research Agenda for Health Economic Evaluation project is to determine the most important methodological evidence gaps for the ten highest burden conditions in the European Union (EU), and to suggest ways of filling these gaps.

Methods: The highest burden conditions in the EU by Disability Adjusted Life Years were determined using the Global Burden of Disease study. Clinical interventions were identified for each condition based on published guidelines, and economic evaluations indexed in MEDLINE were mapped to each intervention. A panel of public health and health economics experts discussed the evidence during a workshop and identified evidence gaps.

Results: The literature analysis contributed to identifying cross-cutting methodological and technical issues, which were considered by the expert panel to derive methodological research priorities.

Conclusions: The panel suggests a research agenda for health economics which incorporates the use of real-world evidence in the assessment of new and existing interventions; increased understanding of cost-effectiveness according to patient characteristics beyond the “-omics” approach to inform both investment and disinvestment decisions; methods for assessment of complex interventions; improved cross-talk between economic evaluations from health and other sectors; early health technology assessment; and standardized, transferable approaches to economic modeling.



Hide All
1. Thomson, S, Foubister, T, Mossialos, E. Financing health care in the European Union: Challenges and policy responses. Geneva: World Health Organization; 2009.
2. Busse, R, Blumel, M, Scheller-Kreinsen, D, Zentner, A. Tackling chronic disease in Europe - Strategies, interventions and challenges. Obs Stud Ser No 20. 2010. (accessed December 1, 2013).
3. European Commission and the Economic Policy Committee (AWG). Joint report on health systems. Occasional Papers 74. 2010. (accessed December 3, 2013).
4. OECD. OECD StatExtracts. Published 2015 (accessed December 3, 2013).
5. Banta, D. The development of health technology assessment. Health Policy (New York). 2003;63:121132. doi:10.1016/S0168-8510(02)00059-3.
6. Elshaug, AG, Hiller, JE, Moss, JR. Exploring policy-makers’ perspectives on disinvestment from ineffective healthcare practices. Int J Technol Assess Health Care. 2008;24:19.
7. Elshaug, AG, Hiller, JE, Tunis, SR, Moss, JR. Challenges in Australian policy processes for disinvestment from existing, ineffective health care practices. Aust New Zealand Health Policy. 2007;4:23. doi:10.1186/1743-8462-4-23.
8. Aguiar-Ibáñez, R, Nixon, J, Glanville, J, et al. Economic evaluation databases as an aid to healthcare decision makers and researchers. Expert Rev Pharmacoecon Outcomes Res. 2005;5:721732. doi:10.1586/14737167.5.6.721.
9. Buxton, MJ. Economic evaluation and decision making in the UK. Pharmacoeconomics. 2006;24:11331142. doi:10.2165/00019053-200624110-00009.
10. Murray, CJL, Vos, T, Lozano, R, et al. Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: A systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380:21972223. doi:10.1016/S0140-6736(12)61689-4.
11. Koster J. PubReMiner. Published 2014 (accessed December 15, 2013).
12. Karyotaki, E, Tordrup, D, Buntrock, C, et al. Economic evidence for the clinical management of major depressive disorder: A systematic review and quality appraisal of economic evaluations alongside randomised controlled trials. Epidemiol Psychiatr Sci. 2016. [Epub ahead of print]. doi:10.1017/S2045796016000421.
13. Ali, S, Stone, MA, Peters, JL, Davies, MJ, Khunti, K. The prevalence of co-morbid depression in adults with Type 2 diabetes: A systematic review and meta-analysis. Diabet Med. 2006;23:11651173. doi:10.1111/j.1464-5491.2006.01943.x.
14. Janssen-Heijnen, ML, Schipper, RM, Razenberg, PP, Crommelin, MA, Coebergh, J-WW. Prevalence of co-morbidity in lung cancer patients and its relationship with treatment: A population-based study. Lung Cancer. 1998;21:105113. doi:10.1016/S0169-5002(98)00039-7.
15. Currie, SR, Wang, J. Chronic back pain and major depression in the general Canadian population. Pain. 2004;107:5460. doi:10.1016/j.pain.2003.09.015.
16. Forster, A, Young, J. Incidence and consequences offalls due to stroke: A systematic inquiry. BMJ. 1995;311:8386. doi:10.1136/bmj.311.6997.83.
17. Wolf, PA, D'Agostino, RB, Kannel, WB, Bonita, R, Belanger, AJ. Cigarette smoking as a risk factor for stroke: The Framingham Study. JAMA. 1988;259:10251029.
18. Hecht, SS. Tobacco smoke carcinogens and lung cancer. J Natl Cancer Inst. 1999;91:11941210. doi:10.1093/jnci/91.14.1194.
19. Mannino, DM, Buist, AS. Global burden of COPD: Risk factors, prevalence, and future trends. Lancet. 2007;370:765773.
20. Yusuf, S, Reddy, S, Ounpuu, S, Anand, S. Global burden of cardiovascular diseases: Part I: General Considerations, the epidemiologic transition, risk factors, and impact of urbanization. Circulation. 2001;104:27462753. doi:10.1161/hc4601.099487.
21. Feldman, DE, Rossignol, M, Shrier, I, Abenhaim, L. Smoking: A risk factor for development of low back pain in adolescents. Spine (Phila Pa 1976). 1999;24:2492.
22. MacMahon, S, Peto, R, Collins, R, et al. Blood pressure, stroke, and coronary heart disease: Part 1, prolonged differences in blood pressure: Prospective observational studies corrected for the regression dilution bias. Lancet. 1990;335:765774.
23. Kang, JG, Park, C-Y. Anti-obesity drugs: A review about their effects and safety. Diabetes Metab J. 2012;36:1325. doi:10.4093/dmj.2012.36.1.13.
24. Manson, JE. A prospective study of exercise and incidence of diabetes among US male physicians. JAMA. 1992;268:63. doi:10.1001/jama.1992.03490010065031.
25. Barnett, K, Mercer, SW, Norbury, M, Watt, G, Wyke, S, Guthrie, B. Epidemiology of multimorbidity and implications for health care, research, and medical education: A cross-sectional study. Lancet. 2012;380: 3743.
26. Tordrup, D, Attwill, A, Crosby, L, Bertollini, R. Research agenda for health economic evaluation (forthcoming). August 1, 2015).
27. Britton, A, McKee, M, Black, N, McPherson, K, Sanderson, C, Bain, C. Threats to applicability of randomised trials: Exclusions and selective participation. J Health Serv Res Policy. 1999;4: 112121.
28. Hägg, L, Johansson, C, Jansson, J-H, Johansson, L. External validity of the ARISTOTLE trial in real-life atrial fibrillation patients. Cardiovasc Ther. 2014;32:214218. doi:10.1111/1755-5922.12087.
29. Simon, ST, Higginson, IJ, Harding, R, et al. Enhancing patient-reported outcome measurement in research and practice of palliative and end-of-life care. Support Care Cancer. 2012;20:15731578.
30. Cohen, D. Attacks on publicly funded trials: What happens when industry does not want to know the answer. BMJ. 2015;350:h1701.
31. May, P, Normand, C, Morrison, RS. Economic impact of hospital inpatient palliative care consultation: Review of current evidence and directions for future research. J Palliat Med. 2014;17:10541063.
32. Longworth, L, Sculpher, MJ, Bojke, L, Tosh, JC. Bridging the gap between methods research and the needs of policy makers: A review of the research priorities of the National Institute for Health and Clinical Excellence. Int J Technol Assess Health Care. 2011;27:180187. doi:10.1017/S0266462311000043.
33. Drummond, M, Marshall, D. IHE Methodology forum: Prioritizing methodological research in the evaluation of health technologies in Canada. Alberta, Canada: IHE; 2010.
34. Chalkidou, K, Whicher, D, Kary, W, Tunis, S. Comparative effectiveness research priorities: Identifying critical gaps in evidence for clinical and health policy decision making. Int J Technol Assess Health Care. 2009;25:241248. doi:10.1017/S0266462309990225.
35. European Commission. Public consultation on strengthening EU cooperation on Health Technology Assessment (HTA). Published 2016 (accessed December 22, 2016).
36. George, B, Harris, AH, Mitchell, AJ. Cost effectiveness analysis and the consistency of decision making: Evidence from pharmaceutical reimbursement in Australia 1991–96. Australia: Centre for Health Program Evaluation Melbourne; 1999.
37. Devlin, N, Parkin, D. Does NICE have a cost-effectiveness threshold and what other factors influence its decisions? A binary choice analysis. Health Econ. 2004;13:437452. doi:10.1002/hec.864.
38. Claxton, K, Martin, S, Soares, M, et al. Methods for the estimation of the National Institute for Health and Care Excellence cost-effectiveness threshold. Southampton (UK): NIHR Journals Library; 2015 Feb. (Health Technology Assessment, No. 19.14.) Appendix 1, Systematic review of the literature on the cost-effectiveness threshold. (accessed May 5, 2015)
39. Cleemput, I, Neyt, M, Thiry, N, De Laet, C, Leys, M. Threshold values for cost-effectiveness in health care. KCE Rep 100C. 2009. (accessed February 23, 2014).
40. Claxton, K, Sculpher, M, Palmer, S, Culyer, AJ. Causes for concern: Is NICE failing to uphold its responsibilities to all NHS patients? Health Econ. 2015;24:17. doi:10.1002/hec.3130.
41. Claxton, K, Martin, S, Soares, M, et al. Methods for the estimation of the National Institute for Health and Care Excellence cost-effectiveness threshold. Health Technol Assess. 2015;19:1504. doi:10.3310/hta19140.
42. Bobinac, A, van Exel, NJA, Rutten, FFH, Brouwer, WBF. Willingness to pay for a quality-adjusted life-year: The individual perspective. Value Health. 2010;13:10461055. doi:10.1111/j.1524-4733.2010.00781.x.
43. Simoens, S. Pricing and reimbursement of orphan drugs: The need for more transparency. Orphanet J Rare Dis. 2011;6:11721176.
44. Baltussen, R, Niessen, L. Priority setting of health interventions: The need for multi-criteria decision analysis. Cost Eff Resour Alloc. 2006;4:14.
45. Tony, M, Wagner, M, Khoury, H, et al. Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): Field testing of the EVIDEM framework for coverage decisions by a public payer in Canada. BMC Health Serv Res. 2011;11:329. doi:10.1186/1472-6963-11-329.
46. Pearson, S, Littlejohns, P. Reallocating resources: How should the National Institute for Health and Clinical Excellence guide disinvestment efforts in the National Health Service? J Health Serv Res Policy. 2007;12:160165. doi:10.1258/135581907781542987.
47. Hughes, DA, Ferner, RE. New drugs for old: Disinvestment and NICE. BMJ. 2010;340:c572. doi:10.1136/bmj.c572.
48. Eddama, O, Coast, J. Use of economic evaluation in local health care decision-making in England: A qualitative investigation. Health Policy. 2009;89:261270. doi:10.1016/j.healthpol.2008.06.004.
49. Wang, R, Lagakos, SW, Ware, JH, Hunter, DJ, Drazen, JM. Statistics in medicine — Reporting of subgroup analyses in clinical trials. N Engl J Med. 2007;357:21892194.
50. Godwin, M, Ruhland, L, Casson, I, et al. Pragmatic controlled clinical trials in primary care: The struggle between external and internal validity. BMC Med Res Methodol. 2003;3:28. doi:10.1186/1471-2288-3-28.
51. Kaiser, C, Brunner-La Rocca, HP, Buser, PT, et al. Incremental cost-effectiveness of drug-eluting stents compared with a third-generation bare-metal stent in a real-world setting: Randomised Basel Stent Kosten Effektivitäts Trial (BASKET). Lancet. 2005;366:921929. doi:10.1016/S0140-6736(05)67221-2.
52. Pietri, G, Masoura, P. Market access and reimbursement: The increasing role of real-world evidence. Value Health. 2014;17:A450A451. doi:10.1016/j.jval.2014.08.1216.
53. Davis, C, Lexchin, J, Jefferson, T, Gøtzsche, P, McKee, M. “Adaptive pathways” to drug authorisation: Adapting to industry? BMJ. 2016;354:i4437
54. New, JP, Bakerly, ND, Leather, D, Woodcock, A. Obtaining real-world evidence: The Salford Lung Study. Thorax. 2014;69:11521154. doi:10.1136/thoraxjnl-2014-205259.
55. Tashkin, DP, Ferguson, GT. Combination bronchodilator therapy in the management of chronic obstructive pulmonary disease. Respir Res. 2013;14:49. doi:10.1186/1465-9921-14-49.
56. Tosh, J, Stevenson, M, Akehurst, R. Health economic modelling of treatment sequences for rheumatoid arthritis: A systematic review. Curr Rheumatol Rep. 2014;16:447. doi:10.1007/s11926-014-0447-2.
57. Brilleman, SL, Purdy, S, Salisbury, C, Windmeijer, F, Gravelle, H, Hollinghurst, S. Implications of comorbidity for primary care costs in the UK: A retrospective observational study. Br J Gen Pract. 2013;63:e274-82. doi:10.3399/bjgp13X665242.
58. Tappenden, P, Chilcott, J, Brennan, A, Squires, H, Stevenson, M. Whole disease modeling to inform resource allocation decisions in cancer: A methodological framework. Value Health. 2012;15:11271136.
59. Tappenden, P, Chilcott, J, Brennan, A, Squires, H, Glynne-Jones, R, Tappenden, J. Using whole disease modeling to inform resource allocation decisions: Economic evaluation of a clinical guideline for colorectal cancer using a single model. Value Health. 2013;16: 542553.
60. Tosh, J, Kearns, B, Brennan, A, et al. Innovation in health economic modelling of service improvements for longer-term depression: Demonstration in a local health community. BMC Health Serv Res. 2013;13:150. doi:10.1186/1472-6963-13-150.
61. Hunger, M, Thorand, B, Schunk, M, et al. Multimorbidity and health-related quality of life in the older population: Results from the German KORA-age study. Health Qual Life Outcomes. 2011;9:53. doi:10.1186/1477-7525-9-53.
62. Logsdon, RG, Gibbons, LE, McCurry, SM, Teri, L. Assessing quality of life in older adults with cognitive impairment. Psychosom Med. 2002;64:510519.
63. Bausewein, C, Simon, ST, Benalia, H, et al. Implementing patient reported outcome measures (PROMs) in palliative care-users’ cry for help. Heal Qual Life Outcomes. 2011;9:111.
64. Personal Social Services Research Unit (PSSRU). Adult Social Care Outcomes Toolkit - ASCOT. Published 2015 (accessed January 3, 2017).
65. Peña-Longobardo, LM, Oliva-Moreno, J, Hidalgo-Vega, Á, Miravitlles, M. Economic valuation and determinants of informal care to disabled people with Chronic Obstructive Pulmonary Disease (COPD). BMC Health Serv Res. 2015;15:101. doi:10.1186/s12913-015-0759-6.
66. Ramsey, S, Willke, R, Briggs, A, et al. Good research practices for cost-effectiveness analysis alongside clinical trials: The ISPOR RCT-CEA Task Force report. Value Health. 2005;8:521533. doi:10.1111/j.1524-4733.2005.00045.x.
67. Drummond, MF, Jefferson, TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. BMJ. 1996;313:275283. doi:10.1136/bmj.313.7052.275.
68. IMS Core Diabetes Model. Published 2013 (accessed August 1, 2015).
69. Kristensen, FB, Lampe, K, Chase, DL, et al. Practical tools and methods for health technology assessment in Europe: Structures, methodologies, and tools developed by the European Network for Health Technology Assessment, EUnetHTA. Int J Technol Assess Health Care. 2009;25 (Suppl 2):18. doi:10.1017/S0266462309990626.
70. NICE International. The Gates Reference Case - What it is, why it's important, and how to use it. 2014. (accessed July 11, 2017).
71. Husereau, D, Drummond, M, Petrou, S, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—Explanation and elaboration: A report of the ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force. July 11, 2017).
72. World Health Organization. The economics of the social determinants of health and health inequalities: A resource book. ISBN 978 92 4 154862 5. Geneva: WHO; 2013.
73. Lozano, R, Naghavi, M, Foreman, K, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380 (9859):2095-128.



Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed