Skip to main content Accessibility help


  • Louise Brereton (a1) (a2), Philip Wahlster (a3) (a4), Kati Mozygemba (a3), Kristin Bakke Lysdahl (a5) (a6), Jake Burns (a7), Stephanie Polus (a7), Marcia Tummers (a8), Pietro Refolo (a9), Dario Sacchini (a9), Wojciech Leppert (a10) (a11), James Chilcott (a1), Christine Ingleton (a12), Clare Gardiner (a12) and Elizabeth Goyder (a1)...


Objectives: Internationally, funders require stakeholder involvement throughout health technology assessment (HTA). We report successes, challenges, and lessons learned from extensive stakeholder involvement throughout a palliative care case study that demonstrates new concepts and methods for HTA.

Methods: A 5-step “INTEGRATE-HTA Model” developed within the INTEGRATE-HTA project guided the case study. Using convenience or purposive sampling or directly / indirectly identifying and approaching individuals / groups, stakeholders participated in qualitative research or consultation meetings. During scoping, 132 stakeholders, aged ≥ 18 years in seven countries (England, Italy, Germany, The Netherlands, Norway, Lithuania, and Poland), highlighted key issues in palliative care that assisted identification of the intervention and comparator. Subsequently stakeholders in four countries participated in face–face, telephone and / or video Skype meetings to inform evidence collection and / or review assessment results. An applicability assessment to identify contextual and implementation barriers and enablers for the case study findings involved twelve professionals in the three countries. Finally, thirteen stakeholders participated in a mock decision-making meeting in England.

Results: Views about the best methods of stakeholder involvement vary internationally. Stakeholders make valuable contributions in all stages of HTA; assisting decision making about interventions, comparators, research questions; providing evidence and insights into findings, gap analyses and applicability assessments. Key challenges exist regarding inclusivity, time, and resource use.

Conclusion: Stakeholder involvement is feasible and worthwhile throughout HTA, sometimes providing unique insights. Various methods can be used to include stakeholders, although challenges exist. Recognition of stakeholder expertise and further guidance about stakeholder consultation methods is needed.



Hide All
1. Cicchetti, A, Iacopino, V, Carletto, A, Marchetti, M, Mennini, FS. Il ruolo degli stakeholder nel processo di HTA. Ital Health Technol Assess. 2011;4:69.
2. Nielsen, CP, Lauritsen, SW, Kristensen, FB, et al. Involving stakeholders and developing a policy for stakeholder involvement in the European network for Health Technology Assessment, EUnetHTA. Int J Technol Assess Health Care. 2009;25:8491.
3. Yazdizadeh, B, Shahmoradi, S, Majdzadeh, R, et al. Stakeholder involvement in health technology assessment at national level: A study from Iran. Int J Technol Assess Health Care. 2016;32:181.
4. OECD. The OECD Health Project: Health technology and decision making. Paris Organisation for Economic Co-operation and Development (OECD); 2005. (accessed September 13, 2016).
5. Gauvin, FP, Abelson, J, Giacomini, M, Eyles, J, Lavis, JN. “It all depends”: Conceptualizing public involvement in the context of health technology assessment agencies. Soc Sci Med. 2010;70:15181526.
6. Daniels, N, van der Wilt, GJ. Health Technology Assessment, deliberative process, and ethically contested issues. Int J Technol Assess Health Care. 2016;32:1015.
7. INVOLVE Briefing notes for researchers: Involving the public in NHS, public health and social care research. 2012 INVOLVE, Eastleigh. (accessed October 3, 2016).
8. Popay, J, Collins, M. PiiAF: The public involvement impact assessment framework guidance. Lancaster: University of Lancaster; 2014.
9. Abelson, J, Giacomini, M, Lehoux, P, Gauvin, FP. Bringing ‘the public’ into health technology assessment and coverage policy decisions: From principles to practice. Health Policy. 2007;82:3750.
10. Pizzo, E, Doyle, C, Matthews, R, Barlow, J. Patient and public involvement: How much do we spend and what are the benefits? Health Expect. 2015;18:19181926.
11. Brereton, L, Wahlster, P, Lysdahl, K, et al. Integrated assessment of home based palliative care with and without reinforced caregiver support: ‘A demonstration of INTEGRATE-HTA methodological guidances’–Executive Summary. 2016. (accessed October 3, 2016).
12. Gomes, B, Calanzani, N, Curiale, V, McCrone, P, Higginson, IJ. Effectiveness and cost-effectiveness of home palliative care services for adults with advanced illness and their caregivers. Cochrane Database Syst Rev. 2013;6:CD007760.
13. Wahlster, P, Brereton, L, Burns, J, et al. Guidance on the integrated assessment of complex health technologies-The INTEGRATE-HTA Model. 2016. (accessed October 3, 2016).
14. National Institute for Health and Care Excellence (NICE). Methods for the development of NICE public health guidance (3rd ed.) (accessed December 12, 2014).
15. Brereton, L, Ingleton, C, Gardiner, C, et al. Lay and professional stakeholder involvement in scoping palliative care issues: Methods used in seven European countries. Palliat Med. 2017;31:181192.
16. Lampe, K, Mäkelä, M. EUnetHTA WP4–HTA Core Model for Medical and Surgical Interventions–Version 1.0 2008. (accessed January 10, 2015).
17. Lysdahl, KB, Brereton, L, Oortwijn, W, et al, eds. Guidance for assessing effectiveness, economic aspects, ethical aspects, socio-cultural aspects and legal aspects in complex technologies 2016. (accessed October 3, 2016).
18. Burns, J, Chilcott, JB, van Hoorn, R, Kievit, W, Rehfeuss, E. Guidance to assess effectiveness aspects. Guidance for assessing effectiveness, economic aspects, ethical aspects, socio-cultural aspects and legal aspects in complex technologies 2016. (accessed October 3, 2016).
19. Rohwer, A, Pfadenhauer, L, Burns, J, et al. Logic models help make sense of complexity in systematic reviews and health technology assessments. J Clin Epidemiol. 2017;83:3747.
20. Sampietro-Colom, L, Thomas, S. Rethinking stakeholder engagement and technology access in health technology assessment: Reactions to policy forum discussions. Int J Technol Assess Health Care. 2016;32: 200202.
21. Brandon, PR, Fukunaga, LL. The state of the empirical research literature on stakeholder involvement in program evaluation. Am J Eval. 2013;35:2644.
22. Gomes, B, Higginson, IJ. Evidence on home palliative care: Charting past, present, and future at the Cicely Saunders Institute–WHO Collaborating Centre for Palliative Care, Policy and Rehabilitation. Prog Palliat Care. 2013;21:204213.
23. Cavazza, M, Jommi, C. Stakeholders involvement by HTA Organisations: Why is so different?. Health Policy, 2012;105:236245.
24. Stephens, JM, Handke, B, Doshi, JA. International survey of methods used in health technology assessment (HTA): Does practice meet the principles proposed for good research. J Comp Eff Res. 2012;2:2944.



Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed