Skip to main content Accessibility help


  • Philip Wahlster (a1) (a2), Louise Brereton (a3) (a4), Jacob Burns (a5), Björn Hofmann (a6) (a7), Kati Mozygemba (a1) (a8), Wija Oortwijn (a9), Lisa Pfadenhauer (a10), Stephanie Polus (a10), Eva Rehfuess (a10), Imke Schilling (a1) (a8), Gert Jan van der Wilt (a11) and Ansgar Gerhardus (a1) (a8)...


Objectives: Current health technology assessment (HTA) is not well equipped to assess complex technologies as insufficient attention is being paid to the diversity in patient characteristics and preferences, context, and implementation. Strategies to integrate these and several other aspects, such as ethical considerations, in a comprehensive assessment are missing. The aim of the European research project INTEGRATE-HTA was to develop a model for an integrated HTA of complex technologies.

Methods: A multi-method, four-stage approach guided the development of the INTEGRATE-HTA Model: (i) definition of the different dimensions of information to be integrated, (ii) literature review of existing methods for integration, (iii) adjustment of concepts and methods for assessing distinct aspects of complex technologies in the frame of an integrated process, and (iv) application of the model in a case study and subsequent revisions.

Results: The INTEGRATE-HTA Model consists of five steps, each involving stakeholders: (i) definition of the technology and the objective of the HTA; (ii) development of a logic model to provide a structured overview of the technology and the system in which it is embedded; (iii) evidence assessment on effectiveness, economic, ethical, legal, and socio-cultural aspects, taking variability of participants, context, implementation issues, and their interactions into account; (iv) populating the logic model with the data generated in step 3; (v) structured process of decision-making.

Conclusions: The INTEGRATE-HTA Model provides a structured process for integrated HTAs of complex technologies. Stakeholder involvement in all steps is essential as a means of ensuring relevance and meaningful interpretation of the evidence.



Hide All
1. European Network for Health Technology Assessment (EUnetHTA). HTA definition. 2016. (accessed December 5, 2016).
2. Medical Research Council. Developing and evaluating complex interventions: New guidance. United Kingdom: MRC; 2008.
3. Horton, R. Offline: The error of our health technology assessment ways. Lancet. 2013;382:1318.
4. Benton, V, Stewart, T. Multiple criteria decision analysis - An integrated approach. New York: Springer US; 2002.
5. Wahlster, P, Goetghebeur, M, Kriza, C, et al. Balancing costs and benefits at different stages of medical innovation: A systematic review of Multi-criteria decision analysis (MCDA). BMC Health Serv Res. 2015; 15: 262.
6. Wahlster, P, Goetghebeur, M, Schaller, S, Kriza, C, Kolominsky-Rabas, P, EMN’, National Leading-Edge Cluster Medical Technologies ‘Medical Valley. Exploring the perspectives and preferences for HTA across German healthcare stakeholders using a multi-criteria assessment of a pulmonary heart sensor as a case study. Health Res Policy Syst. 2015;13:24.
7. Brereton, L, Wahlster, P, Lysdahl, KB, et al. Integrated assessment of home based palliative care with and without reinforced caregiver support: A demonstration of INTEGRATE-HTA methodological guidances. 2016. (accessed December 5, 2016).
8. Lysdahl, KB, Mozygemba, K, Burns, J, et al. Guidance for assessing effectiveness, economic aspects, ethical aspects, socio-cultural aspects and legal aspects in complex technologies 2016. (accessed December 5, 2016).
9. Pfadenhauer, L, Rohwer, A, Burns, J, et al. Guidance for the assessment of context and implementation in health technology assessments (HTA) and systematic reviews of complex interventions: The context and implementation of complex interventions (CICI) framework. 2016. (accessed December 5, 2016).
10. Booth, A, Noyes, J, Flemming, K, et al. Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions 2016. (accessed December 5, 2016).
11. Rohwer, A, Booth, A, Pfadenhauer, L, et al. Guidance on the use of logic models in health technology assessments of complex interventions. 2016. (accessed December 5, 2016).
12. Van Hoorn, R, Tummers, M, Kievit, W, Van der Wilt, GJ. Guidance for the assessment of treatment moderation and patients' preferences. 2016. (accessed December 5, 2016).
13. Wahlster, P, Brereton, L, Burns, J, et al. Integrated assessment of complex health technologies - The INTEGRATE-HTA Model. 2016. (accessed December 5, 2016).
14. Brereton, L, Ingleton, C, Gardiner, C, et al. Lay and professional stakeholder involvement in scoping palliative care issues: Methods used in seven European countries. Palliat Med. 2017;31:181192.
15. Squires, JE, Valentine, JC, Grimshaw, JM. Systematic reviews of complex interventions: Framing the review question. J Clin Epidemiol. 2013;66:12151222.
16. Goetghebeur, MM, Wagner, M, Khoury, H, et al. Evidence and value: Impact on DEcisionMaking–the EVIDEM framework and potential applications. BMC Health Serv Res. 2008;8:270.
17. Lampe, K, Makela, M, Garrido, MV, et al. The HTA Core Model: A novel method for producing and reporting health technology assessments. Int J Technol Assess Health Care. 2009;25 (Suppl 2):920.
18. International Network of Agencies for Health Technology Assessment (IANHTA). Glossary. 2015. (accessed December 5, 2016).
19. The Joanna Briggs Institute. Joanna Briggs Institute Reviewers’ Manual 2014 Edition. Australia: Joanna Briggs Institute, 2014.
20. Rohwer, A, Pfadenhauer, L, Burns, J, et al. Logic models help make sense of complexity in systematic reviews and health technology assessments. J Clin Epidemiol. 2017;83:3747.
21. Polus, S, Pfadenhauer, L, Brereton, L, et al. A consultation guide for assessing the applicability of health technologies - a case study. Int J Technol Assess Health Care. 2017 [This issue.]
22. DeGroot, MH. Reaching a consensus. J Am Stat Assoc. 1974;69:118121.
23. Lysdahl, KB, Hofmann, B. Complex health care interventions: Characteristics relevant for ethical analysis in health technology assessment. GMS Health Technol Assess. 2016;12:Doc01.
24. Bond, K, Weeks, L. Using the INTEGRATE-HTA guidance: Experience from CADTH. Int J Technol Assess Health Care. 2017 [This issue.]



Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed