Skip to main content Accessibility help
Hostname: page-component-544b6db54f-bkjnw Total loading time: 0.318 Render date: 2021-10-18T07:06:22.846Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": true, "newCiteModal": false, "newCitedByModal": true, "newEcommerce": true, "newUsageEvents": true }


Published online by Cambridge University Press:  19 February 2018

Vladimir Vukovic
Institute of Public Health, Catholic University of the Sacred Heart, Rome
Carlo Favaretti
Institute of Public Health, Catholic University of the Sacred Heart, Rome
Walter Ricciardi
President of the Italian National Institute of Health, Rome
Chiara de Waure
Institute of Public Health, Catholic University of the Sacred Heart, Rome Department of Experimental Medicine, University of Perugia


Objectives: Evaluation is crucial for integration of e-Health/m-Health into healthcare systems and health technology assessment (HTA) could offer sound methodological basis for these evaluations. Aim of this study was to look for HTA reports on e-Health/m-Health technologies and to analyze their transparency, consistency and thoroughness, with the goal to detect areas that need improvement.

Methods: PubMed, ISI-WOS, and University of York – Centre for Reviews and Dissemination–electronic databases were searched to identify reports on e-Health/m-Health technologies, published up until April 1, 2016. The International Network of Agencies for Health Technology Assessment (INAHTA) checklist was used to evaluate transparency and consistency of included reports. Thoroughness was assessed by checking the presence of domains suggested by the European network for Health Technology Assessment (EUnetHTA) HTA Core Model.

Results: Twenty-eight reports published between 1999 and 2015 were included. Most were delivered by non-European countries (71.4 percent) and only 35.7 percent were classified as full reports. All the HTA reports defined the scope of research whereas more than 80 percent provided author details, summary, discussed findings, and conclusion. On the contrary, policy and research questions were clearly defined in around 30 percent and 50 percent of reports. With respect to the EUnetHTA Core Model, around 70 percent of reports dealt with effectiveness and economic evaluation, more than 50 percent described health problem and approximately 40 percent organizational and social aspects.

Conclusions: E-Health/m-Health technologies are increasingly present in the field of HTA. Yet, our review identified several missing elements. Most of the reports failed to respond to relevant assessment components, especially ethical, social and organizational implications.

Copyright © Cambridge University Press 2018 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)


1. Ahern, DK, Kreslake, JM, Phalen, JM, Bock, B. What is eHealth (6): perspectives on the evolution of eHealth research. J Med Internet Res. 2006;8:e4.CrossRefGoogle ScholarPubMed
2. Silber, D. The case for eHealth. Stud Health Technol Inform. 2003;100:327.Google Scholar
3. Saleh, A, Mosa, M, Yoo, I, Sheets, L. A systematic review of healthcare applications for smartphones. BMC Med Inform Decis Mak. 2012;12:67.Google Scholar
4. World Health Organization. mHealth: new horizons for health through mobile technologies. Observatory [Internet]. 2011. (accessed January 18, 2018).Google Scholar
5. Free, C, Phillips, G, Felix, L, Galli, L, Patel, V, Edwards, P. The effectiveness of M-health technologies for improving health and health services: a systematic review protocol. BMC Res Notes. 2010;3:250.CrossRefGoogle ScholarPubMed
6. European Commission. Green Paper on mobile Health (“mHealth”). 2014. (accessed January 18, 2018).Google Scholar
7. Cortez, N. The mobile health revolution? UC Davis Law Review. 2014; 47: (accessed January 18, 2018).Google Scholar
8. Research2Guidance. The mobile health global market report 2013–2017: the commercialisation of mHealth apps. 2013; 3.Google Scholar
9. Cortez, NG, Cohen, IG, Kesselheim, AS. Health Law, Ethics, and Human Rights. FDA Regulation of Mobile Health Technologies. 2014.Google Scholar
10. World Health Organization Global Observatory for eHealth. Telemedicine: opportunities and developments in Member States. Observatory [Internet]. 2010; 2:96. (accessed January 19, 2018).Google Scholar
11. May, C, Mort, M, Williams, T, Mair, F, Gask, L. Health technology assessment in its local contexts: studies of telehealthcare. Soc Sci Med. 2003;57:697710.CrossRefGoogle ScholarPubMed
12. Hofstede, J, de Bie, J, van Wijngaarden, B, Heijmans, M. Knowledge, use and attitude toward eHealth among patients with chronic lung diseases. Int J Med Inform. 2014;83:967974.CrossRefGoogle ScholarPubMed
13. Currie, M, Philip, LJ, Roberts, A. Attitudes towards the use and acceptance of eHealth technologies: a case study of older adults living with chronic pain and implications for rural healthcare. BMC Health Serv Res. 2015;15:162.CrossRefGoogle ScholarPubMed
14. Leys, M. Health care policy: qualitative evidence and health technology assessment. Health Policy. 2003;65:217226.CrossRefGoogle ScholarPubMed
15. Gagnon, M-P, Scott, RE. Striving for evidence in e-health evaluation: lessons from health technology assessment. J Telemed Telecare. 2005;11 (Suppl 2):S34-S36.CrossRefGoogle ScholarPubMed
16. Pandor, A, Thokala, P, Gomersall, T, et al. Home telemonitoring or structured telephone support programmes after recent discharge in patients with heart failure: systematic review and economic evaluation. Health Technol Assess. 2013; 17.Google Scholar
17. Kreps, GL. Evaluating new health information technologies: expanding the frontiers of health care delivery and health promotion. Stud Health Technol Inform. 2002;80:205212.Google ScholarPubMed
18. Moher, D, Liberati, A, Tetzlaff, J, Altman, DG, Group, PRISMA. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Int J Surg. 2010;8:336341.CrossRefGoogle ScholarPubMed
19. Hailey, D. Towards Transparency in Health Technology Assessment A Checklist for HTA Reports. Int J Technol Assess Health Care [Internet]. 2003;19 (1):17. Available from: CrossRefGoogle Scholar
20. European Network for Health Technology Assessment. HTA core model for diagnostic technologies 1.0R. 2008; 1-176.Google Scholar
21. The EUnetHTA JA 2. HTA Core Model ® Online User guide. 2015; 2:1-27.Google Scholar
22. Simpson, J, Doze, S, Urness, D, Hailey, D, Jacobs, P. An assessment of routine telepsychiatry services (Structured abstract) [Internet]. Health Technology Assessment Database. 1999. Google Scholar
23. Canadian Agency for Drugs Technologies in Health. Telehealth services for the treatment of psychiatric issues: clinical effectiveness, safety, and guidelines . Ottawa: CADTH; 2015.Google Scholar
24. Sullivan, T, Hiller, J. Horizon scanning technology prioritising summary. The use of SMS text messaging to improve outpatient attendance. May 2007. Adelaide; Australian Department of Health and Ageing; 2007.Google Scholar
25. Pwc. Socio-economic impact of mHealth. An assessment report for the European Union. 2013. (accessed January 19, 2018).Google Scholar
26. Willem, A, Buijink, G, Visser, BJ, Marshall, L, Willem, A, Buijink, G. Medical apps for smartphones: lack of evidence undermines quality and safety. Evid Based Med. 2013;18:9092.Google Scholar
27. Sorenson, C, Drummond, M, Kanavos, P. Ensuring value for money in health care: the role of health technology assessment in the European Union-European Observatory on Health Systems and Policies. 2008.Google Scholar
28. INAHTA. A checklist for health technology assessment reports. Edmonton, Alberta, Canada: INAHTA; 2007:17.Google Scholar
29. Presidenza del Consiglio dei Ministri. “Mobile-health” e applicazioni per la salute: aspetti bioetici. Rome: Presidenza del Consiglio dei Ministri; 2015:1–24.Google Scholar
30. European Network for Health Technology Assessment. EUnetHTA Joint Action 2, Work Package 8. HTA Core Model ® version 3.0. 2016 [refd 2017 Jun 13]. (accessed January 18, 2018).Google Scholar
31. Callens, S. The EU legal framework on e-health. In Mossialos E, Permanand G, Baeten R, Hervey T (Eds.). Health Systems Governance in Europe: The Role of European Union Law and Policy (Health Economics, Policy and Management, pp. 561–588). Cambridge: Cambridge University Press; 2010.Google Scholar
32. U.S. Department of Health Human Services Food and Drug Administration. Mobile medical applications: guidance for industry and Food and Drug Administration Staff. Silver Spring, MD: USFDA; 2015:144.Google Scholar
33. Hailey, D, Ohinmaa, A, Roine, R. Study quality and evidence of benefit in recent assessments of telemedicine. J Telemed Telecare. 2004;10:318324.CrossRefGoogle ScholarPubMed
34. Williams, T, May, C, Mair, F, Mort, M, Gask, L. Normative models of health technology assessment and the social production of evidence about telehealth care. Health Policy. 2003;64:3954.CrossRefGoogle ScholarPubMed
35. Kidholm, K, Ekeland, AG, Jensen, LK, et al. A model for assessment of telemedicine applications: mast. Int J Technol Assess Health Care. 2012;28:4451.CrossRefGoogle ScholarPubMed
36. Agarwal, S, Lefevre, AE, Lee, J, et al. Guidelines for reporting of health interventions using mobile phones: mobile health (mHealth) evidence reporting and assessment (mERA) checklist. BMJ. 2016;352; i1174.CrossRefGoogle ScholarPubMed
37. Husereau, D, Henshall, C, Sampietro-Colom, L, Thomas, S. Changing health technology assessment paradigms? Int J Technol Assess Health Care. 2016;32:191199.CrossRefGoogle ScholarPubMed
38. Centre for Reviews Dissemination (CRD). Evidence briefing on teleconsultation . Heslington, York: CRD; 2012.Google Scholar
39. Centre for ReviewsDissemination (CRD). Telehealth for patients with long term conditions . Heslington, York: CRD; 2013.Google Scholar
40. Chen, S, Wright, M-D, Nkansah, E. Tele-ophthalmology for detecting eye diseases: clinical and cost-effectiveness. Health Technology Inquiry Service - CADTH. 2008. (accessed January 18, 2018).Google Scholar
41. Deshpande, A, Khoja, S, McKibbon, A, Rizo, C, Jadad, AR. Telehealth for acute stroke management (telestroke): systematic review of analytic studies and environmental scan of relevant initiatives [Technology report number 99]. 2008. (accessed January 18, 2018).Google Scholar
42. Franek, J. Home telehealth for patients with chronic obstructive pulmonary disease (COPD): an evidence-based analysis. Ont Health Technol Assess Ser. 2012;12:158.Google ScholarPubMed
43. Giansanti, D, Castrichella, L, Giovagnoli, MR. The design of a health technology assessment system in telepathology. Telemed J E Health. 2008;14:570575.CrossRefGoogle ScholarPubMed
44. Giansanti, D, Morelli, S, Maccioni, G, et al. A web-based Health technology assessment in tele-echocardiography: the experience within an Italian project. Ann Ist Super Sanita. 2009;45:392397.Google ScholarPubMed
45. Giansanti, D, Pochini, M, Giovagnoli, MR. Integration of tablet technologies in the e-laboratory of cytology: a health technology assessment. Telemed J E Health. 2014;20:909915.CrossRefGoogle ScholarPubMed
46. Hailey, D, Roine, R, Ohinmaa, A. Assessment of telemedicine applications - an update [Internet]. Health Technology Assessment Database. 2001. (accessed January 18, 2018).Google Scholar
47. Hailey, D, Bulger, T, Stayberg, S, Urness, D. Application of an assessment framework to an evolving telemental health program. 2002. (accessed January 18, 2018).Google Scholar
48. Hailey, D, Roine, R, Ohinmaa, A. IHE report evidence of benefits from telemental health: a systematic review. 2007. (accessed January 18, 2018).Google Scholar
49. Hailey, D, Paquin, M-J, Maciejewski, O, Harris, L, Casebeer, A, Fick, G, et al. The use and benefits of teleoncology. 2007; (January): 1–176. (accessed January 18, 2018).Google Scholar
50. Hailey, D, Roine, R, Ohinmaa, A. The effectiveness of telemental health applications: a review (Structured abstract). Can Psychiatry. 2008;53:769778.CrossRefGoogle Scholar
51. Johansson, Wild C. Telemedicine in acute stroke management: systematic review. Int J Technol Assess Health Care. 2009;26:149155.CrossRefGoogle Scholar
52. Mitchell, MD, Williams, K, Brennan, PJ, Umscheid, CA. Integrating local data into hospital-based healthcare technology assessment: two case studies. Int J Technol Assess Health Care. 2010;26:294300.CrossRefGoogle ScholarPubMed
53. Mundy, L, Hiller, J, Merlin, T. Telemedicine for the implementation of stroke therapy for patients in rural and remote areas. Horizon scanning prioritising summary 2006; 12(6). Canberra: Commonwealth of Australia.;jsessionid=A91C65358BBB8898C0A57ED6BA7B342D?p=262&tag=ruralareas&tab=424&contentType=167&section=10540 (accessed January 18, 2018).Google Scholar
54. Murphy, G, Prichett-Pejic, W, Severn, M. Non-emergency Telecardiology Consultation Services: rapid review of clinical and cost outcomes. Health Technology Assessment Database. 2010. (accessed January 18, 2018).Google Scholar
55. National Horizon Scanning Center. New and emerging technology briefing national remote monitoring of implantable cardiac devices. Patient Manag. 2006.Google Scholar
56. Ndegwa, S, Prichett-Pejic, W, McGill, S. Teledermatology services: rapid review of diagnostic, clinical management, and economic outcomes. 2010. (accessed January 18, 2018).Google Scholar
57. Noorani, HZ, Picot, J. Assessment of videoconferencing in telehealth in Canada. 2001. (accessed January 18, 2018).Google Scholar
58. Ohinmaa, A, Chatterley, P, Nguyen, T, Jacobs, P. Telehealth in substance abuse and addiction: review of the literature on smoking, alcohol, drug abuse and gambling. 2010. (accessed January 18, 2018).Google Scholar
59. Parrella, A, Mundy, L, Hiller, J, Merlin, T. Nurse-led telephone program to monitor heart failure patients after hospital discharge. 2006.$File/Nurse-led%20telephone%20program%20to%20monitor%20heart%20failure%20patients%20after%20hospital%20discharge.pdf (accessed January 18, 2018).Google Scholar
60. Stacey, D, Noorani, HZ, Fisher, A, Robinson, D, Joyce, J, Pong, RW. Telephone triage services: systematic review and a survey of Canadian call centre programs. 2003. (accessed January 18, 2018).Google Scholar
61. Sullivan, T, Mundy, L, Janet, H, Merlin, T. Remote monitoring systems for patients implanted with cardiac devices (cardioverter defibrillators and pacemakers). 2006 .$File/Remote%20monitoring%20systmes%20for%20patients%20implanted%20with%20cardiac%20devices%20March2006.pdf (accessed January 18, 2018).Google ScholarPubMed
62. Vinck, I, Stroobandt, S, Gerkens, S, De Laet, C. Remote monitoring for patients with implanted defibrillators. Technology evaluation and broader regulatory framework. 2010. (accessed January 18, 2018).Google Scholar
Supplementary material: File

Vukovic et al. supplementary material 1

Supplementary Table

Download Vukovic et al. supplementary material 1(File)
File 28 KB
Cited by

Send article to Kindle

To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Available formats

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Available formats

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *