Hostname: page-component-8448b6f56d-sxzjt Total loading time: 0 Render date: 2024-04-20T00:20:31.264Z Has data issue: false hasContentIssue false
  • English
  • Français

OPTIMIZING USABILITY OF AN ECONOMIC DECISION SUPPORT TOOL: PROTOTYPE OF THE EQUIPT TOOL

Published online by Cambridge University Press:  19 February 2018

Kei Long Cheung ,
Mickaël Hiligsmann ,
Maximilian Präger ,
Teresa Jones ,
Judit Józwiak-Hagymásy ,
Celia Muñoz ,
Adam Lester-George ,
Subhash Pokhrel Open the ORCID record for Subhash Pokhrel [Opens in a new window] ,
Ángel López-Nicolás  and
Marta Trapero-Bertran
...Show all authors
Kei Long Cheung
Affiliation:
Caphri School of Public Health and Primary Care, Health Services Research, Maastricht Universitykl.cheung@maastrichtuniversity.nl
Mickaël Hiligsmann
Affiliation:
Caphri School of Public Health and Primary Care, Health Services Research, Maastricht University
Maximilian Präger
Affiliation:
Helmholtz Zentrum München (GmbH); German Research Center for Environmental Health (Institute of Health Economics and Health Care Management)
Teresa Jones
Affiliation:
Health Economics Research Group, Brunel University London
Judit Józwiak-Hagymásy
Affiliation:
Syreon Research Institute
Celia Muñoz
Affiliation:
Centre for Research in Economics and Health (CRES), University Pompeu Fabra
Adam Lester-George
Affiliation:
LeLan Solutions
Subhash Pokhrel
Affiliation:
Health Economics Research Group, Brunel University London
Ángel López-Nicolás
Affiliation:
Polytechnic University of Cartagena
Marta Trapero-Bertran
Affiliation:
Centre for Research in Economics and Health (CRES), University Pompeu Fabra
Silvia M.A.A. Evers
Affiliation:
Caphri School of Public Health and Primary Care, Health Services Research, Maastricht University; Trimbos Institute, Netherlands Institute of Mental Health and Addiction
Hein de Vries
Affiliation:
Caphri School of Public Health and Primary Care, Health Promotion, Maastricht University
Get access Rights & Permissions [Opens in a new window]

Abstract

Objectives: Economic decision-support tools can provide valuable information for tobacco control stakeholders, but their usability may impact the adoption of such tools. This study aims to illustrate a mixed-method usability evaluation of an economic decision-support tool for tobacco control, using the EQUIPT ROI tool prototype as a case study.

Methods: A cross-sectional mixed methods design was used, including a heuristic evaluation, a thinking aloud approach, and a questionnaire testing and exploring the usability of the Return of Investment tool.

Results: A total of sixty-six users evaluated the tool (thinking aloud) and completed the questionnaire. For the heuristic evaluation, four experts evaluated the interface. In total twenty-one percent of the respondents perceived good usability. A total of 118 usability problems were identified, from which twenty-six problems were categorized as most severe, indicating high priority to fix them before implementation.

Conclusions: Combining user-based and expert-based evaluation methods is recommended as these were shown to identify unique usability problems. The evaluation provides input to optimize usability of a decision-support tool, and may serve as a vantage point for other developers to conduct usability evaluations to refine similar tools before wide-scale implementation. Such studies could reduce implementation gaps by optimizing usability, enhancing in turn the research impact of such interventions.

Keywords

Usability evaluationDecision-support toolThinking aloudHeuristic evaluation
Type
Assessments
Information
International Journal of Technology Assessment in Health Care , Volume 34 , Issue 1 , 2018 , pp. 68 - 77
Copyright
Copyright © Cambridge University Press 2018 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

1. Park, AL, McDaid, D, Weiser, P, et al. Examining the cost effectiveness of interventions to promote the physical health of people with mental health problems: A systematic review. BMC Public Health. 2013;13:787.Google Scholar
2. Nicod, E, Kanavos, P. Commonalities and differences in HTA outcomes: A comparative analysis of five countries and implications for coverage decisions. Health Policy. 2012;108:167–177.Google Scholar
3. Drummond, MF, Schwartz, JS, Jönsson, B, et al. Key principles for the improved conduct of health technology assessments for resource allocation decisions. Int J Technol Assess Health Care. 2008;24: 244–258.Google Scholar
4. van Velden ME, Severens JL, Novak, A. Economic evaluations of healthcare programmes and decision making. Pharmacoeconomics. 2005;23:1075–1082.Google Scholar
5. Oliver, K, Innvar, S, Lorenc, T, Woodman, J, Thomas, J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;14:2.CrossRefGoogle ScholarPubMed
6. Macintyre, S, Chalmers, I, Horton, R, Smith, R. Using evidence to inform health policy: Case study. BMJ. 2001;322:222.CrossRefGoogle ScholarPubMed
7. Drummond, M. Economic evaluation in health care: Is it really useful or are we just kidding ourselves? Aust Econ Rev. 2004;37: 3–11.Google Scholar
8. Garrido, MV. Health technology assessment and health policy-making in Europe: Current status, challenges and potential. Geneva: WHO Regional Office Europe; 2008.Google Scholar
9. NICE. 2012. The NICE Tobacco Return on Investment Tool. http://www.nice.org.uk/usingguidance/implementationtools/returnoninvesment/TobaccoROITool.jsp Google Scholar
10. Lim, SS, Vos, T, Flaxman, AD, et al. A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990–2010: A systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2013;380: 2224–2260.Google Scholar
11. WHO. WHO Framework Convention on Tobacco Control. 2003. http://www.who.int/fctc/en/ (accessed December 17, 2015).Google Scholar
12. European Commission. “Public Health: Tobacco Policy.” 2012. http://ec.europa.eu/health/tobacco/policy/index_en.htm (accessed December 17, 2015).Google Scholar
13. Pokhrel, S, Evers, S, Leidl, R, et al. EQUIPT: Protocol of a comparative effectiveness research study evaluating cross-context transferability of economic evidence on tobacco control. BMJ Open. 2014;4:e006945.CrossRefGoogle ScholarPubMed
14. van Gemert-Pijnen, JE, Nijland, N, van Limburg, M, et al. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. 2011;13:e111.Google Scholar
15. Eysenbach, G. The law of attrition. J Med Internet Res. 2005;7:e11.CrossRefGoogle ScholarPubMed
16. Horsky, J, Schiff, GD, Johnston, D, Mercincavage, L, Bell, D, Middleton, B. Interface design principles for usable decision support: A targeted review of best practices for clinical prescribing interventions. J Biomed Inform. 2012;45:1202–1216.Google Scholar
17. ISO. 9241-11:1998. Ergonomic requirements for office work with visual display terminals (VDT)s – Part 11: Guidance on usability. 1998. https://www.iso.org/standard/16883.html (accessed January 16, 2018).Google Scholar
18. Bevan, N. Usability is quality of use. Adv Hum Factors Ergon. 1995;20:349.Google Scholar
19. Wichansky, AM. Usability testing in 2000 and beyond. Ergonomics. 2000;43:998–1006.Google Scholar
20. Cho, V, Cheng, TE, Lai, WJ. The role of perceived user-interface design in continued usage intention of self-paced e-learning tools. Comput Educ. 2009;53:216–227.Google Scholar
21. Becker, S, Mottay, FE. A global perspective on web site usability. IEEE Software. 2001;18:54–61.Google Scholar
22. Ahern, DK. Challenges and opportunities of eHealth research. Am J Prev Med. 2007;32:S75-S82.Google Scholar
23. Nielsen, J. Usability inspection methods. New York: John Wiley and Sons; 1994;17:25–62.Google Scholar
24. Voncken-Brewster, V, Moser, A, Van Der Weijden T, Nagykaldi Z, De Vries H, Tange H. Usability evaluation of an online, tailored self-management intervention for chronic obstructive pulmonary disease patients incorporating behavior change techniques. JMIR Res Protoc. 2013;2:e3.Google Scholar
25. Cheung, KL, Evers, SM, Hiligsmann, M, et al. Understanding the stakeholders’ intention to use economic decision-support tools: A cross-sectional study with the Tobacco Return on Investment tool. Health Policy. 2016;120:46–54.CrossRefGoogle ScholarPubMed
26. Centre for Disease Control and Prevention. 2011. Introduction to program evaluation for public health programs: A self-study guide. http://www.cdc.gov/eval/guide/CDCEvalManual.pdf (accessed January 21, 2018).Google Scholar
27. Nielsen, J, Landauer, TK, editors. A mathematical model of the finding of usability problems. Proceedings of the INTERACT'93 and CHI'93 conference on Human factors in computing systems, Amsterdam, The Netherlands, 1993: ACM.Google Scholar
28. Jaspers, MW. A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence. Int J Med Inform. 2009;78:340–353.Google Scholar
29. Brooke, J. SUS-A quick and dirty usability scale. Usability Eval Ind. 1996;189:4–7.Google Scholar
30. Jaspers, MW, Steen, T, van Den Bos, C, Geenen, M. The think aloud method: A guide to user interface design. Int J Med Inform. 2004;73:781–795.Google Scholar
31. Nielsen, J, editor. Reliability of severity estimates for usability problems found by heuristic evaluation. Posters and short talks of the 1992 SIGCHI conference on Human factors in computing systems, Monterey, California, 1992: ACM.Google Scholar
32. Brooke, J. SUS: A retrospective. J Usability Stud. 2013;8:29–40.Google Scholar
33. Jeffries, R, Miller, JR, Wharton, C, Uyeda, K, editors. User interface evaluation in the real world: A comparison of four techniques. Proceedings of the SIGCHI conference on Human factors in computing systems, Montreal, Canada, 1991: ACM.CrossRefGoogle Scholar
34. Lai, T-Y, Larson, EL, Rockoff, ML, Bakken, S. User Acceptance of HIV TIDES—tailored interventions for management of depressive symptoms in persons living with HIV/AIDS. JAMA. 2008;15:217–226.Google Scholar
35. Bailey, RW, Allan, RW, Raiello, P, editors. Usability testing vs. heuristic evaluation: A head-to-head comparison. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 1992: SAGE Publications.Google Scholar
36. Rogers, EM. Diffusion of innovations. New York: Simon and Schuster; 2010.Google Scholar

Cheung et al. supplementary material 1

Supplementary Video

Download Cheung et al. supplementary material 1(Video)
Video 144.2 MB
Supplementary material: File

Cheung et al. supplementary material 2

Cheung et al. supplementary material

Download Cheung et al. supplementary material 2(File)
File 24.3 KB
Supplementary material: File

Cheung et al. supplementary material 3

Cheung et al. supplementary material

Download Cheung et al. supplementary material 3(File)
File 46 KB
Supplementary material: File

Cheung et al. supplementary material 4

Cheung et al. supplementary material

Download Cheung et al. supplementary material 4(File)
File 30.7 KB