Skip to main content Accessibility help
×
Home

INTERACTION BETWEEN OBJECTIVE PERFORMANCE MEASURES AND SUBJECTIVE USER PERCEPTIONS IN THE EVALUATION OF MEDICAL DEVICES: A CASE STUDY

  • Matthew D. Haydock (a1), Anubhav Mittal (a2), Carissa F. Wilkes (a3), David H. Lim (a2), Elizabeth Broadbent (a3) and John A. Windsor (a2) (a4)...
  • Please note an addendum has been issued for this article.

Abstract

Objectives: Medical technology is a large and expanding industry. Introducing new medical devices is important but several challenges exist in implementing the optimal method of evaluation. Both objective and subjective measures can be used for evaluation. The former is the mainstay of evaluation, yet subjective assessment is often the basis for the introduction of new medical technology. The aim of this study was to determine the interaction and concordance between objective and subjective assessment of new medical technology.

Methods: This study used both objective performance measures and subjective user perceptions in the evaluation of a new medical device designed to improve the accuracy of gravity-assisted delivery of intravenous fluids, compared with the current, widely used “roller-clamp” device. The concordance of objective and subjective assessments was evaluated using comparative analysis.

Results: Objective assessment of the accuracy of intravenous fluid delivery revealed no difference between the two devices (p = .636). Subjective assessment revealed that the new device was perceived to be significantly more accurate (p = .001). This lack of concordance can be partially explained by both device and demand characteristics.

Conclusions: This case study reveals a significant discordance between the objective and subjective assessments. It provides some explanation for why new medical devices are adopted without objective evidence of benefit. This phenomenon has been termed “persuasive design” and its influence should be controlled for in the evaluation, purchase and introduction of new medical devices. This should help reduce the risk and associated cost of premature introduction.

Copyright

Footnotes

Hide All
*

An addendum notice for this article has been published (DOI: 10.1017/S0266462319000564) and appears at the end of the text as well.

The authors thank the registered and student nurses who kindly volunteered their time to participate in this study, Sir Ray Avery who provided access to materials for testing, the University of Auckland Department of Mechanical Engineering for their technical support and access to their laboratory and Auckland City Hospital for the use of procedure rooms for carrying out the clinical study. Grant support: None. This study is not based on any previous communication

Footnotes

References

Hide All
1. Edmondson, AC, Bohmer, RM, Pisano, GP. Disrupted routines: Team learning and new technology implementation in hospitals. Adm Sci Q. 2001;46:685716.
2. Jennett, B. Health technology assessment. BMJ. 1992;305:6768.
3. McPherson, K. International differences in medical care practices. Health Care Financ Rev. 1989;(Spec No):9–20.
4. Antman, EM, Lau, J, Kupelnick, B, Mosteller, F, Chalmers, TC. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA. 1992;268:240248.
5. Bernard, A, Vaneau, M, Fournel, I, et al. Methodological choices for the clinical development of medical devices. Med Devices (Auckl). 2014;7:325334.
6. Maisel, WH. Medical device regulation: An introduction for the practicing physician. Ann Intern Med. 2004;140:296302.
7. de Veer, AJE, Fleuren, MAH, Bekkema, N, Francke, AL. Successful implementation of new technologies in nursing care: A questionnaire survey of nurse-users. BMC Med Inform Decis Making. 2011;11:67.
8. Manu, FA, Sriram, V. Innovation, marketing strategy, environment, and performance. J Bus Res. 1996;35:7991.
9. Ginsburg, G. Human factors engineering: A tool for medical device evaluation in hospital procurement decision-making. J Biomed Inform. 2005;38:213219.
10. Shah, SGS, Robinson, I. Benefits of and barriers to involving users in medical device technology development and evaluation. Int J Technol Assess Health Care. 2007;23:131137.
11. Hróbjartsson, A, Gøtzsche, PC. Is the Placebo Powerless? N Engl J Med. 2001;344:15941602.
12. Hróbjartsson, A, Gøtzsche, PC. Is the placebo powerless? Update of a systematic review with 52 new randomized trials comparing placebo with no treatment. J Intern Med. 2004;256:91100.
13. The Center for Universal Design, North Carolina State University. The Principles of Universal Design. 1997;2013. http://www.ncsu.edu/project/design-projects/udi/center-for-universal-design/the-principles-of-universal-design/ (accessed February 18, 2013).
14. Bissett, IP, Brandt, TP, Windsor, JA. Survey of intravenous fluid therapies and accuracy of gravity-fed infusions in a teaching hospital. Samoa Med J. 2010;2:2528.
15. Gutierrez Alejandro, A, Calvo Buey, JA, Marcos Camina, RM. [Study for the decrease of errors in the records of hydric balances of critical patients admitted to an intensive care unit]. Enferm Intensiva. 2005;16:100109.
16. Han, PY, Coombes, ID, Green, B. Factors predictive of intravenous fluid administration errors in Australian surgical care wards. Qual Saf Health Care. 2005;14:179184.
17. Rooker, JC, Gorard, DA. Errors of intravenous fluid infusion rates in medical inpatients. Clin Med. 2007;7:482485.
18. Faasse, K, Cundy, T, Gamble, G, Petrie, KJ. The effect of an apparent change to a branded or generic medication on drug effectiveness and side effects. Psychosom Med. 2013;75:9096.
19. Finniss, DG, Kaptchuk, TJ, Miller, F, Benedetti, F. Biological, clinical, and ethical advances of placebo effects. Lancet. 2010;375:686695.
20. Moerman, DE. Meaningful placebos–Controlling the uncontrollable. N Engl J Med. 2011;365:171172.
21. Moerman, DE, Jonas, WB. Deconstructing the placebo effect and finding the meaning response. Ann Intern Med. 2002;136:471476.
22. de Craen, AJM, Roos, PJ, de Vries, AL, Kleijnen, J. Effect of colour of drugs: Systematic review of perceived effect of drugs and of their effectiveness. BMJ. 1996;313:16241626.
23. Branthwaite, A, Cooper, P. Analgesic effects of branding in treatment of headaches. Br Med J (Clin Res Ed). 1981;282:15761578.
24. Desharnais, R, Jobin, J, Côté, C, Lévesque, L, Godin, G. Aerobic exercise and the placebo effect: A controlled study. Psychosom Med. 1993;55:149154.
25. Orne, MT. Demand characteristics and the concept of quasi-controls. Artifacts in behavioral research: Robert Rosenthal and Ralph L Rosnow's Classic Books. 2009:110.
26. Bivins, BA, Rapp, RP, Powers, P, Butler, JL, Haack, D. Electronic flow control and roller clamp control in intravenous therapy: A clinical comparison. Arch Surg. 1980;115:7072.
27. Crass, RE, Vance, JR. In vivo accuracy of gravity-flow i.v. infusion systems. Am J Hosp Pharm. 1985;42:328331.
28. Flack, FC, Whyte, TD. Behaviour of standard gravity-fed administration sets used for intravenous infusion. Br Med J. 1974;3:439443.
29. Rithalia, SV, Rozkovec, A. Evaluation of a simple device for regulating intravenous infusions. Intensive Care Med. 1979;5:4143.
30. Husch, M, Sullivan, C, Rooney, D, et al. Insights from the sharp end of intravenous medication errors: Implications for infusion pump technology. Qual Saf Health Care. 2005;14:8086.
31. Himal, HS. Minimally invasive (laparoscopic) surgery. Surg Endosc. 2002;16:16471652.
32. Patel, HRH, Linares, A, Joseph, JV. Robotic and laparoscopic surgery: Cost and training. Surg Oncol. 2009;18:242246.
33. Yarbrough, AK, Smith, TB. Technology acceptance among physicians: A New take on TAM. Med Care Res Rev. 2007;64:650672.
34. Fogg, BJ. A behavior model for persuasive design. New York, NY: ACM; 2009 doi:10.1145/1541948.1541999
35. Redström, J. Persuasive design: Fringes and foundations. In: IJsselsteijn, WA, de Kort, YAW, Midden, C, Eggen, B, van den Hoven, E, eds. Persuasive technology. Berlin: Springer; 2006:112122.

Keywords

Type Description Title
UNKNOWN
Supplementary materials

Haydock supplementary material
Supplementary Figure 1a-b

 Unknown (17 KB)
17 KB
UNKNOWN
Supplementary materials

Haydock supplementary material
Supplementary Figure 2

 Unknown (204 KB)
204 KB
UNKNOWN
Supplementary materials

Haydock supplementary material
Supplementary Figure 3

 Unknown (31 KB)
31 KB
WORD
Supplementary materials

Haydock Supplementary Material
Table 1

 Word (74 KB)
74 KB
WORD
Supplementary materials

Haydock Supplementary Material
Table 2

 Word (116 KB)
116 KB

INTERACTION BETWEEN OBJECTIVE PERFORMANCE MEASURES AND SUBJECTIVE USER PERCEPTIONS IN THE EVALUATION OF MEDICAL DEVICES: A CASE STUDY

  • Matthew D. Haydock (a1), Anubhav Mittal (a2), Carissa F. Wilkes (a3), David H. Lim (a2), Elizabeth Broadbent (a3) and John A. Windsor (a2) (a4)...
  • Please note an addendum has been issued for this article.

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed

An addendum has published for this article: