Skip to main content Accessibility help

Economic Evaluation of Diagnostic Technology: Methodological Challenges and Viable Solutions

  • Franco Sassi (a1), Martin McKee (a2) and Jennifer A. Roberts (a2)


The principles of economic evaluation are increasingly accepted by clinicians and policy makers as evidence from a significant number of studies becomes available to support their decisions. However, methods of assessment still need to be improved. This paper reports a comprehensive review of methodological challenges in the economic evaluation of diagnostic technology, where such challenges are more evident. This review formed the basis for a prioritized research agenda, with four main areas: modeling techniques for dealing with complexity; measures of the opportunity cost of shared resources; techniques for eliciting decision makers' utility functions for diagnostic tests; and ways of assessing the robustness of decisions. A number of methodoligical solutions are proposed, aimed at capturing elements and relationships that are usually neglected and fully recognizing the presence of an inductive cognitive component in decision-making processes.



Hide All
1.Adams, M. E., McCall, N. T., Gray, D. T., et al. Economic analysis in randomized control trials. Medical Care, 1992, 30, 231–43.
2.Arthur, W. B.Inductive reasoning and bounded rationality. American Economic Review, 1994, 84, 406–11.
3.Asch, D. A., Patton, J. P., & Hershey, J. C.Knowing for the sake of knowing: The value of prognostic information. Medical Decision Making, 1990, 10, 4757.
4.Banta, H. D., & McNeil, B. J.Evaluation of the CAT scanner and other diagnostic technologies. Health Care Management Review, 1978, 4, 719.
5.Barger-Lux, M. J., & Heaney, R. P.For better and worse: The technological imperative in health care. Social Science and Medicine, 1986, 22, 1313–20.
6.Begg, C. B.Biases in the assessment of diagnostic tests. Statistics in Medicine, 1987, 6, 411–23.
7.Bekker, H., Denniss, G., Modell, M., et al. The impact of population based screening for carriers of cystic fibrosis. Journal of Medical Genetics, 1994, 31, 364–68.
8.Bell, D. E.Regret in decision making under uncertainty. Operational Research, 1982, 30, 961–81.
9.Bell, D. E.Disappointment in decision making under uncertainty. Operational Research, 1985, 33, 127.
10.Berwick, D. M., Fineberg, H. V., & Weinstein, M. C.When doctors meet numbers. American Journal of Medicine, 1981, 71, 991–98.
11.Berwick, D. M., & Weinstein, M. C.What do patients value? Willingness to pay for ultrasound in normal pregnancy. Medical Care, 1985, 23, 881–93.
12.Best, G., Parston, G., & Rosenhead, J.Robustness in practice: The regional planning of health services. Journal of the Operational Research Society, 1986, 37, 463–78.
13.Casscells, W., Schoenberger, A., & Graboys, T. B.Interpretation by physicians of clinical laboratory results. New England Journal of Medicine, 1978, 299, 9991001.
14.Christensen-Szalanski, J. J. J. Improving the practical utility of judgment research. In Bremer, B., Jungermann, H., Lourens, P. & Sevon, G. (eds.), New directions in research on decision making New York: North Holland, 1986.
15.Coughlin, S. S., Trock, B., Criqui, M. H., et al. The logistic modeling of sensitivity, specificity, and predictive value of a diagnostic test. Journal of Clinical Epidemiology, 1992, 45, 17.
16.Coyle, D., Increasing the impact of economic evaluations on health care decision-making, CHE Discussion Paper 108, York: University of York, 1993.
17.Crystal, R. A., & Brewster, A. W.Cost benefit and cost effectiveness analyses in the health field: An introduction. Inquiry, 1966, 3, 313.
18.Culyer, A. J.Health service efficiency: Appraising the appraisers. Discussion Paper 10, York: University of York, Centre for Health Economics, 1985.
19.Detmer, D. E., Fryback, D. G., & Gassner, K.Heuristics and biases in medical decisionmaking. Journal of Medical Education, 1978, 53, 682–83.
20.Detsky, A. S., Redelmeier, D., & Abrams, H. B.What's wrong with decision analysis? Can the left brain influence the right? Journal of Chronic Diseases, 1987, 40, 831–38.
21.Diamond, G. A.Clinical epistemology of sensitivity and specificity. Journal of Clinical Epidemiology, 1992, 45, 913.
22.Drummond, M. F.Evaluation of health technology: Economic issues for health policy and policy issues for economic appraisal. Social Science and Medicine, 1994, 38, 1593–600.
23.Drummond, M. F., Ludbrook, A., Lowson, K., & Steele, A.Studies in economic appraisal in health care. Oxford: Oxford University Press, 1986.
24.Drummond, M. F., Stoddart, G., & Torrance, G.Methods for the economic evaluation of health care programmes. Oxford: Oxford University Press, 1987.
25.Dyer, J. S., & Sarin, R. K.Measurable multiattribute value functions. Operational Research, 1979, 27, 810–22.
26.Dyson, R. G.Viewpoint: Robustness and optimality as criteria for decisions. Operational Research Quarterly, 1973, 24, 317–18.
27.Eddy, D. M.Screening for cancer: Theory, analysis and design. Englewood Cliffs, NJ: Prentice-Hall 1980.
28.Eddy, D. M., & Clanton, C. H.The art of diagnosis: Solving the clinicopathological exercise. New England Journal of Medicine, 1982, 306, 1263–68.
29.Eisenberg, J. M., & Nicklin, D.Use of diagnostic services by physicians in community practice. Medical Care, 1981, 19, 297309.
30.Epstein, A. M., Begg, C. B., & McNeil, B. J.The effects of physicians' training and personality on test ordering for ambulatory patients. American Journal of Public Health 1984, 74, 1271–73.
31.Epstein, A. M., Begg, C. B., & McNeil, B. J.The use of ambulatory testing in prepaid and fee-for-service group practices: Relation to perceived profitability. New England Journal of Medicine, 1986, 314, 1089–94.
32.Escarce, J.Medicare patients' use of overpriced procedures before and after the Omnibus Budget Reconciliation Act of 1987. American Journal of Public Health, 1993, 833, 4955.
33.Escarce, J. J., Bloom, B. S., Hillman, A. L., et al. Diffusion of laparoscopic cholecystectomy among general surgeons in the United States. Medical Care, 1995, 33, 256–71.
34.Feinstein, A. R.Clinical judgment. Baltimore, MD: Williams and Wilkins, 1967.
35.Feldman, W.How serious are the adverse effects of screening? Journal of General Internal Medicine, 1990, 5(suppl.), 850–53.
36.Feussner, J. R., & Matchar, D. B.When and how to study the carotid arteries. Annals of Internal Medicine, 1988, 109, 805–18.
37.Fineberg, H. V., Bauman, R., & Sosman, M.Computerized cranial tomography: Effect on diagnostic and therapeutic plans. Journal of the American Medical Association, 1977, 238, 224–27.
38.Fraser-Hill, M. A., Renfrew, D. L., & Hilsenrath, P. E.Percutaneous needle biopsy of musculoskeletal lesions, 2: Cost-effectiveness. American Journal of Roentgenology, 1992, 158, 813–18.
40.Fuchs, V. R.The growing demand for medical care. New England Journal of Medicine, 1968, 279, 190–95.
41.Gabel, J. R., & Redisch, M. A.Alternative physician payment methods: Incentives, efficiency and national health insurance. Milbank Memorial Quarterly, 1979, 57, 3859.
42.Gafni, A., & Birch, S.Guidelines for the adoption of new technologies: A prescription for uncontrolled growth in expenditures and how to avoid the problem. Canadian Medical Association Journal, 1993, 148, 913–17.
43.Gerard, K.Cost-utility in practice: A policy maker's guide to the state of the art. Health Policy, 1992, 21, 249–80.
44.Glaser, W. A.Paying the doctor: Systems of remuneration and their effects. Baltimore, MD: John Hopkins University Press, 1970.
45.Godyn, J. J., Tomaszewski, J. E., & Zmijewski, C. M.Specificity of a reproducible qualitative urine examination is not a constant test characteristic. American Journal of Clinical Pathology, 1991, 95, 265–66.
46.Grover, S. A., Abrahamowicz, M., Joseph, L., et al. The benefits of treating hyperlipidemia to prevent coronary heart disease. Journal of the American Medical Association, 1992, 267, 816–22.
47.Gunning-Schepers, L.The health benefits of prevention. Rotterdam: Institut Maatschappelijke Gezondheidszorg, Erasmus Universiteit, 1988.
48.Gupta, S. K., & Rosenhead, J.Robustness in sequential investment decisions. Management Sciences, 1968, 15, 1829.
49.Guyatt, G., & Drummond, M. F.Guidelines for the clinical and economic assessment of health technologies: The case of magnetic resonance. International Journal of Technology Assessment in Health Care, 1985, 1, 551–66.
50.Guyatt, G. H., Tugwell, P. X., Feeny, D. H., et al. A framework for clinical evaluation of diagnostic technologies. Canadian Medical Association Journal, 1986, 134, 587–94.
51.Guyatt, G. H., Tugwell, P. X., Feeny, D. H., et al. The role of before-after studies of therapeutic impact in the evaluation of diagnostic technologies. Journal of Chronic Diseases, 1986, 39, 295304.
52.Hanley, J. A., & McNeil, B. J.The meaning and use of the area under a receiver operating characteristics (ROC) curve. Radiology, 1982, 143, 2936.
53.Hasselblad, V., & Hedges, L. V. Meta-analysis of screening or diagnostic tests. Psychological Bulletin, in press.
54.Haynes, R. B., Sackett, D. L., Taylor, D. W., et al. Increased absenteeism from work after detection and labelling of hypertension patients. New England Journal of Medicine, 1978, 229, 741–44.
55.Heise, D. R.Causal analysis. New York: Wiley, 1975.
56.Hickson, G. B., Altemeier, W. A., & Perrin, J. M.Physician reimbursement by salary or fee-for-service: Effect on physician practice behavior in a randomized prospective study. Pediatrics, 1987, 80, 344–50.
57.Hlatky, M. A., Pryor, D. B., Harrel, F. E., et al. Factors affecting sensitivity and specificity of exercise electrocardiography. American Journal of Medicine, 1984, 77, 6471.
58.Holtgrave, D. R., Lawler, F., & Spann, S. J.Physicians' risk attitudes, laboratory usage, and referral decisions: The case of an academic family practice center. Medical Decision Making, 1991, 11, 125–30.
59.Hurley, J., & Labelle, R.Relative fees and the utilization of physicians' services in Canada. Working Paper 94–6, Hamilton: McMaster University, Centre for Health Economics and Policy Analysis, 1994.
60.Hurley, J., Labelle, R., & Rice, T. The relationship between physician fees and utilization of medical services in Ontario. In Scheffler, R., & Rossiter, L. (eds.), Advances in health economics and health services research. Greenwich, CT:JAI Press, 1990.
61.Hutton, J. Economic appraisal of health technology: A review of the current state of the art in EC countries. In Drummond, M. F. (ed.), Economic appraisal of health technology in the European Community. Oxford: Oxford Medical Publications, 1987.
62.Jaime Caro, J., Trindade, E., & McGregor, M.The cost-effectiveness of replacing highosmolality with low-osmolality contrast media. American Journal of Roentgenology, 1992, 159, 869–74.
63.Johanson, J. F., & Sonnenberg, A.Efficient management of diarrhea in the acquired immunodeficiency syndrome (AIDS): A medical decision analysis. Annals of Internal Medicine, 1990, 112, 942–48.
64.Johnson, H. A.Predictive value and informational value of diagnostic test results. Annals of Clinical and Laboratory Sciences, 1993, 23, 159–64.
65.Kahneman, D., & Tversky, A.Prospect theory: An analysis of decision under risk. Econometrica, 1979, 47, 263–91.
66.Kassirer, J. P.Diagnostic reasoning. Annals of Internal Medicine, 1989, 110, 893900.
67.Kassirer, J. P.Our stubborn quest for diagnostic certainty: A cause of excessive testing. New England Journal of Medicine, 1989, 320, 1489–91.
68.Kassirer, J. P., & Gorry, G. A.Clinical problem solving: A behavioral analysis. Annals of Internal Medicine, 1978, 89, 245–55.
69.Kassirer, J. P., & Kopelman, R. I.Cognitive errors in diagnosis: Instantiation, classification, and consequences. American Journal of Medicine, 1989, 86, 433–41.
70.Keeney, R. L., & Raiffa, H.Decisions with multiple objectives. New York: John Wiley, 1976.
71.Klarman, H. E.Present status of cost-benefit analysis in the health field. American Journal of Public Health, 1967, 57, 1948–53.
72.Lashner, B. A., & Silverstein, M. D.Evaluation and therapy of the patient with fecal occult blood loss: A decision analysis. American Journal of Gastroenterology, 1990, 85, 1088–95.
73.Laupacis, A., Feeny, D., Detsky, A. S., & Tugwell, P. X.How attractive does a new technology have to be to warrant adoption and utilization? Tentative guidelines for using clinical and economic evaluations. Canadian Medical Association Journal, 1992, 146, 473–81.
74.Laupacis, A., Feeny, D., Detsky, A. S., & Tugwell, P. X.Tentative guidelines for using clinical and economic evaluations revisited. Canadian Medical Association Journal, 1993, 148, 927–29.
75.Lave, J. R., Pashos, C. L., Anderson, G. F., et al. Costing medical care: Using Medicare administrative data. Medical Care, 1994, 32 (suppl.7), js77–js89.
76.Lee, J. T., & Sanchez, L. A.Interpretation of ‘cost-effective’ and soundness of economic evaluations in the pharmacy literature. American Journal of Hospital Pharmacy, 1991, 48, 2622–27.
77.Luce, B. R., & Elixhauser, A.Estimating costs in the economic evaluation of medical technologies. International Journal of Technology Assessment in Health Care, 1990, 6, 5775.
78.McNeil, B. J., Keller, E., & Adelstein, S. J.Primer on certain elements of medical decision making. New England Journal of Medicine, 1975, 293, 211–15.
79.Mooney, G.What else do we want from our health services? Social Science and Medicine, 1994, 39, 151–54.
80.Mooney, G., & Lange, M.Ante-natal screening: What constitutes ‘benefit’? Social Science and Medicine, 1993, 37, 873–78.
81.Moskowitz, A. J., Kuipers, B. J., & Kassirer, J. P.Dealing with uncertainty, risks and tradeoffs in clinical decisions. Annals of Internal Medicine, 1988, 108, 435–49.
82.Mulkay, M., Ashmore, M., & Pinch, T. Dependency and dispair: Health economics and the health-care system. In Hutton, J., Hutton, S., Pinch, T., & Shiell, A., (eds.), Dependency to enterprise. London: Routledge, 1991, 207–18.
83.Mushlin, A. I., Mooney, C., Grow, V., & Phelps, C. E.The value of diagnostic information to patients with suspected multiple sclerosis. Archives of Neurology, 1994, 5, 6772.
84.Mutgi, A., Williams, J. W., & Nettleman, M. D.Renal colic: Utility of the plain abdominal roentgenogram. Archives of Internal Medicine, 1991, 151, 1589–92.
85.Naylor, C. D., Williams, J. I., Basinski, A., & Goel, V.Technology assessment and cost-effectiveness: Misguided guidelines? Canadian Medical Association Journal, 1993, 148, 921–24.
86.Nightingale, S. D.Risk preference and laboratory use. Medical Decision Making, 1987, 7, 168–72.
87.Nightingale, S. D.Risk preference and admitting rates of emergency room physicians. Medical Care, 1988, 26, 8487.
88.Nightingale, S. D., Yarnold, P. R., & Greenberg, M. S.Sympathy, empathy, and physician resource utilization. Journal of General Internal Medicine, 1991, 6, 420–23.
89.O'Connor, P., Detsky, A. S., Tansey, C., Kucharczyk, W., & The Rochester-Toronto MRI Study Group. Effects of diagnostic testing for multiple sclerosis on patient health perceptions. Archives of Neurology, 1994, 51, 4651.
90.Oliver, R. M., & Smith, J. Q. (eds.). Influence diagrams, belief nets and decision analysis. Chichester: John Wiley, 1989.
91.Ornstein, S. M., Markert, G. P., Johnson, A. H., et al. The effect of physician personality on laboratory test ordering for hypertensive patients. Medical Care, 1988, 26, 536–43.
92.Pauker, S. G., & Kassirer, J. P.The threshold approach to clinical decision making. New England Journal of Medicine, 1980, 302, 1109–17.
93.Poses, R. M., Cebul, R. D., & Wigton, R. S.You can lead a horse to water: Improving physicians' knowledge of probabilities may not affect their decisions. Medical Decision Making, 1995, 15, 6575.
94.Ramoni, M., Stefanelli, M., Magnani, L., & Barosi, G.An epistemological framework for medical knowledge-based systems. IEEE Transactions on Systems, Man and Cybernetic 1992, 22, 1361–74.
95.Ranshoff, D. F., & Feinstein, A. R.Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. New England Journal of Medicine, 1978, 299, 926–30.
96.Rembold, C. M., & Watson, D.Posttest probability calculation by weights. Annals of Internal Medicine, 1988, 108, 115–20.
97.Ried, W.Willingness to pay for diagnostic services: A new approach to modelling patient benefits in health care. Health Economics, 1994, 3, 255–66.
98.Rosenhead, J.Operational research in health services planning. European Journal of Operational Research, 1978, 2, 7585.
99.Rosenhead, J. Robustness analysis: Keeping your options open. In Rosenhead, J. (ed.), Rational Analysis in a problematic world, Chichester: John Wiley, 1989.
100.Rosenhead, J., Elton, M., & Gupta, S. K.Robustness and optimality as criteria for strategic decisions. Operational Research Quarterly, 1972, 23, 413–31.
101.Rozanski, A., Diamond, G. A., Berman, D., et al. The declining specificity of exercise radionuclide ventriculography. New England Journal of Medicine, 1983, 309, 518–22.
102.Ruwe, P. A., Wright, J., Randall, R. L., et al. Can MR imaging effectively replace diagnostic arthroscopy? Radiology, 1992,183, 335–39.
103.Shickle, D., & Chadwick, R.The ethics of screening: Is ‘screeningitis’ an incurable disease? Journal of Medical Ethics, 1994, 20,1228.
104.Silk, A. D.Our stubborn quest for diagnostic certainty [letter: relpy to Kassirer]. New England Journal of Medicine, 1989, 321,1272.
105.Simel, D. L., Samsa, G. P., & Matchar, D. B.Likelihood ratios for continuous test results: Making the clinicians' job easier or harder? Journal of Clinical Epidemiology, 1992, 46, 8593.
107.Sonnenberg, F. A., Roberts, M. S., Tsevat, J., et al. Toward a peer review process for medical decision analysis models. Medical Care, 1994, 32(suppl.7), js52–js64.
108.Sox, H. C., Margulies, I., & Sox, C. H.Psychologically mediated effects of diagnostic tests. Annals of Internal Medicine, 1981, 95, 680–85.
109.Sutherland, S.Irrationality, Harmondsworth; Penguin, 1994.
110.Swan, I. R. C., & Gatehouse, S.Clinical and financial audit of diagnostic protocols for lesions of the cerebellopontine angle. British Medical Journal, 1991, 302, 701–04.
111.Swets, J. A.Measuring the accuracy of diagnostic systems. Science, 1988, 240, 1285–93.
112.Swets, J. A., Pickett, R. M., Whitehead, S. F., et al. Assessment of diagnostic technologies. Science, 1979, 205, 753–59.
113.Szczepura, A. K., Fletcher, J., & Fitz-Patrick, J. D.Cost effectiveness of magnetic resonance imaging in the neurosciences. British Medical Journal, 1991, 303, 1435–39.
114.Thompson, S. G.Why sources of heterogeneity in meta-analyses should be investigated. British Medical Journal, 1994, 309, 1351–55.
115.Torrance, G. W., Boyle, M. H., & Horwood, S. P.Applications of multi-attribute utility theory to measure social preferences for health states. Operational Research, 1982, 30, 1043–69.
116.Torrance, G. W., Zhang, Y., Feeny, D., et al. Multi-attribute preference functions for a comprehensive health status classification system. Working Paper 92–18. Hamilton: McMaster University, Centre for Health Economics and Policy Analysis, 1992.
117.Tsevat, J., Wong, J. B., Pauker, S. G., & Steinberg, M. H.Neonatal screening for sickle cell disease: A cost-effectiveness analysis. Journal of Pediatrics, 1991, 118, 546–54.
118.Tversky, A., & Kahneman, D.Judgment under uncertainty: Heuristics and biases. Science, 1974, 185, 1124–31.
119.Udvarhelyi, I. S., Colditz, G. A., Rai, A., & Epstein, A. M.Cost-effectiveness and cost-benefit analyses in the medical literature: Are the methods being used correctly? Annals of Internal Medicine, 1992, 116, 238–44.
106.Somoza, E., & Mocsman, D.Comparing and optimizing diagnostic tests: An information theoretical approach. Medical Decision Making, 1992, 12, 179–88.
120.Wack, P.Scenarios: Uncharted waters ahead. Harvard Business Review, 1985, 63, 7389.
121.Wagner, J. L.The feasibility of economic evaluation of diagnostic procedures. Social Science and Medicine, 1983, 17, 861–69.
122.Wagner, J. L.Economic evaluation in neuroradiology. Journal of Neuroradiology, 1983, 10, 96100.
123.Walt, G.How far does research influence policy? European Journal of Public Health, 1994, 4, 233–35.
124.Weinstein, M. C.Methodologic considerations in planning clinical trials of cost-effectiveness of magnetic resonance imaging. International Journal of Technology Assessment in Health Care, 1985, 1, 567–81.
125.Weinstein, M. C.Challenges for cost-effectiveness research. Medical Decision Making, 1986, 6,194–98.
126.Weinstein, M. C., Coxson, P. G., Williams, L. W., et al. Forecasting coronary heart disease incidence, mortality, and cost: The Coronary Heart Disease Policy Model. American Journal of Public Health, 1987, 77, 1417–26.
127.Weinstein, M. C., Fineberg, H. V., Elstein, A. S., et al. Clinical decision analysis Philadelphia: Saunders, 1980.
128.Weinstein, M. C., & Stason, W. B.Cost-effectiveness of interventions to prevent or treat coronary heart disease. Annual Review of Public Health, 1985, 6, 4163.
129.White, D. J.Viewpoint: Robustness and optimality as criteria for decisions. Operational Research Quarterly, 1973, 24, 311–13.
130.Wiggins, S., Whyte, P., Huggins, M., et al. , for the Canadian Collaborative Study Group of Predictive Testing. The psychological consequences of predictive testing for Huntington's disease. New England Journal of Medicine, 1992, 327, 1401–05.
131.Yarnold, P. R., Nightingale, S. D., Curry, R. H., & Martin, G. J.Psychological androgyny and preference in loss-framed gambles of medical students: Possible implications for resource utilization. Medical Decision Making, 1991, 11, 176–79.
132.Zaat, J. O. M., & van Eijk, T. M.General practitioners' uncertainty, risk preference, and use of laboratory tests. Medical Care, 1992, 30, 846–54.
133.Zylak, C. J., & Gafni, A.A methodologic overview of the evaluation of costs and benefits in diagnostic radiology. Investigative Radiology, 1992, 27, 483–88.

Economic Evaluation of Diagnostic Technology: Methodological Challenges and Viable Solutions

  • Franco Sassi (a1), Martin McKee (a2) and Jennifer A. Roberts (a2)


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed