Skip to main content Accessibility help
  • Get access
    Check if you have access via personal or institutional login
  • Cited by 1
  • Print publication year: 2014
  • Online publication date: October 2014

12 - Heterogeneity and uncertainty


Medicine is a science of uncertainty and an art of probability.

Sir William Osler


Decision trees and Markov cohort models, as described and illustrated in the previous chapters, are essentially macrosimulation models. Such models simulate cohorts or groups of subjects. A number of limitations exist to the use of these models. Markov cohort models, for example, have ‘no memory’, implying that subjects in a particular state are a homogeneous group. Techniques to overcome these limitations, such as expanding the number of states, using tunnel states, or using alternative modeling techniques, were discussed in Chapter 10. These techniques can get very complex when dealing with extensive heterogeneity within a population. Microsimulation using Monte Carlo analysis provides another powerful technique to account for heterogeneity across subjects. Microsimulation with Monte Carlo analysis was introduced in Chapter 10 as an alternative method for evaluating a Markov model. In this chapter it will be discussed at greater length in the context of simulating heterogeneity.

In the previous chapters we represented uncertainty with probabilities. Implicitly the assumption was that, even though we were unsure of whether an event would take place, we could nevertheless predict or estimate the probability (or relative frequency) that it would occur. In essence we were using deterministic models. In reality, however, we are also uncertain of the degree of uncertainty. In other words, rather than dealing with a fixed probability we are actually dealing with a distribution of possible values of probabilities. Not only are we uncertain about the probabilities we use in our models, but we are also uncertain about the effectiveness outcomes and cost estimates included in the analysis. Thus, every parameter value we enter into our models is better represented as a probabilistic variable rather than a deterministic variable. If there is a single uncertain parameter, e.g., the relative risk reduction of an intervention, then the 95% confidence interval (CI) of this parameter is commonly used to indicate the uncertainty of the effect. Uncertainty in two or more components requires more complex methods, such as Monte Carlo probabilistic sensitivity analysis, which we will also discuss in this chapter.

Groot Koerkamp, B, Weinstein, MC, Stijnen, T, Heijenbrok-Kal, MH, Hunink, MG. Uncertainty and patient heterogeneity in medical decision models. Med Decis Making. 2010;30(2):194–205.
Groot Koerkamp, B, Stijnen, T, Weinstein, MC, Hunink, MG. The combined analysis of uncertainty and patient heterogeneity in medical decision models. Med Decis Making. 2011;31(4):650–61.
Claxton, K. The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. J Health Econ. 1999;18(3):341–64.
Claxton, K, Sculpher, M, Drummond, M. A rational framework for decision making by the National Institute For Health and Clinical Excellence (NICE). Lancet. 2002;360(9334):711–15.
Manning, WG, Fryback, DG, Weinstein, MC. Reflecting uncertainty in cost-effectiveness analysis. In: Gold MR, Siegel JE, Russell LB, Weinstein MC, eds. Cost-Effectiveness in Health and Medicine. New York; 1996. pp. 247–75.
Briggs, AH, Weinstein, MC, Fenwick, EA, et al. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Med Decis Making. 2012;32(5):722–32.
Basu, A, Meltzer, D. Value of information on preference heterogeneity and individualized care. Med Decis Making. 2007;27(2):112–27.
Hunink, MGM, Goldman, L, Tosteson, AN, et al. The recent decline in mortality from coronary heart disease, 1980–1990. The effect of secular trends in risk factors and treatment. JAMA. 1997;277(7):535–42.
Lazar, LD, Pletcher, MJ, Coxson, PG, Bibbins-Domingo, K, Goldman, L. Cost-effectiveness of statin therapy for primary prevention in a low-cost statin era. Circulation. 2011;124(2):146–53.
van Ravesteyn, NT, Heijnsdijk, EA, Draisma, G, de Koning, HJ. Prediction of higher mortality reduction for the UK Breast Screening Frequency Trial: a model-based approach on screening intervals. Brit J Cancer. 2011;105(7):1082–8.
Doubilet, P, Begg, CB, Weinstein, MC, Braun, P, McNeil, BJ. Probabilistic sensitivity analysis using Monte Carlo simulation. A practical approach. Med Decis Making. 1985;5(2):157–77.
Hoffman, FO, Hammonds, JS. Propagation of uncertainty in risk assessments: the need to distinguish between uncertainty due to lack of knowledge and uncertainty due to variability. Risk Anal. 1994;14(5):707–12.
DerSimonian, R, Laird, N. Meta-analysis in clinical trials. Controlled Clinical Trials. 1986;7:177–88.
Brown, ML, Fintor, L. Cost-effectiveness of breast cancer screening: preliminary results of a systematic review of the literature. Breast Cancer Res Treat. 1993;25(2):113–18.
Campbell, H, Briggs, A, Buxton, M, Kim, L, Thompson, S. The credibility of health economic models for health policy decision-making: the case of population screening for abdominal aortic aneurysm. J Health Serv Res Policy. 2007;12(1):11–17.
Mandelblatt, JS, Cronin, KA, Berry, DA, et al. Modeling the impact of population screening on breast cancer mortality in the United States. Breast. 2011;20 Suppl 3:S75–81.
Kuntz, KM, Weinstein, MC. Life expectancy biases in clinical decision modeling. Med Decis Making. 1995;15:158–69.
Law, AM, Kelton, WD. Simulation Modeling and Analysis. 3rd edn. Boston: McGraw-Hill Higher Education; 2000.
Efron, B, Tibshirani, RJ. An Introduction to the Bootstrap. New York: Chapman & Hall; 1993.
Duintjer Tebbens, RJ, Thompson, KM, Hunink, MG, et al. Uncertainty and sensitivity analyses of a dynamic economic evaluation model for vaccination programs. Med Decis Making. 2008;28(2):182–200.
Briggs, AH, Wonderling, DE, Mooney, CZ. Pulling cost-effectiveness analysis up by its bootstraps: a non-parametric approach to confidence interval estimation. Health Econ. 1997;6(4):327–40.
Mennemeyer, ST, Cyr, LP. A bootstrap approach to medical decision analysis. J Health Econ. 1997;16(6):741–7.
Critchfield, GC, Willard, KE, Connelly, DP. Probabilistic sensitivity analysis methods for general decision models. Comput Biomed Res. 1986;19(3):254–65.
Parmigiani, G, Samsa, GP, Ancukiewicz, M, et al. Assessing uncertainty in cost-effectiveness analyses: application to a complex decision model. Med Decis Making. 1997;17(4):390–401.
Cooper, H, Hedges, LV, Valentine, JC. The Handbook of Research Synthesis and Meta-Analysis. New York: Russel Sage Foundation; 2009.
Higgins, JPT, Green, S. Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Collaboration 2011.
Briggs, AH, Goeree, R, Blackhouse, G, O’Brien, BJ. Probabilistic analysis of cost-effectiveness models: choosing between treatment strategies for gastroesophageal reflux disease. Med Decis Making. 2002;22(4):290–308.
Briggs, AH, Ades, AE, Price, MJ. Probabilistic sensitivity analysis for decision trees with multiple branches: use of the Dirichlet distribution in a Bayesian framework. Med Decis Making. 2003;23(4):341–50.
van Hout, BA, Al, MJ, Gordon, GS, Rutten, FF. Costs, effects and C/E-ratios alongside a clinical trial. Health Econ. 1994;3(5):309–19.
Wakker, P, Klaassen, MP. Confidence intervals for cost/effectiveness ratios. Health Econ. 1995;4(5):373–81.
Chaudhary, MA, Stearns, SC. Estimating confidence intervals for cost-effectiveness ratios: an example from a randomized trial. Stat Med. 1996;15(13):1447–58.
Polsky, D, Glick, HA, Willke, R, Schulman, K. Confidence intervals for cost-effectiveness ratios: a comparison of four methods. Health Econ. 1997;6(3):243–52.
Heitjan, DF, Moskowitz, AJ, Whang, W. Bayesian estimation of cost-effectiveness ratios from clinical trials. Health Econ. 1999;8(3):191–201.
Stinnett, AA, Mullahy, J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making. 1998;18(2 Suppl):S68–80.
Fieller, EC. The distribution of the index in a normal bivariate population. Biometrika. 1932;24:428–40.
Muradin, GS, Hunink, MG. Cost and patency rate targets for the development of endovascular devises to treat femoropopliteal arterial disease. Radiology. 2001;221(1):137–45.
Visser, K, Kock, MC, Kuntz, KM, et al. Cost-effectiveness targets for multi-detector row CT angiography in the work-up of patients with intermittent claudication. Radiology. 2003;227(3):647–56.
Hunink, MGM, Kuntz, KM, Fleischmann, KE, Brady, TJ. Noninvasive imaging for the diagnosis of coronary artery disease: focusing the development of new diagnostic technology. Annals of Internal Medicine. 1999;131(9):673–80.
Phelps, CE, Mushlin, AI. Focusing technology assessment using medical decision theory. Med Decis Making. 1988;8:279–89.
Hunink, MGM, Bult, JR, de Vries, J, Weinstein, MC. Uncertainty in decision models analyzing cost-effectiveness: the joint distribution of incremental costs and effectiveness evaluated with a nonparametric bootstrap method. Med Decis Making. 1998;18(3):337–46.
Groot Koerkamp, B, Hunink, MG, Stijnen, T, et al. Limitations of acceptability curves for presenting uncertainty in cost-effectiveness analysis. Med Decis Making. 2007;27(2):101–11.
Ades, AE, Lu, G, Claxton, K. Expected value of sample information calculations in medical decision modeling. Med Decis Making. 2004;24(2):207–27.
Groot Koerkamp, B, Nikken, JJ, Oei, EH, et al. Value of information analysis used to determine the necessity of additional research: MR imaging in acute knee trauma as an example. Radiology. 2008;246(2):420–5.