Hostname: page-component-78c5997874-j824f Total loading time: 0 Render date: 2024-11-17T18:10:06.264Z Has data issue: false hasContentIssue false
  • English
  • Français

Eliciting and Combining Subjective Judgments About Uncertainty

Published online by Cambridge University Press:  10 March 2009

Robert L. Wolpert
Robert L. Wolpert
Affiliation:
Duke University
Get access Rights & Permissions [Opens in a new window]

Extract

Frequently, health care decisions must be made before compelling and unequivocal evidence is available about the benefits, risks, and costs of a proposed new health technology. It is common for decision makers to seek the advice and opinions of experts to supplement the available evidence. This article reviews some of the methods used or recommended for eliciting the opinions of experts about uncertain events and for combining these opinions with those of others and with available empirical evidence.

Type
Special Section: Alternative Methods for Assessing Technology, Part 1
Information
International Journal of Technology Assessment in Health Care , Volume 5 , Issue 4 , October 1989 , pp. 537 - 557
Copyright
Copyright © Cambridge University Press 1989

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Chaloner, K. M. & Duncan, G. T.Assessment of a beta prior distribution: PM elicitation. The Statistician, 1983, 32, 174–80.CrossRefGoogle Scholar
Clemen, R. T.Calibration and the aggregation of probabilities. Management Science, 1986, 32, 312–14.CrossRefGoogle Scholar
Clemen, R. T. & Winkler, R. L.Limits for the precision and value of information from dependent sources. Operations Research, 1985, 33, 427–42.CrossRefGoogle Scholar
Clemen, R. T. & Winkler, R. L.Combining economic forecasts. Journal of Business and Economic Statistics, 1986, 4, 39–46.CrossRefGoogle Scholar
Clemen, R. T. & Winkler, R. L. Calibrating and combining precipitation probability forecasts. In Viertl, R. (ed.), Probability and Bayesian statistics. New York: Plenum, 1987, 97–110.CrossRefGoogle Scholar
Dalkey, N. C. & Helmer, O.An experimental application of the Delphi method to the use of experts. Management Science, 1963, 9, 458.CrossRefGoogle Scholar
de, Finetti B.Probability, induction, and statistics. New York: Wiley, 1972.Google Scholar
DeGroot, M. H.Reaching a consensus. Journal of the American Statistical Association, 1974, 69, 118–21.CrossRefGoogle Scholar
Fishburn, P. C.Subjective expected utility: A review of normative theories. Theory and Decision, 1981, 13, 139–99.CrossRefGoogle Scholar
French, S.Updating of belief in the light of someone else's opinion. Journal of the Royal Statistical Society (Series A), 1980, 143, 43–48.CrossRefGoogle Scholar
French, S.Consensus of opinion. European Journal of Operations Research, 1981, 7, 332–40.CrossRefGoogle Scholar
French, S. Group consensus probability distributions: A critical survey in Bayesian statistics. In Bernardo, J. M., De, M. H. Groot, Lindley, D. V. & Smith, A. F. M. (eds.), Bayesian statistics 2, Amsterdam: North-Holland, 1983.Google Scholar
French, S.Calibration and the expert problem. Management Science, 1986, 32, 315–21.CrossRefGoogle Scholar
Genest, C. & Zidek, J. V.Combining probability distributions. Statistical Science, 1986, 1, 114–48.CrossRefGoogle Scholar
Hogarth, R. M.Cognitive processes and the assessment of subjective probability distributions. Journal of the American Statistical Association, 1975, 70, 271–89.CrossRefGoogle Scholar
Kadane, J. B. Predictive and structural methods for eliciting prior distributions. In Zellner, A. (ed.), Studies in Bayesian analysis in econometrics and statistics in honor of Harold Jeffreys. Amsterdam: North-Holland, 1980, 89–93.Google Scholar
Kadane, J. B., Dickey, J. M., Winkler, R. L. et al. , Interactive elicitation of opinion for a normal linear model. Journal of the American Statistical Association, 1980, 75, 845–54.CrossRefGoogle Scholar
Kadane, J. B. & Winkler, R. L. De Finetti's methods of elicitation. In Viertl, R. (ed.), Probability and Bayesian statistics. New York: Plenum, 1987, 97–110.Google Scholar
Keeney, R. L. & Raiffa, H.Decisions with multiple objectives: Preferences and value tradeoffs. New York: Wiley, 1976.Google Scholar
Lindley, D. V.The improvement of probability judgements. Journal of the Royal Statistical Society (Series A), 1982, 145, 117–126.CrossRefGoogle Scholar
Lindley, D. V.Another look at an axiomatic approach to expert resolution. Management Science, 1986, 32, 303–06.CrossRefGoogle Scholar
Lindley, D. V., Tversky, A. & Brown, R. V.On the reconciliation of probability assessments (with discussion). Journal of the Royal Statistical Society (Series A), 1979, 142, 146–80.CrossRefGoogle Scholar
Makridakis, S. & Winkler, R. L.Averages of forecasts: Some empirical results. Management Science, 1983, 29, 987–96.CrossRefGoogle Scholar
Merkhofer, M. W. Quantifying judgemental uncertainty: Methodology, experiences, and insights. IEEE Transactions on Systems, Man, and Cybernetics, 1987, SMC-17, 741–52.CrossRefGoogle Scholar
Morris, P. A.Decision analysis expert use. Management Science, 1974, 20, 1233–41.CrossRefGoogle Scholar
Morris, P. A.An axiomatic approach to expert resolution. Management Science, 1983, 29, 24–32.CrossRefGoogle Scholar
Morris, P. A.Observations on expert aggregation. Management Science, 1986, 32, 321–28.CrossRefGoogle Scholar
Raiffa, H. & Schlaifer, R.Applied statistical decision theory. Cambridge, MA: Harvard University Press, 1961.Google Scholar
Schervish, M. J.Comments.on some axioms for combining expert judgements. Management Science, 1986, 32, 306–12.CrossRefGoogle Scholar
Spetzler, C. S. & Staäl, von Holstein C-A. S.Probability encoding in decision analysis. Management Science, 1975, 22, 340–58.CrossRefGoogle Scholar
Tversky, A. & Kahneman, D.Availability: A heuristic for judging frequency and probability. Cognitive Psychology, 1973, 5, 207–32.CrossRefGoogle Scholar
Tversky, A. & Kahneman, D.The framing of decisions and the psychology of choice. Science, 1981, 211, 453–58.CrossRefGoogle ScholarPubMed
Tversky, A. & Kahneman, D.Extensional versus intuitive reasoning: The conjunctive fallacy in probability judgement. Psychological Review, 1983, 90, 293–315.CrossRefGoogle Scholar
Tversky, A. & Kahnexzman, D. Rational choice and the framing of decisions. Preprint, 1986.CrossRefGoogle Scholar
von, Winterfeldt D. & Edwards, W.Decision analysis and behavioral research. Cambridge: Cambridge University Press, 1986.Google Scholar
West, M. Bayesian analysis of agent opinion. ISDS discussion paper 89–11, Institute of Statistics and Decision Science, Duke University, 1989.Google Scholar
West, M. Modeling agent forecast distributions. ISDS discussion paper 89–13, Institute of Statistics and Decision Sciences, Duke University, 1989.Google Scholar
Winkler, R. L.The consensus of subjective probability distributions. Management Science, 1968, 15, B61–B75.CrossRefGoogle Scholar
Winkler, R. L. Prior information, predictive distributions, and Bayesian model-building. In Zellner, A. (ed.), Studies in Bayesian analysis in econometrics and statistics in honour of Harold Jeffreys. Amsterdam: North-Holland, 1980, 95–109.Google Scholar
Winkler, R. L.Combining probability distributions from dependent information sources. Management Science, 1981, 27, 479–88.CrossRefGoogle Scholar
Winkler, R. L.Expert resolution. Management Science, 1986, 32, 298–303.CrossRefGoogle Scholar
Winkler, R. L. On “good probability appraisers.” In Goel, P. & Zellner, A. (eds.), Bayesian inference and decision techniques. New York: Elsevier Science Publishers, 1986, 265–78.Google Scholar
Winkler, R. L. & Murphy, A. H.“Good” probability assessors. Journal of Applied Meteorology, 1968, 7, 751–58.2.0.CO;2>CrossRefGoogle Scholar
Winkler, R. L., Smith, W. S. & Kulkarni, R. B.Adaptive forecasting models based on predictive distributions. Management Science, 1978, 24, 977–86.CrossRefGoogle Scholar
Zidek, J. V. Group decision analysis and its application to combining opinions. Preprint, 1986.Google Scholar