Abdulla, A., Azevedo, I., and Morgan, M.G. (2013). “Expert Assessments of the Cost of Light Water Small Modular Reactors,” Proceedings of the National Academy of Sciences, 110(24), pp. 9686–9691.CrossRefGoogle ScholarPubMed

Anadon, L.D., Bosetti, V., Bunn, M., and Lee, A. (2012). “Expert Judgments about RD&D and the Future of Nuclear Energy,” Environmental Science & Technology, 41(21), pp. 11497–11504.Google Scholar

Anadon, L.D., Nemet, G.F., and Verdolini, E. (2013). “The Future Costs of Nuclear Power using Multiple Expert Elicitations: Effects of RD&D and Elicitation Design,” Environmental Research Letters, 8(3), p. 034020.CrossRefGoogle Scholar

Aspinall, W. (2010). “A Route to More Tractable Expert Advice,” Nature, 463, pp. 294–295.Google Scholar

Bolger, F. and Rowe, G. (2014). “Delphi: Somewhere between Scylla and Charybdis?,” Proceedings of the National Academy of Science, 111(41), p. E4284.Google Scholar

Bolger, F. and Wright, G. (2011). “Improving the Delphi Process: Lessons from Social Psychological Research,” Technological Forecasting and Social Change, 78(9), pp. 1500–1513.Google Scholar

Budnitz, R.J. et al. (1995). “Recommendations for Probabilistic Seismic Hazard Analysis: Guidance on Uncertainty and the Use of Experts,” UCRL-ID 122160, Lawrence Livermore National Laboratory.Google Scholar

Budnitz, R.J., et al. (1997). “Recommendations for Probabilistic Seismic Hazard Analysis: Guidance on Uncertainty and Use of Experts,” NUREG/CR-6372, Vol. II, U.S. Nuclear Regulatory Commission.Google Scholar

Budnitz, R.J., Apostolakis, G., Boore, D.M., Cluff, L.S., Coppersmith, K.J., Cornell, C.A., and Morris, P.A. (1998). “Use of Technical Expert Panels: Applications to Probabilistic Seismic Hazard Analysis,” Risk Analysis, 18(4), pp. 463–469.Google Scholar

Chan, G., Anadon, L.D., Chan, M., and Lee, A. (2011). “Expert Elicitation of Cost, Performance, and RD&D Budgets for Coal Power with CCS,” Energy Procedia, 4, pp. 2685–2692.Google Scholar

Charba, J.P. and Klein, W.H. (1980). “Skill in Precipitation Forecasting in the National Weather Service,” Bulletin of the American Meteorological Society, 61, pp. 1546–1555.Google Scholar

Christensen-Szalanski, J.J.J. and Bushyhead, J.B. (1981). “Physicians’ Use of Probabilistic Information in a Real Clinical Setting,” Journal of Experimental Psychology: Human Perception and Performance, 7(4), pp. 928–935.Google Scholar

Clark, W.C., Tomich, T.P., van Noordwijk, M., Guston, D., Catacutan, D., Dickson, N.M., and McNie, E. (2011). “Boundary Work for Sustainable Development: Natural Resource Management at the Consultative Group on International Agricultural Research (CGIAR),” Proceedings of the National Academy of Sciences, 113(17), 8pp.Google Scholar

Clemen, R.T. (2008). “Comment on Cooke’s Classical Method,” Reliability Engineering and System Safety, 93, pp. 760–765.Google Scholar

Clemen, R.T. and Winkler, R.L. (1999). “Combining Probability Distributions from Experts in Risk Analysis,” Risk Analysis, 19, pp. 187–203.Google Scholar

Clemen, R.T. and Winkler, R.L. (2007). “Chapter 9: Aggregating Probability Distributions,” in Edwards, W., Miles, R.F. Jr., and von Winterfeldt, D. (eds.), Advances in Decision Analysis: From Foundations to Applications, Cambridge University Press, pp. 154–176.Google Scholar

Colson, A.R. and Cooke, R.M. (2017). “Cross Validation for the Classical Model of Structured Expert Judgment,” Reliability Engineering and System Safety, 163, pp. 109–120.Google Scholar

Cooke, R.M. (1991). Experts in Uncertainty: Opinion and Subjective Probability in Science, Oxford University Press, 336pp.Google Scholar

Cooke, R.M. and Goossens, L.L.H.J. (2008). “TU Delft Expert Judgment Data Base,” Reliability Engineering and System Safety, 93, pp. 657–674.Google Scholar

Cooke, R.M., Wilson, A.M., Tuomisto, J.T., Morales, O., Tainio, M., and Evans, J.S. (2007). “A Probabilistic Characterization of the Relationship between Fine Particulate Matter and Mortality: Elicitation of European Experts,” Environmental Science & Technology, 41, pp. 6598–6605.Google Scholar

Curtright, A.E., Morgan, M.G., and Keith, D.W. (2008). “Expert Assessment of Future Photovoltaic Technology,” Environmental Science & Technology, 42(24), pp. 9031–9038.Google Scholar

Dalkey, N. and Helmer, O. (1972). “An Experimental Application of the Delphi Method to the Use of Experts,” RAND Corporation report RM- 727/1- (Abridged).Google Scholar

de Finetti, B. (1974). Theory of Probability: A Critical Introductory Treatment, Wiley, 2 vols.Google Scholar

DeGroot, M.H. (1970). Optimal Statistical Decisions, McGraw-Hill, 489pp.Google Scholar

Devilee, J.L.A. and Knol, A.B. (2011). “Software to Support Expert Elicitation: An Exploratory Study of Existing Software Packages,” RIVM Letter Report 630003001/2011 (Dutch National Institute of Public Health and Environment), 98pp. Available at: www.rivm.nl/en/Documents_and_publications/Scientific/Reports/2012/mei/Software_to_support_expert_elicitation_An_exploratory_study_of_existing_software_packages.Google Scholar

Dowlatabadi, H. and Morgan, M.G. (1993). “A Model Framework for Integrated Studies of the Climate Problem,” Energy Policy, 21(3), pp. 209–221.Google Scholar

EPA (2011). “Expert Elicitation Task Force White Paper.” Available at: www.epa.gov/stpc/pdfs/ee-white-paper-final.pdf.Google Scholar

Evans, J.S., Graham, J.D., Gray, D.M., and Sielken, R.L., Jr. (1994a). “A Distributional Approach to Characterizing Low-Dose Cancer Risk,” Risk Analysis, 14(1), pp. 25–34.Google Scholar

Evans, J.S., Gray, G.M., Sielken, R.L., Smith, A.E., Valdezflores, C., and Graham, J.D. (1994b). “Using of Probabilistic Expert Judgment in Uncertainty Analysis of Carcinogenic Potency,” Regulatory Toxicology and Pharmacology, 20 (1 pt.1), pp. 15–36.Google Scholar

Funtowicz, S.O. and Ravetz, J.R. (1990). Uncertainty and Quality in Science for Policy, Kluwer, 229pp.Google Scholar

Garthwaite, P.H., Kadane, J.B., and O’Hagan, A. (2005). “Statistical Methods for Eliciting Probability Distributions,” Journal of the American Statistical Association, 100, pp. 680–700.Google Scholar

Good, I.J. (1971). “46656 Varieties of Bayesians,” The American Statistician, 25(5), pp. 62–63.Google Scholar

Henrion, M. and Fischhoff, B. (1986). “Assessing Uncertainty in Physical Constants,” American Journal of Physics, 54(9), pp. 791–798.Google Scholar

Hoek, G., Boogaard, H., Knol, A. et al. (2010). “Concentration Response Functions for Ultrafine Particles and All-Cause Mortality and Hospital Admissions: Results of a European Expert Panel Elicitation,” Environmental Science & Technology, 44, pp. 476–482.Google Scholar

Hora, S.C. (2007). “Chapter 9: Eliciting Probability from Experts,” in Ward, E.W., Miles, R.F. Jr., and von Winterfeldt, D. (eds.), Advances in Decision Analysis: From Foundations to Applications, Cambridge University Press, pp. 129–153.CrossRefGoogle Scholar

IPCC (2001). “Climate Change 2001: The Scientific Basis,” in Houghton, J.T. et al. (eds.), Contribution of Working Group I to the Third Assessment Report of the Intergovernmental Panel on Climate Change, Cambridge University Press, 881pp.Google Scholar

IPCC (2007). “Climate Change 2007: The Physical Science Basis,” in Solomon, S. et al. (eds.), Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, Cambridge University Press, 800pp.Google Scholar

IPCC (2013). “Summary for Policymakers in Climate Change 2013: The Physical Science Basis,” in Stocker, T.F. et al. (eds.), Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, Cambridge University Press.Google Scholar

Jaynes, E.T. (2003). Probability Theory: The Logic of Science, Cambridge University Press, 727pp.Google Scholar

Kadane, J. and Fischhoff, B. (2013). “A Cautionary Note of Global Recalibration,” Judgment and Decision Making, 8(1), pp. 25–28.Google Scholar

Kahneman, D., Slovic, P., and Tversky, A. (eds.) (1982). Judgment under Uncertainty: Heuristics and Biases, Cambridge University Press, 555pp.Google Scholar

Karvetski, C.W., Olson, K.C., Mandel, D.R., and Twardy, C.R. (2013). “Probabilistic Coherence Weighting for Optimizing Expert Forecasts,” Decision Analysis, 10, pp. 305–326.Google Scholar

Knol, A.B., de Hartog, J.J., Boogaard, H. et al. (2009). “Expert Elicitation on Ultrafine Particles: Likelihood of Health Effects and Causal Pathways,” Particle and Fibre Toxicology, 6, p. 19.Google Scholar

Knol, A.B., Slottje, P., van der Sluijs, J.P., and Lebret, E. (2010). “The Use of Expert Elicitation in Environmental Health Impact Assessment: A Seven Step Procedure,” Environmental Health, 9, p. 19.CrossRefGoogle ScholarPubMed

Krayer von Krauss, M.P., Casman, E.A., and Small, M.J. (2004). “Elicitation of Expert Judgments of Uncertainty in the Risk Assessment of Herbicide-Tolerant Oilseed Crops,” Risk Analysis, 24(6), pp. 1515–1527.Google Scholar

Lichtenstein, S., Fischhoff, B., and Phillips, L. (1982). “Chapter 22: Calibration of Probabilities: The State of the Art to 1980,” in Kahneman, D., Slovic, P., and Tversky, A. (eds.), Judgment under Uncertainty: Heuristics and Biases, Cambridge University Press, pp. 306–334.Google Scholar

Lin, S.W. and Cheng, C.H. (2009). “The Reliability of Aggregated Probability Judgments Obtained through Cooke’s Classical Method,” Journal of Modeling in Management, 4, pp. 149–161.Google Scholar

Mandel, D. and Barnes, A. (2014). “Accuracy of Forecasts in Strategic Intelligence,” Proceedings of the National Academy of Sciences, 111(30), pp. 10984–10989.Google Scholar

Mastrandrea, M.D. et al. (2010). “Guidance Note for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties, Intergovernmental Panel on Climate Change (IPCC).” Available at: www.ipcc.ch/pdf/supporting-material/uncertainty-guidance-note.pdf.Google Scholar

Morgan, M.G. (2014). “The Use (and Abuse) of Expert Elicitation in Support of Decision Making for Public Policy,” PNAS, 111(20), pp. 7176–7184.Google Scholar

Morgan, M.G., Adams, P., and Keith, D.W. (2006). “Elicitation of Expert Judgments of Aerosol Forcing,” Climatic Change, 75, pp. 195–214.Google Scholar

Morgan, M.G. and Dowlatabadi, H. (1996). “Learning from Integrated Assessment of Climate Change,” Climatic Change, 34, pp. 337–368.Google Scholar

Morgan, M.G. with Dowlatabadi, H., Henrion, M., Keith, D., Lempert, R., McBride, S., Small, M., and Wilbanks, T. (2009). CCSP 5.2 Best Practice Approaches for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Decisionmaking, Report by the Climate Change Science Program and the Subcommittee on Global Change Research, National Oceanic and Atmospheric Administration, 96pp.Google Scholar

Morgan, M.G. and Henrion, M. with a chapter by Small, M. (1990). Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis, Cambridge University Press, 332pp.Google Scholar

Morgan, M.G. and Keith, D. (1995). “Subjective Judgments by Climate Experts,” Environmental Science & Technology, 29(10), pp. 468A–476A.Google Scholar

Morgan, M.G., Morris, S.C., Henrion, M., Amaral, D.A.L., and Rish, W.R. (1984). “Technical Uncertainty in Quantitative Policy Analysis: A Sulfur Air Pollution Example,” Risk Analysis, 4, pp. 201–216.Google Scholar

Morgan, M.G., Morris, S.C., Rish, W.R., and Meier, A.K. (1978). “Sulfur Control in Coal-Fired Power Plants: A Probabilistic Approach to Policy Analysis,” Journal of the Air Pollution Control Association, 28, pp. 993–997.Google Scholar

Morgan, M.G., Pitelka, L.F., and Shevliakova, E. (2001). “Elicitation of Expert Judgments of Climate Change Impacts on Forest Ecosystems,” Climatic Change, 49(3), pp. 279–307.CrossRefGoogle Scholar

Morris, D.E., Oakley, J.E., and Crowe, J.A. (2014). “A Web-Based Tool for Eliciting Probability Distributions from Experts,” Environmental Modelling and Software, 52, pp. 1–4.Google Scholar

Moss, R. and Schneider, S.H. (2000). “Uncertainties in the IPCC TAR: Recommendations to Lead Authors for More Consistent Assessment and Reporting,” in Pachauri, R. et al. (eds.), Guidance Papers on the Cross Cutting Issues of the Third Assessment Report of the IPCC, pp. 33–51. Available at: www.ipcc.ch/pdf/supportingmaterial/guidance-papers-3.Google Scholar

Murphy, A.H. and Winkler, R.L. (1977). “Can Weather Forecasters Formulate Reliable Probability Forecasts of Precipitation and Temperature?,” National Weather Digest, 2, pp. 2–9(a).Google Scholar

National Assessment Synthesis Team (2001). Climate Change Impacts on the United States: The Potential Consequences of Climate Variability and Change, Cambridge University Press, 612pp.Google Scholar

NDU (1978). “February,” in Climate Change to the Year 2000: A Survey of Expert Opinion, Report published by the National Defense University in cooperation with the U.S. Department of Agriculture, the Defense Advanced Research Projects Agency, the National Oceanic and Atmospheric Administration, and Institute for the Future. [For a critique, see Stewart, T.R. and Glantz, M.H. (1985). “Expert Judgment and Climate Forecasting: A Methodological Critique of Climate Change to the Year 2000,” Climatic Change, 7, pp. 159–183.]Google Scholar

O’Hagan, A., Buck, C.E., Daneshkhah, A., Eiser, J.R., Garthwaite, P.H., Jenkinson, D.J., Oakley, J.E., and Rakow, T. (2006). Uncertain Judgments: Eliciting Experts’ Probabilities, John Wiley & Sons, 321pp.Google Scholar

O’Hagan, A. and Oakley, J.E. (2010). “SHELF: The Sheffield Elicitation Framework, Version 2.0, School of Mathematics and Statistics,” University of Sheffield. Available at: www.tonyohagan.co.uk/shelf.Google Scholar

Oppenheimer, M., O’Neill, B.C., Webster, M., and Agerwala, S. (2007). “The Limit of Consensus,” Science, 317, pp. 1505–1506.Google Scholar

Presidential/Congressional Commission on Risk Assessment and Risk Management (1997). Vol. I: Framework for Environmental Health Risk Management; Vol. II: Risk Assessment and Risk Management in Regulatory Decision Making. Available at: http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=55006.Google Scholar

Rao, A.B., Rubin, E.S., Keith, D.W., and Morgan, M.G. (2006). “Evaluation of Potential Cost Reductions from Improved Amine-Based CO₂ Capture Systems,” Energy Policy, 34, pp. 3765–3772.Google Scholar

Refsgaard, J.C., Van der Sluijs, J.P., Brown, J., and Van der Keur, P. (2005). “A Framework for Dealing with Uncertainty Due to Model Structure Error,” Water Resources, 29, pp. 1586–1597.Google Scholar

Roman, H.A., Walker, K.D., Walsh, T.L., Conner, L., Richmond, H.M., Hubbell, B.J., and Kinney, P.L. (2008). “Expert Judgment Assessment of the Mortality Impact of Changes in Ambient Fine Particulate Matter in the U.S.,” Environmental Science & Technology, 42, pp. 2268–2274.Google Scholar

Rowe, G. and Wright, G. (1999). “The Delphi Technique as a Forecasting Tool: Issues and Analysis,” International Journal of Forecasting, 15(4), pp. 353–375.Google Scholar

Sackman, H. (1975). Delphi Critique: Expert Opinion, Forecasting, and Group Process, Lexington Books, 142pp.Google Scholar

Schneider, S. et al. (2007). “Climate Change 2007: Impacts, Adaptation, and Vulnerabilities,” in Parry, M.L. et al. (eds.), Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, Cambridge University Press, pp. 779–810.Google Scholar

Spetzler, C.S. and Staël von Holstein, C.-A.S. (1975). “Probability Encoding in Decision Analysis,” Management Science, 22(3), pp. 340–358.Google Scholar

Tversky, A. and Kahneman, D. (1974). “Judgments under Uncertainty: Heuristics and Biases,” Science, 185(4157), pp. 1124–1131.Google Scholar

Van der Sluijs, J.P., Craye, M., Funtowicz, S., Kloprogge, P., Ravetz, J., and Risbey, J. (2005a). “Combining Quantitative and Qualitative Measures of Uncertainty in Model-Based Environmental Assessment: The NUSAP System,” Risk Analysis, 25(2), pp. 481–492.Google Scholar

Van der Sluijs, J.P., Risbey, J.S., and Ravetz, J. (2005b). “Uncertainty Assessment of VOC Emissions from Paint in the Netherlands using the NUSAP System,” Environmental Monitoring and Assessment, 105, pp. 229–259.Google Scholar

Wallsten, T.S., Budescu, D.V., Rapoport, A., Zwick, R., and Forsyth, B. (1986). “Measuring the Vague Meanings of Probability Terms,” Journal of Experimental Psychology: General, 155(4), pp. 348–365.Google Scholar

Wallsten, T.S. and Whitfield, R.G. (1989). “A Risk Assessment for Selected Lead-Induced Health Effects: An Example of a General Methodology,” Risk Analysis, 9(2), pp. 197–207.Google Scholar

Wardekker, J.A., van der Sluijs, J.P., Janssen, P.H.M., Kloprogge, P., and Petersen, A.C. (2008). “Uncertainty Communication in Environmental Assessments: Views from the Dutch Science–Policy Interface,” Environmental Science and Policy, 11, pp. 627–641.Google Scholar

Woudenberg, F. (1991). “An Evaluation of Delphi,” Technology Forecasting and Social Change, 40(2), pp. 131–150.Google Scholar

Zickfeld, K., Levermann, A., Morgan, M.G., Kuhlbrodt, T., Rahmstorf, S., and Keith, D.W. (2007). “Expert Judgments on the Response on the Atlantic Meridional Overturning Circulation to Climate Change,” Climatic Change, 82, pp. 235–265.Google Scholar

Zickfeld, K., Morgan, M.G., Frame, D., and Keith, D.W. (2010). “Expert Judgments about Transient Climate Response to Alternative Future Trajectories of Radiative Forcing,” Proceedings of the National Academy of Sciences, 107, pp. 12451–12456.Google Scholar