Hostname: page-component-8448b6f56d-mp689 Total loading time: 0 Render date: 2024-04-23T09:44:22.067Z Has data issue: false hasContentIssue false
  • English
  • Français

Why Average When You Can Stack? Better Methods for Generating Accurate Group Credences

Published online by Cambridge University Press:  25 April 2022

David Kinney Open the ORCID record for David Kinney [Opens in a new window]
David Kinney*
Affiliation:
Princeton University, 222 Peretsman Scully Hall, Princeton, NJ, US Email: david.kinney@princeton.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Formal and social epistemologists have devoted significant attention to the question of how to aggregate the credences of a group of agents who disagree about the probabilities of events. Moss (2011) and Pettigrew (2019) argue that group credences can be a linear mean of the credences of each individual in the group. By contrast, I argue that if the epistemic value of a credence function is determined solely by its accuracy, then we should, where possible, aggregate the underlying statistical models that individuals use to generate their credence functions, using “stacking” techniques from statistics and machine learning first developed by Wolpert (1992).

Type
Article
Information
Philosophy of Science , Volume 89 , Issue 4 , October 2022 , pp. 845 - 863
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of the Philosophy of Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

Many thanks to David Wolpert for first introducing me to the literature on stacking. For their helpful comments on various drafts, I am also grateful to Hein Duijf, Remco Heesen, James Nguyen, Richard Pettigrew, Joe Roussos, Jeremy Strasser, David Watson, Kevin Zollman, and two anonymous reviewers for this journal, as well as audiences at the London School of Economics and Political Science (LSE) Choice Group Seminar, the 2020 Formal Epistemology Workshop, the 2020 Conference on Bayesian Epistemology: Perspectives and Challenges at the Munich Center for Mathematical Philosophy, and the 2020 Workshop on the Wisdom and Madness of Crowds at the Institute for Logic, Language, and Computation at the University of Amsterdam.

References

Berker, Selim. 2013. “The Rejection of Epistemic Consequentialism.” Philosophical Issues 23 (1):363–87.CrossRefGoogle Scholar
Bernardo, José M., and Smith, Adrian F. M.. 1994. Bayesian Theory. New York: John Wiley & Sons.CrossRefGoogle Scholar
Bradley, Richard. 2007. “Reaching a Consensus.” Social Choice and Welfare 29 (4):609–32.CrossRefGoogle Scholar
Breiman, Leo. 1996a. “Bagging Predictors.” Machine Learning 24 (2):123–40.CrossRefGoogle Scholar
Breiman, Leo. 1996b. “Stacked Regressions.” Machine Learning 24 (1):49–64.CrossRefGoogle Scholar
Clyde, Merlise, and Iversen, Edwin S.. 2013. “Bayesian Model Averaging in the M-Open Framework.” In Bayesian Theory and Applications, edited by Damien, P., Dellaportas, P., Polson, N. G., and Stephens, D. A., 483–98. Oxford: Oxford University Press.CrossRefGoogle Scholar
Dawid, A. Philip. 1982. “The Well-Calibrated Bayesian.” Journal of the American Statistical Association 77 (379):605–10.CrossRefGoogle Scholar
Dawid, A. Philip. 1984. “Present Position and Potential Developments: Some Personal Views. Statistical Theory. The Prequential Approach.” Journal of the Royal Statistical Society: Series A (General) 147 (2):278–90.CrossRefGoogle Scholar
Dietrich, Franz. 2019. “A Theory of Bayesian Groups.” Noûs 53 (3):708–36.CrossRefGoogle Scholar
Dietrich, Franz, and List, Christian. 2017. “Probabilistic Opinion Pooling Generalized. Part One: General Agendas.” Social Choice and Welfare 48 (4):747–86.CrossRefGoogle Scholar
Goldman, Alvin. 2001. “The Unity of the Epistemic Virtues.” In Virtue Epistemology: Essays on Epistemic Virtue and Responsibility, edited by Fairweather, A. and Zagzebski, L., 30–48. Oxford: Oxford University Press.Google Scholar
Jehle, David, and Fitelson, Branden. 2009. “What Is the ‘Equal Weight View’?” Episteme 6 (3):280–93.CrossRefGoogle Scholar
Joyce, James M. 1998. “A Nonpragmatic Vindication of Probabilism.” Philosophy of Science 65 (4):575–603.CrossRefGoogle Scholar
Kadane, Joseph B., and Lichtenstein, Sarah. 1982. A Subjectivist View of Calibration. Technical Report. Springfield, OR: Decision Research.CrossRefGoogle Scholar
Kuan, Ko-Hung. 2021. “Beyond Linear Conciliation.” Synthese 198 (12):11483–504.CrossRefGoogle Scholar
Le, Tri, and Clarke, Bertrand. 2017. “A Bayes Interpretation of Stacking for ${\cal M}$ -Complete and ${\cal M}$ -Open Settings.” Bayesian Analysis 12 (3):807–29.CrossRefGoogle Scholar
Lehrer, Keith, and Wagner, Carl. 1983. “Probability Amalgamation and the Independence Issue: A Reply to Laddaga.” Synthese 55 (3):339–46.CrossRefGoogle Scholar
Masoudnia, Saeed, and Ebrahimpour, Reza. 2014. “Mixture of Experts: A Literature Survey.” Artificial Intelligence Review 42 (2):275–93.CrossRefGoogle Scholar
Moss, Sarah. 2011. “Scoring Rules and Epistemic Compromise.” Mind 120 (480):1053–69.CrossRefGoogle Scholar
Pettigrew, Richard. 2019. “On the Accuracy of Group Credences.” In Oxford Studies in Epistemology, vol. 6, edited by Gendler, T. S. and Hawthorne, J., 137–60. Oxford: Oxford University Press.CrossRefGoogle Scholar
Rougier, Jonathan. 2016. “Ensemble Averaging and Mean Squared Error.” Journal of Climate 29 (24):8865–70.CrossRefGoogle Scholar
Russell, Jeffrey Sanford, Hawthorne, John, and Buchak, Lara. 2015. “Groupthink.” Philosophical Studies 172 (5):1287–1309.CrossRefGoogle Scholar
Schapire, Robert E., and Freund, Yoav. 2013. Boosting: Foundations and Algorithms. Bingley, UK: Emerald Group.Google Scholar
Sill, Joseph, Takács, Gábor, Mackey, Lester, and Lin, David. 2009. “Feature-Weighted Linear Stacking.” arXiv preprint, arXiv:0911.0460.Google Scholar
Staffel, Julia. 2015. “Disagreement and Epistemic Utility-Based Compromise.” Journal of Philosophical Logic 44 (3):273–86.Google Scholar
Steele, Katie. 2012. “Testimony as Evidence: More Problems for Linear Pooling.” Journal of Philosophical Logic 41 (6):983–99.CrossRefGoogle Scholar
Van der Laan, Mark J., Polley, Eric C., and Hubbard, Alan E.. 2007. “Super Learner.” Statistical Applications in Genetics and Molecular Biology 6 (1):article 25.CrossRefGoogle ScholarPubMed
Wagner, Carl. 1985. “On the Formal Properties of Weighted Averaging as a Method of Aggregation.” Synthese 62 (1):97–108.CrossRefGoogle Scholar
Wolpert, David H. 1992. “Stacked Generalization.” Neural Networks 5 (2):241–59.CrossRefGoogle Scholar
Yao, Yuling, Pirš, Gregor, Vehtari, Aki, and Gelman, Andrew. 2021. “Bayesian Hierarchical Stacking.” arXiv preprint, arXiv:2101.08954.Google Scholar
Yuksel, Seniha Esen, Wilson, Joseph N., and Gader, Paul D.. 2012. “Twenty Years of Mixture of Experts.” IEEE Transactions on Neural Networks and Learning Systems 23 (8):1177–93.CrossRefGoogle ScholarPubMed