Skip to main content Accessibility help
  • Get access
    Check if you have access via personal or institutional login
  • Cited by 5
  • Print publication year: 2005
  • Online publication date: February 2010

19 - Is Confidence in Decisions Related to Feedback? Evidence from Random Samples of Real-World Behavior


A central theme of this book is how people learn from samples. In this chapter, we look at this issue from two viewpoints. First, we question the manner in which we – as scientists – sample the environments of the experimental participants who engage in the judgment and decision-making tasks that we study. To what extent are these samples representative of the natural decision-making ecology that our participants face in their everyday lives? Second, by actually sampling people's activities in their natural ecologies, we seek to characterize how they experience these environments. In particular, we investigate how one feature of environments (the presence or absence of feedback) affects inferences (the confidence people express in their decisions).

The first question is, of course, not new. For many years, psychologists have been concerned about how to generalize behavior from experimental evidence (see, e.g., Brunswik, 1956; Chapanis, 1961, 1967; Cronbach, 1975; Ebbesen & Konečni, 1980; Hammond, 1978; Hogarth, 1986; Lipshitz et al., 2001). I do not propose to add to this debate. Instead, the contribution is to demonstrate how the use of readily available technology can greatly facilitate random sampling of decision behavior outside the psychological laboratory. Indeed, as I shall argue in the following, obtaining appropriate samples of human decision behavior is not as difficult as might be imagined by researchers trained within experimental paradigms.

Related content

Powered by UNSILO
Brunswik, E. (1944). Distal focusing of perception: Size constancy in a representative sample of situations. Psychological Monographs, 56 (254), 1–49
Brunswik, E. (1956). Perception and the representative design of psychological experiments. Berkeley: University of California Press
Camerer, C. F. (1981). The validity and utility of expert judgment. PhD dissertation, University of Chicago
Chapanis, A. (1961). Men, machines, and models. American Psychologist, 16, 113–131
Chapanis, A. (1967). The relevance of laboratory studies to practical situations. Ergonomics, 10, 557–577
Cronbach, L. J. (1975). Beyond the two disciplines of scientific psychology. American Psychologist, 30, 116–127
Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York: Harper-Collins
Csikszentmihalyi, M., & Larson, R. (1987). Validity and reliability of the experience-sampling method. Journal of Nervous and Mental Disease, 175, 526–536
Ebbesen, E. B., & Konecni, V. J. (1980). On the external validity of decision-making research: What do we know about decisions in the real world? In Wallsten, T. S. (Ed.), Cognitive processes in choice and decision behavior. Hillside, NJ: Lawrence Erlbaum
Einhorn, H. J. (1980). Learning from experience and suboptimal rules in decision making. In Wallsten, T. S. (Ed.), Cognitive processes in choice and decision behavior. Hillside, NJ: Lawrence Erlbaum
Einhorn, H. J., & Hogarth, R. M. (1978). Confidence in judgment: Persistence of the illusion of validity. Psychological Review, 85, 395–416
Ericsson, A., & Simon, H. A. (1984). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press
Fiedler, K. (2000). Beware of samples! A cognitive-ecological sampling approach to judgment biases. Psychological Review, 107, 659–676
Gigerenzer, G., Hoffrage, U., & Kleinbölting, H. (1991). Probabilistic mental models: A Brunswikian theory of confidence. Psychological Review, 98, 506–528
Goldstein, D. G., et al. (2001). Group report: Why and when do simple heuristics work? In Gigerenzer, G. & Selten, R. (Eds.), Bounded rationality: The adaptive toolbox. Cambridge, MA: MIT Press
Goleman, D. (1998). Working with emotional intelligence. New York: Bantam
Hammond, K. R. (1978). Psychology's scientific revolution: Is it in danger? Reprint 211, Center for Research on Judgment and Policy, University of Colorado, Boulder, CO
Harvey, N. (1997). Confidence in judgment. Trends in Cognitive Science, 1, 78–82
Hogarth, R. M. (1986). Generalization in decision research: The role of formal models. IEEE Transactions in Systems, Man, and Cybernetics, SMC-16(3), 439–449
Hogarth, R. M. (2001). Educating intuition. Chicago: University of Chicago Press
Hurlburt, R. T. (1997). Randomly sampling thinking in the natural environment. Journal of Consulting and Clinical Psychology, 67, 941–949
Isen, A. (1993). Positive affect and decision making. In Lewis, M. & Haviland, J. M. (Eds.), Handbook of emotions (pp. 261–277). New York: Guilford
Juslin, P. (1993). An explanation of the hard-easy effect in studies of realism of confidence in one's general knowledge. European Journal of Cognitive Psychology, 5, 55–71
Juslin, P. (1994). The overconfidence phenomenon as a consequence of informal experimenter-guided selection of almanac items. Organizational Behavior and Human Decision Processes, 57, 226–246
Juslin, P., Winman, A., & Olssen, H. (2000). Naïve empiricism and dogmatism in confidence research: A critical examination of the hard-easy effect. Psychological Review, 107, 384–396
Klayman, J., Soll, J. B., González-Vallejo, C., & Barlas, S. (1999). Overconfidence: It depends on how, what, and whom you ask. Organizational Behavior and Human Decision Processes, 79, 216–247
Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982). Calibration of probabilities: The state of the art to 1980. In Kahneman, D., Slovic, P., & Tversky, A. (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 306–334). Cambridge, UK: Cambridge University Press
Lipshitz, R., Klein, G., Orasanu, J., & Salas, E. (2001). Taking stock of naturalistic decision making. Journal of Behavioral Decision Making, 14, 331–352
McFarlane, J., Martin, C. L., & Williams, T. M. (1988). Mood fluctuations: Women versus men and menstrual versus other cycles. Psychology of Women Quarterly, 12, 201–223
Murphy, A., & Winkler, R. L. (1984). Probability forecasting in meteorology. Journal of the American Statistical Association, 79, 489–500
Reber, A. S. (1993). Implicit learning and tacit knowledge: An essay on the cognitive unconscious. New York: Oxford University Press
Russo, J. E., & Schoemaker, P. J. H. (2002). Winning decisions. New York: Doubleday
Schwartz, B. (1982). Reinforcement-induced behavioral stereotypy: How not to teach people to discover rules. Journal of Experimental Psychology: General, 111, 23–59
Sjöberg, L., & Magneberg, R. (1990). Action and emotion in everyday life. Scandinavian Journal of Psychology, 31, 9–27
Taylor, S. E., & Brown, J. D. (1988). Illusion and well-being: A social psychological perspective on mental health. Psychological Bulletin, 103, 193–210
Taylor, S. E., & Brown, J. D. (1994). Positive illusions and well-being revisited: Separating fact from fiction. Psychological Bulletin, 116, 21–27
Teuchmann, K., Totterdell, P., & Parker, S. K. (1999). Rushed, unhappy, and drained: An experience sampling study of relations between time pressure, perceived control, mood, and emotional exhaustion in a group of accountants. Journal of Occupational Health Psychology, 4, 37–54
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 211, 453–458