Skip to main content Accessibility help

Decloaking the privileged expert

  • David J Weiss (a1) and James Shanteau (a2)


When seeking help or advice, one naturally wishes to get that aid from someone who does the task well, from an expert. Determining whether someone merits that label is not trivial, and the judgment can have important consequences. Experts in most domains, from athletes to plumbers, need to exhibit a high level of performance to maintain their status. But there is another group of professionals that is never examined. The expert status of such experts is conferred via criteria such as education or experience. While their credentials may be challenged, their work itself is not seen to bear upon their status. We refer to this group as privileged experts. These experts make assessments or predictions whose correctness is rarely evaluated. Even if prophecy fails, errors are forgiven. We account for this non-evidentiary perspective by positing a widely-held misconception that expertise generalizes. We outline a task-specific, performance-based, limited-term certification scheme that can supplant privilege.



Hide All
Ackerman, P. L., & Cianciolo, A. T. (2002). Ability and task constrain determinants of complex task performance. Journal of Experimental Psychology: Applied, 8, 194208.
Armstrong, J. S. (1980). The seer-sucker theory: The value of experts in forecasting. Technology Review, 83, 1624.
Colvin, G. (2008). Talent is overrated. New York, NY: Penguin.
Cooke, R. (1991). Experts in uncertainty. New York, NY: Oxford University Press.
Duch, J., Waitzman, J. S., & Amaral, L. A. N. (2010). Quantifying the performance of individual players in a team activity. PLoS One, 5, e10937.
Ericsson, K. A., Charness, N., Feltovich, P. J., & Hoffman, R. R. (2006). The Cambridge handbook of expertise and expert performance. New York, NY: Cambridge University Press.
Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100, 363406.
Ericsson, K. A., & Smith, J. (1991). Toward a general theory of expertise: Prospects and limits. Cambridge, England: Cambridge University Press.
Ettenson, R., Shanteau, J., & Krogstad, J. (1987). Expert judgment: Is more information better? Psychological Reeports, 60, 227238.
Gawande, A. (2007). Better: A surgeons notes on performance. New York, NY: Picador.
Green, K. C., & Armstrong, J. S. (2007). The ombudsman: Value of expertise for forecasting decisions in conflicts. Interfaces, 37, 287299.
Herrnstein, R. J., & Murray, C. J. (1994). The bell curve: Intelligence and class structure in American life. New York, NY: The Free Press.
Keeney, R. L., & von Winterfeldt, D. (1989). On the uses of expert judgment on complex technical problems. IEEE Transactions on Engineering Management, 36, 8386.
Keeney, R. L., & von Winterfeldt, D. (1991). Eliciting probabilities from experts in complex technical problems. IEEE Transactions on Engineering Management, 38, 191201.
Lewis, M. (2003). Moneyball: The art of winning an unfair game. New York, NY: W. W. Norton.
Mlodinow, L. (2008). The Drunkard's walk: How randomness rules our lives. New York, NY: Pantheon.
Nagy, G. F. (1981). How are personnel selection decisions made? An analysis of decision strategies in a simulated personnel selection task (Unpublished doctoral dissertation). Kansas State University, Manhattan, KS.
Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231259.
Phelps, R. H. (1977). Expert livestock judgment: A descriptive analysis of the development of expertise (Unpublished doctoral dissertation). Kansas State University, Manhattan, KS.
Phelps, R. H., & Shanteau, J. (1978). Livestock judges: How much information can an expert use? Organizational Behavior and Human Performance, 21, 209219.
Rassafiani, M., Ziviani, J., Rodger, S., & Dalgleish, L. (2008). Identification of occupational therapy clinical expertise: Decision-making characteristics. Australian Occupational Therapy Journal, 15, 105115.
Schunn, C. D., & Anderson, J. R. (1999). The generality/specificity of expertise in scientific reasoning. Cognitive Science, 23, 337370.
Shanteau, J. (1992). Competence in experts: The role of task characteristics. Organizational Behavior and Human Decision Processes, 53, 252266.
Shanteau, J. (2001). What does it mean when experts disagree. In Salas, E. (Ed.), Linking expertise and naturalistic decision making (pp. 229244). Mahwah, NJ: Erlbaum.
Shanteau, J., Friel, B. M., Thomas, R. P., & Raacke, J. (2005). Development of expertise in a dynamic decision-making environment. In Betsch, T. (Ed.), The routines of decision making (pp. 251270). Mahwah, NJ: Erlbaum.
Shanteau, J., Weiss, D. J., Thomas, R., & Pounds, J. (2002). Performance-based assessment of expertise: How can you tell if someone is an expert? European Journal of Operational Research, 136, 253263.
Skånér, Y., Strender, L., & Bring, J. (1998). How do GPs use clinical information in the judgements of heart failure? Scandinavian Journal of Primary Health Care, 16, 95100.
Spearman, C. (1923). The nature of intelligence and the principles of cognition. London, England: Macmillan.
Steinman, J. (1977). Two out of three Ain't bad [Recorded by Meat Loaf]. On Bat out of Hell [Record]. New York, NY: Epic Records.
Sternberg, R. J. (1986). Identifying the gifted through IQ: Why a little bit of knowledge is a dangerous thing. Roeper Review, 8, 143157.
Sternberg, R. J., & Wagner, R. K. (1993). The g-ocentric view of intelligence and job performance is wrong. Current Directions in Psychological Science, 2, 15.
Taleb, N. N. (2004). Fooled by randomness: The hidden role of chance in life and in the markets. New York, NY: Random House.
Taleb, N. N. (2007). The black swan. New York, NY: Random House.
Tetlock, P. (2005). Expert political judgment. Princeton, NJ: Princeton University Press.
Thomas, R. P., Willem, B., Shanteau, J., Raacke, J., & Friel, B. (2001, 03). CWS applied to controllers in a high-fidelity simulation of ATC. Proceedings from the International Symposium on Aviation Psychology, Columbus, OH.
Van der Fels-Klerx, H. J., Cooke, R. M., Nauta, M. N., Goossens, L. H., & Havelaar, A. H. (2005). A structured expert judgment study for a model of Campylobacter transmission during broiler-chicken processing. Risk Analysis, 25, 109124.
Weiss, D. J. (2007). Extracting individual contributions to a team's performance. Teorie e Modelli, 12, 227236.
Weiss, D. J., Brennan, K., Thomas, R., Kirlik, A., & Miller, S. (2009). Criteria for performance evaluation. Judgment and Decision Making, 4, 164174.
Weiss, D. J., Edwards, W., & Shanteau, J. (2009). The measurement of behavior: Indices and standards. In Weiss, J. W., & Weiss, D. J. (Eds.), A science of decision making: The legacy of Ward Edwards (pp. 262268). New York, NY: Oxford University Press.
Weiss, D. J., & Shanteau, J. (2000, 04). Consensus: The hobgoblin of little minds. Paper presented at the Oklahoma-Kansas Judgment/Decision Making Conference, Oklahoma, OK.
Weiss, D. J., & Shanteau, J. (2003). Empirical assessment of expertise. Human Factors, 45, 104116.
Weiss, D. J., Shanteau, J., & Harries, P. (2006). People who judge people. Journal of Behavioral Decision Making, 19, 441454.
Williams, C. A., Haslam, R. A., & Weiss, D. J. (2008). The Cochran-Weiss-Shanteau performance index as an indicator of Upper Limb Disorder risk assessment expertise. Ergonomics, 51, 12191237.



Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed