Hostname: page-component-8448b6f56d-gtxcr Total loading time: 0 Render date: 2024-04-23T07:43:23.501Z Has data issue: false hasContentIssue false
  • English
  • Français

When Correcting for Unreliability of Job Performance Ratings, the Best Estimate Is Still .52

Published online by Cambridge University Press:  10 April 2015

Winny Shen ,
Jeffrey M. Cucina ,
Philip T. Walmsley  and
Benjamin K. Seltzer
Winny Shen*
Affiliation:
University of South Florida
Jeffrey M. Cucina
Affiliation:
U.S. Customs and Border Protection
Philip T. Walmsley
Affiliation:
U.S. Office of Personnel Management
Benjamin K. Seltzer
Affiliation:
Washington & Jefferson College
*
E-mail: winny.shen@uwaterloo.ca, Address: University of Waterloo, 200 University Avenue West, Waterloo, ON, Canada N2L 3G1
Get access Rights & Permissions [Opens in a new window]

Abstract

An abstract is not available for this content so a preview has been provided. Please use the Get access link above for information on how to access this content.
Image of the first page of this content. For PDF version, please use the ‘Save PDF’ preceeding this image.'
Type
Commentaries
Information
Industrial and Organizational Psychology , Volume 7 , Issue 4 , December 2014 , pp. 519 - 524
Copyright
Copyright © Society for Industrial and Organizational Psychology 2014

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Arthur, W., & Villado, A. J. (2008). The importance of distinguishing between constructs and methods when comparing predictors in personnel selection and research. Journal of Applied Psychology, 93, 435–442.Google Scholar
Binning, J. F., & Barrett, G. V. (1989). Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology, 74(3), 478–494.CrossRefGoogle Scholar
Chiaburu, D. S., & Harrison, D. A. (2008). Do peer make the place? Conceptual synthesis and meta-analysis of coworker effects on perceptions, attitudes, OCBs, and performance. Journal of Applied Psychology, 93, 1082–1103.Google Scholar
Conway, J. M., & Huffcutt, A. I. (1997). Psychometric properties of multisource performance ratings: A meta-analysis of subordinate, supervisor, peer, and self-ratings. Human Performance, 10(4), 331–360.Google Scholar
Highhouse, S. (2008). Stubborn reliance on intuition and subjectivity in employee selection. Industrial and Organizational Psychology: Perspectives on Science and Practice, 1, 333–342.Google Scholar
Huffcutt, A. I., Conway, J. M., Roth, P. L., & Stone, N. J. (2001). Identification and meta-analytic assessment of psychological constructs measured in employment interviews. Journal of Applied Psychology, 86, 897–913.CrossRefGoogle ScholarPubMed
Hunter, J. E. (1983). Test validation for 12,000 jobs: An application of job classification and validity generalization to General Aptitude Test Battery (U.S. Employment Service Test Research Report No. 45). Washington, DC: U.S. Department of Labor.Google Scholar
Hutchinson, D., Shen, W., Telford, B., Andel, S., Jang, S., & Ramsay, S. (2014, May). Personality and justice perceptions: An updated meta-analysis. Paper presented at the 29th Annual Conference of the Society for Industrial and Organizational Psychology, Honolulu, HI.Google Scholar
Jensen, A. R. (1998). The g factor: The science of mental ability. Westport, CT: Praeger.Google Scholar
King, L. M., Hunter, J. E., & Schmidt, F. L. (1980). Halo in a multidimensional, forced-choice performance evaluation scale. Journal of Applied Psychology, 65, 507–516.Google Scholar
LeBreton, J. M., Scherer, K. T., & James, L. R. (2014). Corrections for criterion reliability in validity generalization: A false prophet in a land of suspended judgment. Industrial and Organizational Psychology: Perspectives on Science and Practice, 7(4), 478–500.Google Scholar
Meehl, P. E. (1990). Why summaries of research on psychological theories are often uninterpretable. Psychological Reports, 66, 195–244.Google Scholar
Murphy, K. R., & DeShon, R. (2000). Interrater correlations do not estimate the reliability of job performance ratings. Personnel Psychology, 53, 873–900.CrossRefGoogle Scholar
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. New York, NY: McGraw-Hill.Google Scholar
Ones, D. S., & Viswesvaran, C. (2001). Integrity tests and other criterion-focused occupational personality scales (COPS) used in personnel selection. International Journal of Selection and Assessment, 9, 31–39.Google Scholar
Paterson, T., Harms, P., & Crede, M. (2012, April). The meta of all metas: 30 years of meta-analysis reviewed. Paper presented at the 27th Annual Conference of the Society for Industrial and Organizational Psychology, San Diego, CA.CrossRefGoogle Scholar
Putka, D. J., & Sackett, P. R. (2010). Reliability and validity. In Farr, J. L., & Tippins, N. T. (Eds.), Handbook of employee selection (pp. 9–49). New York, NY: Routledge.Google Scholar
Rothstein, H. R. (1990). Interrater reliability of job performance ratings: Growth to asymptote level with increasing opportunity to observe. Journal of Applied Psychology, 75, 322–327.Google Scholar
Sackett, P. R., Borneman, M. J., & Connelly, B. S. (2009). High stakes testing in higher education and employment: Appraising the evidence for validity and fairness. American Psychologist, 63, 215–227.Google Scholar
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262–274.Google Scholar
Schmidt, F. L., Viswesvaran, C., & Ones, D. S. (2000). Reliability is not validity and validity is not reliability. Personnel Psychology, 53, 901–912.Google Scholar
Scullen, S. E., Mount, M. K., & Sytsma, M. R. (1996, April). Self, peer, direct report, and boss ratings of managers' performance. Paper presented at the 11th Annual Conference of the Society for Industrial and Organizational Psychology, San Diego, CA.Google Scholar
Shen, W., Sackett, P. R., Kuncel, N. R., Beatty, A. S., Rigdon, J. L., & Kiger, T. B. (2012). All validities are not created equal: Determinants of variation in SAT validity across schools. Applied Measurement in Education, 25, 197–219.Google Scholar
Spearman, C. (1927). The abilities of man: Their nature and measurement. New York, NY: Macmillan.Google Scholar
Viswesvaran, C., Ones, D. S., & Schmidt, F. L. (1996). Comparative analysis of the reliability of job performance ratings. Journal of Applied Psychology, 81(5), 557–574.Google Scholar