Skip to main content Accessibility help
×
Home

75 Years After Likert: Thurstone Was Right!

  • Fritz Drasgow (a1), Oleksandr S. Chernyshenko (a2) and Stephen Stark (a3)

Abstract

For over three-quarters of a century researchers and practitioners have analyzed rating scale data using methods that assume a dominance response process wherein an individual high on the trait assessed is assumed to answer positively with high probability. This approach derives from Likert's famous 1932 approach to the development and analysis of rating scales. In this paper, we argue that Likert scaling and related methods are misguided. Instead, we propose that methods that have evolved from Thurstone (1927, 1928, 1929) scaling provide a better representation of the choice process underlying rating scale judgments. These methods hypothesize an ideal point response process where the probability of endorsement is assumed to be directly related to the proximity of the statement to the individual's standing on the assessed trait. We review some research showing the superiority of ideal point methods for personality assessment and then describe several settings in which ideal point methods should provide tangible improvements over traditional approaches to assessment.

Copyright

Corresponding author

E-mail: fdrasgow@uiuc.edu, Address: Department of Psychology, University of Illinois, 603 E. Daniel Street, Champaign, IL 61820

References

Hide All
Ashton, M. C., & Lee, K. (2007). Empirical, theoretical, and practical advantages of the HEXACO model of personality structure. Personality and Social Psychology Review, 11, 150166.
Borman, W. C., Buck, D. E., Hanson, M. A., Motowidlo, S. J., Stark, S., & Drasgow, F. (2001). An examination of the comparative reliability, validity, and accuracy of performance ratings made using computerized adaptive rating scales. Journal of Applied Psychology, 86, 965973.
Chernyshenko, O. S., Stark, S., Chan, K.-Y., Drasgow, F., & Williams, B. (2001). Fitting item response theory models to two personality inventories: Issues and insights. Multivariate Behavioral Research, 36, 523562.
Chernyshenko, O. S., Stark, S., Drasgow, F., & Roberts, B. W. (2007). Constructing personality scales under the assumption of an ideal point response process: Toward increasing the flexibility of personality measures. Psychological Assessment, 19, 88106.
Chernyshenko, O. S., Stark, S., Prewett, M. S., Gray, A. A., Stilson, F. R., & Tuttle, M. D. (2009). Normative scoring of multidimensional pairwise preference personality scales using IRT: Empirical comparisons with other formats. Human Performance, 22, 123.
Chernyshenko, O. S., Stark, S., & Williams, A. (2009). Latent trait theory approach to measuring person-organization fit: Conceptual rationale and empirical evaluation. International Journal of Testing, 9, 358380.
Christiansen, N. D., Burns, G. N., & Montgomery, G. E. (2005). Reconsidering forced-choice item formats for applicant personality assessment. Human Performance, 18, 267307.
Conn, S., & Rieke, M. L. (Eds.) (1994). The 16PF fifth edition technical manual. Champaign, IL: Institute for Personality and Ability Testing.
Coombs, C. H. (1964). A theory of data. New York: Wiley.
Davison, M. L. (1977). On a metric, unidimensional unfolding model for attitudinal and developmental data. Psychometrika, 42, 523548.
Drasgow, F., Chernyshenko, O. S., & Stark, S. (2009). Test theory and personality measurement. In Butcher, J. N. (Ed.), Oxford handbook of personality assessment (pp. 5980). New York: Oxford University Press.
Drasgow, F., Chernyshenko, O. S., & Stark, S. (2010). Tailored Adaptive Personality Assessment System (TAPAS). Urbana, IL: Authors.
Drasgow, F., Levine, M. V., Tsien, S., Williams, B., & Mead, A. D. (1995). Fitting polytomous item response theory models to multiple-choice tests. Applied Psychological Measurement, 19, 143165.
Edwards, A. L. (1954). Personal preference schedule. New York: Psychological Corporation.
Edwards, J. R. (1994). The study of congruence in organizational behavior research: Critique and a proposed alternative. Organizational Behavior and Human Decision Processes, 58, 51100 (erratum, 58, 323325).
Hough, L. M., Eaton, N. L., Dunnette, M. D., Kamp, J. D., & McCloy, R. A. (1990). Criterion-related validities of personality constructs and the effect of response distortion on those validities [Monograph]. Journal of Applied Psychology, 75, 581595.
Jackson, D. N., Wroblewski, V. R., & Ashton, M. C. (2000). The impact of faking on employment tests: Does forced-choice offer a solution? Human Performance, 13, 371388.
Levine, M. V. (1984). An introduction to multilinear formula score theory (Personnel and Training Research Programs, Office of Naval Research, Measurement Series No. 84-4). Arlington, VA: Personnel and Training Research Programs.
Lewin, K. (1951). Field theory in social science; selected theoretical papers. D. Cartwright (Ed.), New York: Harper & Row.
Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22, 155.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
McCloy, R. A., Heggestad, E. D., & Reeve, C. L. (2005). A silk purse from the sow's ear: Retrieving normative information from multidimensional forced-choice items. Organizational Research Methods, 8, 222248.
Meade, A. W. (2004). Psychometric problems and issues involved with creating and using ipsative measures for selection. Journal of Occupational and Organizational Psychology, 77, 531551.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. New York: McGraw-Hill.
Reise, S. P., & Waller, N. G. (1990). Fitting the two-parameter model to personality data. Applied Psychological Measurement, 14, 4558.
Roberts, J. S. (2001). GGUM2000: Estimation of parameters in the generalized graded unfolding model. Applied Psychological Measurement, 25, 38.
Roberts, J. S., Donoghue, J. R., & Laughlin, J. E. (2000). A general item response theory model for unfolding unidimensional polytomous responses. Applied Psychological Measurement, 24, 332.
Rounds, J. B., Henly, G. A., Dawis, R. V., Lofquist, L. H., & Weiss, D. J. (1981). Manual for the Minnesota Importance Questionnaire. Minneapolis, MN: University of Minnesota, Vocational Psychology Research.
Stark, S. (2002). A new IRT approach to test construction and scoring designed to reduce the effects of faking in personality assessment. Unpublished doctoral dissertation, University of Illinois at Urbana–Champaign.
Stark, S., & Chernyshenko, O. S. (2007, June). Adaptive testing with the multi-unidimensional pairwise preference (MUPP) model. Paper presented at the 2007 Graduate Management Admissions Council conference on Computerized Adaptive Testing. Minneapolis, MN.
Stark, S., Chernyshenko, O. S., Chan, K.-Y., Lee, W. C., & Drasgow, F. (2001). Effects of the testing situation on item responding: Cause for concern. Journal of Applied Psychology, 86, 943953.
Stark, S., Chernyshenko, O. S., & Drasgow, F. (2005). An IRT approach to constructing and scoring pairwise preference items involving stimuli on different dimensions: The multi-unidimensional pairwise-preference model. Applied Psychological Measurement, 29, 184203.
Stark, S., Chernyshenko, O. S., Drasgow, F., & Williams, B. A. (2006). Examining assumptions about item responding in personality assessment: Should ideal point methods be considered for scale development and scoring? Journal of Applied Psychology, 91, 2539.
Stark, S., & Drasgow, F. (2002). An EM approach to parameter estimation for the Zinnes and Griggs paired comparison ideal point IRT model. Applied Psychological Measurement, 26, 208227.
Tay, L., Drasgow, F., Rounds, J., & Williams, B. A. (2009). Fitting measurement models to vocational interest data: Are dominance models ideal? Journal of Applied Psychology, 94, 12871304.
Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review, 34, 273286.
Thurstone, L. L. (1928). Attitudes can be measured. The American Journal of Sociology, 33, 529554.
Thurstone, L. L. (1929). Theory of attitude measurement. Psychological Review, 36, 222241.
Vasilopoulos, N. L., Cucina, J. M., Dyomina, N. V., Morewitz, C. L., & Reilly, R. R. (2006). Forced-choice personality tests: A measure of personality and cognitive ability? Human Performance, 19, 175199.
White, L. A., & Young, M. C. (1998, August). Development and validation of the Assessment of Individual Motivation (AIM). Paper presented at the 106th Annual Convention of the American Psychological Association, San Francisco, CA.
Zinnes, J. L., & Griggs, R. A. (1974). Probabilistic, multidimensional unfolding analysis. Psychometrika, 39, 327350.

Related content

Powered by UNSILO

75 Years After Likert: Thurstone Was Right!

  • Fritz Drasgow (a1), Oleksandr S. Chernyshenko (a2) and Stephen Stark (a3)

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.