Skip to main content Accessibility help
×
Home

Can Synthetic Validity Methods Achieve Discriminant Validity?

  • Frank L. Schmidt (a1) and In-Sue Oh (a2)

Extract

Our focus is on the difficulties that synthetic validity encounters in attempting to achieve discriminant validity and the implications of these difficulties. Johnson et al. (2010) acknowledge the potential problems involved in attaining discriminant validity in synthetic validity. For example, they report that Peterson et al. (2001), Johnson (2007), and other synthetic validity studies report failure to achieve discriminant validity. What this failure means is that a synthetic validity equation developed to predict validity for Job A does as well in predicting validity for Jobs B, C, D, and so forth as it does for Job A. Johnson et al. then go on to propose that this problem might be overcome by careful attention to both the criterion and predictor sides of synthetic validity. We question whether their proposals can be made to work.

Copyright

Corresponding author

E-mail: frank-schmidt@uiowa.edu, Address: Department of Management and Organizations, Tippie College of Business, University of Iowa, W236 John Pappajohn Business Building, University of Iowa, IA 52242-1994

References

Hide All
Bertua, C., Anderson, N., & Salgado, J. F. (2005). The predictive validity of cognitive ability tests: A U.K. meta-analysis. Journal of Occupational and Organizational Psychology, 78, 387409.
Cattin, P. (1980). Estimation of the predictive power of a regression model. Journal of Applied Psychology, 65, 401414.
Dudley, N., Orvis, K., Lebiecki, J., & Cortina, J. (2006). A meta-analytic investigation of conscientiousness in the prediction of job performance: Examining the intercorrelations and the incremental validity of narrow traits. Journal of Applied Psychology, 91, 4057.
Gutenberg, R. L., Arvey, R. D., Osburn, H. G., & Jeanneret, P. R. (1983). The moderating effects of decision-making/information processing job dimensions on test validities. Journal of Applied Psychology, 68, 602608.
Hoffman, C. C., Holden, L. M., & Gale, K. (2000). So many jobs, so little “N”: Applying expanded validation models to support generalization of cognitive test validity. Personnel Psychology, 53, 955991.
Hoffman, C. C., & McPhail, S. M. (1998). Exploring options for supporting test use in situations precluding local validation. Personnel Psychology, 51, 9871003.
Hulsheger, U. R., Maier, G. W., & Stumpp, T. (2007). Validity of general mental ability for the prediction of job performance and training success in Germany: A meta-analysis. International Journal of Selection and Assessment, 15, 318.
Hunter, J. E. (1983). Validity generalization for 12,000 jobs: An application of synthetic validity and validity generalization to the General Aptitude Test Battery (GATB). Washington, DC: U.S. Department of Labor, Employment Service.
Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.) Newbury Park, CA: Sage.
Hunter, J. E., Schmidt, F. L., & Le, H. (2006). Implications of direct and indirect range restriction for meta-analysis methods and findings. Journal of Applied Psychology, 91, 594612.
Johnson, J. W. (2007). Synthetic validity: A technique of use (finally). In McPhail, S. M.(Ed.), Alternative validation strategies: Developing new and leveraging existing validity evidence (pp. 122158). San Francisco: Jossey-Bass.
Johnson, J. W., Steel, P., Scherbaum, C. A., Hoffman, C. C., Richard Jeanneret, P., & Foster, J. (2010). Validation is like motor oil: Synthetic is better. Industrial and Organizational Psychology: Perspectives on Science and Practice, 3, 305328.
King, L. M., Hunter, J. E., & Schmidt, F. L. (1980). Halo in a multidimensional forced choice performance evaluation scale. Journal of Applied Psychology, 65, 507516.
Morris, D. C., Hoffman, C. C., & Shultz, K. S. (2003, April). A comparison of job components validity estimates to meta-analytic validity estimates. Poster presented at the 18th Annual Conference of the Society for Industrial and Organizational Psychology, Orlando, FL.
Mount, M. K., Judge, T. A., Scullen, S. E., Sytsma, M. R., & Hezlett, S. A. (1998). Trait, rater, and level effects in 360-degree performance ratings. Personnel Psychology, 51, 557576.
Pearlman, K., Schmidt, F. L., & Hunter, J. E. (1980). Validity generalization results for tests used to predict job proficiency and training success in clerical occupations. Journal of Applied Psychology, 65, 373406.
Peterson, N. G., Wise, L. L., Arabian, J., & Hoffman, R. G. (2001). Synthetic validation and validity generalization: When empirical validation is not possible. In Campbell, J. P. & Knapp, D. J. (Eds.), Exploring the limits of personnel selection and classification (pp. 411451). Mahwah, NJ: Erlbaum.
Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., & De Fruyt, F. (2003). International validity generalization of GMA and cognitive abilities: A European community meta-analysis. Personnel Psychology, 56, 573605.
Schmidt, F. L., & Hunter, J. E. (1996). Measurement error in psychological research: Lessons from 26 research scenarios. Psychological Methods, 1, 199223.
Schmidt, F. L., Hunter, J. E., & Pearlman, K. (1981). Task difference and validity of aptitude tests in selection: A red herring. Journal of Applied Psychology, 66, 166185.
Schmidt, F. L., Law, K., Hunter, J. E., Rothstein, H. R., Pearlman, K., & McDaniel, M. (1993). Refinements in validity generalization methods: Implications for the situational specificity hypothesis. Journal of Applied Psychology, 78, 313.
Schmidt, F. L., Shaffer, J. A., & Oh, I.-S. (2008). Increased accuracy of range restriction corrections: Implications for the role of personality and general mental ability in job and training performance. Personnel Psychology, 61, 827868.
Steel, P., & Kammeyer-Mueller, J. (2009). Using a meta-analytic perspective to enhance job component validation. Personnel Psychology, 62, 533552.
Verive, J. M., & McDaniel, M. A. (1996). Short-term memory tests in personnel selection: Low adverse impact and high validity. Intelligence, 23, 1532.
Viswesvaran, C., Schmidt, F. L., & Ones, D. S. (2005). Is there a general factor in ratings of job performance? A meta-analytic framework for disentangling substantive and error influences. Journal of Applied Psychology, 90, 108131.

Related content

Powered by UNSILO

Can Synthetic Validity Methods Achieve Discriminant Validity?

  • Frank L. Schmidt (a1) and In-Sue Oh (a2)

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.