Skip to main content Accessibility help
×
×
Home

“Life Doesn't Happen at the Between-Person Level,” or a Cautionary Note on Generating Scientific Inferences Through Meta-Analyses

  • Hannah L. Samuelson (a1), Jessica R. Fernandez (a1) and James A. Grand (a1)

Extract

The implicit philosophy for how research and practice in industrial and organizational (I-O) psychology has pursued inferences about our field's core phenomena has largely been based on a nomothetic, variable-based, and aggregate/“large-sample” ideal. As Tett, Hundley, and Christiansen (2017) expertly highlight, there are more insightful means for drawing inferences about the nature of such aggregate relationships based on meta-analytic techniques than the current practice in the organizational sciences. However, the motivating force behind our commentary has less to do with the issues raised by Tett et al. (2017) concerning the practice of using meta-analysis for purposes of validity generalization and more to do with the practice of using meta-analysis for purposes of scientific inference. Between-person philosophies in which the end-goal is to identify general conclusions that apply to the aggregate (cf. Hanges & Wang, 2012) have historically guided our scientific inferences and have supported the proliferation of meta-analytic techniques (including what Tett et al. describe as tertiary analyses based on such findings). These philosophies have led to a dearth of understanding at the within-person and social system levels—the levels at which most of our meaningful phenomena exist (e.g., Hamaker, 2012; Von Bertalanffy, 1950). Learning, performing, decision making, communicating, sense-making, feeling/expressing emotion: These are the concepts that drive the lived experiences of individuals both inside and outside of the workplace, and all are vulnerable to being misunderstood or misinterpreted by focusing only on aggregate evidence at the between-person level. Consequently, we wish to first supplement Tett et al.’s recommendations for drawing generalizability inferences in meta-analysis and suggest a “pre-emptive” question (i.e., Question 0) to the list of four they advance in their focal article.

Copyright

Corresponding author

Correspondence concerning this article should be addressed to Hannah L. Samuelson, Department of Psychology, 4094 Campus Dr., 3143 Biology-Psychology Building, University of Maryland, College Park, MD 20742. E-mail: hsamuels@umd.edu

References

Hide All
Axelrod, R. (1997). Advancing the art of simulation in the social sciences. In Conte, R., Hegselmann, R., & Terna, P. (Eds.), Simulating social phenomena (pp. 2140). Berlin: Springer.
Bechtel, W. (1988). Philosophy of science: An overview for cognitive science. Hillsdale, NJ: Lawrence Erlbaum.
Cronin, M. A., Weingart, L.R., & Todorova, G. (2011). Dynamics in groups: Are we there yet? The Academy of Management Annals, 5, 571612.
Davis, J. P., Eisenhardt, K. M., & Bingham, C. B. (2007). Developing theory through simulation methods. Academy of Management Review, 32 (2), 480499.
Grand, J. A., Braun, M. T., Kuljanin, G., Kozlowski, S. W., & Chao, G. T. (2016). The dynamics of team cognition: A process-oriented theory of knowledge emergence in teams. Journal of Applied Psychology, 101 (10), 13531385.
Hamaker, E. L. (2012). Why researchers should think “within-person”: A paradigmatic rationale. In Mehl, M. R. & Conner, T. S. (Eds.), Handbook of research methods for studying daily life (pp. 4361). New York: Guilford.
Hanges, P. J., & Wang, M. (2012). Seeking the holy grail in organizational science: Uncovering causality through research design. In Kozlowski, S. W. J. (Ed.), The Oxford handbook of organizational psychology (Vol. 1, pp. 79116). New York: Oxford University Press.
Harrison, J. R., Lin, Z., Carroll, G. R., & Carley, K. M. (2007). Simulation modeling in organizational and management research. Academy of Management Review, 32 (4), 12291245.
Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting for error and bias in research findings. Thousand Oaks, CA: Sage.
Kozlowski, S. W., Chao, G. T., Grand, J. A., Braun, M. T., & Kuljanin, G. (2013). Advancing multilevel research design: Capturing the dynamics of emergence. Organizational Research Methods, 16 (4), 581615.
Kozlowski, S. W., Chao, G. T., Grand, J. A., Braun, M. T., & Kuljanin, G. (2016). Capturing the multilevel dynamics of emergence: Computational modeling, simulation, and virtual experimentation. Organizational Psychology Review, 6 (1), 333.
McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79, 599616.
Schmidt, A. M., & DeShon, R. P. (2007). What to do? The effects of discrepancies, incentives, and time on dynamic goal prioritization. Journal of Applied Psychology, 92, 928941.
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124 (2), 262274.
Tett, R. P., Hundley, N. A., & Christiansen, N. D. (2017). Meta-analysis and the myth of generalizability. Industrial and Organizational Psychology: Perspectives on Science and Practice, 10 (3), 421–456.
Vancouver, J. B., Thompson, C. M., Tischner, E. C., & Putka, D. J. (2002). Two studies examining the negative effect of self-efficacy on performance. Journal of Applied Psychology, 87, 506516.
Vancouver, J. B., Thompson, C. M., & Williams, A. A. (2001). The changing signs in the relationships among self-efficacy, personal goals, and performance. Journal of Applied Psychology, 86, 605620.
Vancouver, J. B., Weinhardt, J. M., & Schmidt, A. M. (2010). A formal, computational theory of multiple-goal pursuit: Integrating goal-choice and goal-striving processes. Journal of Applied Psychology, 95 (6), 9851008.
Von Bertalanffy, L. (1950). An outline of general systems theory. British Journal of for the Philosophy of Science, 1, 136164.
Wiesner, W. H., & Cronshaw, S. F. (1988). A meta‐analytic investigation of the impact of interview format and degree of structure on the validity of the employment interview. Journal of Occupational Psychology, 61, 275290.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Industrial and Organizational Psychology
  • ISSN: 1754-9426
  • EISSN: 1754-9434
  • URL: /core/journals/industrial-and-organizational-psychology
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed