Skip to main content Accessibility help
×
Home
Hostname: page-component-559fc8cf4f-67gxp Total loading time: 0.275 Render date: 2021-03-04T10:54:51.142Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": false, "newCiteModal": false, "newCitedByModal": true }

“Life Doesn't Happen at the Between-Person Level,” or a Cautionary Note on Generating Scientific Inferences Through Meta-Analyses

Published online by Cambridge University Press:  30 August 2017

Hannah L. Samuelson
Affiliation:
Department of Psychology, University of Maryland
Jessica R. Fernandez
Affiliation:
Department of Psychology, University of Maryland
James A. Grand
Affiliation:
Department of Psychology, University of Maryland
Corresponding
E-mail address:

Extract

The implicit philosophy for how research and practice in industrial and organizational (I-O) psychology has pursued inferences about our field's core phenomena has largely been based on a nomothetic, variable-based, and aggregate/“large-sample” ideal. As Tett, Hundley, and Christiansen (2017) expertly highlight, there are more insightful means for drawing inferences about the nature of such aggregate relationships based on meta-analytic techniques than the current practice in the organizational sciences. However, the motivating force behind our commentary has less to do with the issues raised by Tett et al. (2017) concerning the practice of using meta-analysis for purposes of validity generalization and more to do with the practice of using meta-analysis for purposes of scientific inference. Between-person philosophies in which the end-goal is to identify general conclusions that apply to the aggregate (cf. Hanges & Wang, 2012) have historically guided our scientific inferences and have supported the proliferation of meta-analytic techniques (including what Tett et al. describe as tertiary analyses based on such findings). These philosophies have led to a dearth of understanding at the within-person and social system levels—the levels at which most of our meaningful phenomena exist (e.g., Hamaker, 2012; Von Bertalanffy, 1950). Learning, performing, decision making, communicating, sense-making, feeling/expressing emotion: These are the concepts that drive the lived experiences of individuals both inside and outside of the workplace, and all are vulnerable to being misunderstood or misinterpreted by focusing only on aggregate evidence at the between-person level. Consequently, we wish to first supplement Tett et al.’s recommendations for drawing generalizability inferences in meta-analysis and suggest a “pre-emptive” question (i.e., Question 0) to the list of four they advance in their focal article.

Type
Commentaries
Copyright
Copyright © Society for Industrial and Organizational Psychology 2017 

Access options

Get access to the full version of this content by using one of the access options below.

References

Axelrod, R. (1997). Advancing the art of simulation in the social sciences. In Conte, R., Hegselmann, R., & Terna, P. (Eds.), Simulating social phenomena (pp. 2140). Berlin: Springer.CrossRefGoogle Scholar
Bechtel, W. (1988). Philosophy of science: An overview for cognitive science. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
Cronin, M. A., Weingart, L.R., & Todorova, G. (2011). Dynamics in groups: Are we there yet? The Academy of Management Annals, 5, 571612.CrossRefGoogle Scholar
Davis, J. P., Eisenhardt, K. M., & Bingham, C. B. (2007). Developing theory through simulation methods. Academy of Management Review, 32 (2), 480499.CrossRefGoogle Scholar
Grand, J. A., Braun, M. T., Kuljanin, G., Kozlowski, S. W., & Chao, G. T. (2016). The dynamics of team cognition: A process-oriented theory of knowledge emergence in teams. Journal of Applied Psychology, 101 (10), 13531385.CrossRefGoogle ScholarPubMed
Hamaker, E. L. (2012). Why researchers should think “within-person”: A paradigmatic rationale. In Mehl, M. R. & Conner, T. S. (Eds.), Handbook of research methods for studying daily life (pp. 4361). New York: Guilford.Google Scholar
Hanges, P. J., & Wang, M. (2012). Seeking the holy grail in organizational science: Uncovering causality through research design. In Kozlowski, S. W. J. (Ed.), The Oxford handbook of organizational psychology (Vol. 1, pp. 79116). New York: Oxford University Press.Google Scholar
Harrison, J. R., Lin, Z., Carroll, G. R., & Carley, K. M. (2007). Simulation modeling in organizational and management research. Academy of Management Review, 32 (4), 12291245.CrossRefGoogle Scholar
Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting for error and bias in research findings. Thousand Oaks, CA: Sage.CrossRefGoogle Scholar
Kozlowski, S. W., Chao, G. T., Grand, J. A., Braun, M. T., & Kuljanin, G. (2013). Advancing multilevel research design: Capturing the dynamics of emergence. Organizational Research Methods, 16 (4), 581615.CrossRefGoogle Scholar
Kozlowski, S. W., Chao, G. T., Grand, J. A., Braun, M. T., & Kuljanin, G. (2016). Capturing the multilevel dynamics of emergence: Computational modeling, simulation, and virtual experimentation. Organizational Psychology Review, 6 (1), 333.CrossRefGoogle Scholar
McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79, 599616.CrossRefGoogle Scholar
Schmidt, A. M., & DeShon, R. P. (2007). What to do? The effects of discrepancies, incentives, and time on dynamic goal prioritization. Journal of Applied Psychology, 92, 928941.CrossRefGoogle Scholar
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124 (2), 262274.CrossRefGoogle Scholar
Tett, R. P., Hundley, N. A., & Christiansen, N. D. (2017). Meta-analysis and the myth of generalizability. Industrial and Organizational Psychology: Perspectives on Science and Practice, 10 (3), 421–456.CrossRefGoogle Scholar
Vancouver, J. B., Thompson, C. M., Tischner, E. C., & Putka, D. J. (2002). Two studies examining the negative effect of self-efficacy on performance. Journal of Applied Psychology, 87, 506516.CrossRefGoogle ScholarPubMed
Vancouver, J. B., Thompson, C. M., & Williams, A. A. (2001). The changing signs in the relationships among self-efficacy, personal goals, and performance. Journal of Applied Psychology, 86, 605620.CrossRefGoogle ScholarPubMed
Vancouver, J. B., Weinhardt, J. M., & Schmidt, A. M. (2010). A formal, computational theory of multiple-goal pursuit: Integrating goal-choice and goal-striving processes. Journal of Applied Psychology, 95 (6), 9851008.CrossRefGoogle ScholarPubMed
Von Bertalanffy, L. (1950). An outline of general systems theory. British Journal of for the Philosophy of Science, 1, 136164.Google Scholar
Wiesner, W. H., & Cronshaw, S. F. (1988). A meta‐analytic investigation of the impact of interview format and degree of structure on the validity of the employment interview. Journal of Occupational Psychology, 61, 275290.CrossRefGoogle Scholar

Full text views

Full text views reflects PDF downloads, PDFs sent to Google Drive, Dropbox and Kindle and HTML full text views.

Total number of HTML views: 10
Total number of PDF views: 114 *
View data table for this chart

* Views captured on Cambridge Core between 30th August 2017 - 4th March 2021. This data will be updated every 24 hours.

Send article to Kindle

To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

“Life Doesn't Happen at the Between-Person Level,” or a Cautionary Note on Generating Scientific Inferences Through Meta-Analyses
Available formats
×

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

“Life Doesn't Happen at the Between-Person Level,” or a Cautionary Note on Generating Scientific Inferences Through Meta-Analyses
Available formats
×

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

“Life Doesn't Happen at the Between-Person Level,” or a Cautionary Note on Generating Scientific Inferences Through Meta-Analyses
Available formats
×
×

Reply to: Submit a response


Your details


Conflicting interests

Do you have any conflicting interests? *