Hostname: page-component-78c5997874-94fs2 Total loading time: 0 Render date: 2024-11-17T21:23:55.286Z Has data issue: false hasContentIssue false

Enabling practical research for the benefit of organizations and society

Published online by Cambridge University Press:  11 November 2020

Jennifer L. Geimer*
Affiliation:
SHL
Richard N. Landers
Affiliation:
University of Minnesota Twin Cities
Emily G. Solberg
Affiliation:
SHL
*
*Corresponding author. Email: Jennifer.Geimer@shl.com
Rights & Permissions [Opens in a new window]

Abstract

Type
Commentaries
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
© The Author(s), 2020. Published by Cambridge University Press on behalf of the Society for Industrial and Organizational Psychology.

Highhouse et al. (Reference Highhouse, Zickar and Melick2020) concluded that “the research-practice gap may not be as severe as is often claimed” (p. 12) because only a small proportion of practitioners who responded indicated that the flagship industrial-organizational (I-O) psychology journals (Journal of Applied Psychology, Personnel Psychology) were “rarely of interest.” However, the authors acknowledged that participation in the study likely reflected some degree of interest in journals. Upon review, the sample appears to be biased toward academics and includes a relatively small proportion of practitioners; its composition was 23% practitioner, whereas the broader composition of the membership of the Society for Industrial and Organizational Psychology (SIOP), excluding student affiliates, is currently 72% practitioner (J. Tegge, personal communication, February 24, 2020), suggesting that interest in responding to the survey was not distributed randomly by practitioner status. Additionally, a sample size of 128 would represent approximately 3% of the current population of self-reported practitioners in the SIOP membership database. Thus, we suspect that nonrandom sampling caused missingness not at random, biasing practitioner estimates toward practitioners predisposed to finding value in journals. Given our own understanding of and interactions with the practitioner community, we contend that the disproportionately high number of practitioners who declined to complete the survey relative to academic respondents did so because they generally did not find journals relevant to their work whatsoever. For such practitioners, responding to a survey about journals would likely be perceived as a waste of time. Given this, we concluded that the estimates for practitioners are likely biased and nonrepresentative.

Perhaps as further evidence of this bias, a common theme within the qualitative results was the persistence of the gap by respondents that appeared to be practitioners describing an ongoing need to better integrate research with practice and cautioning against publishing evidence of “minute differences” that are irrelevant to real-world issues. Some of the participants even explicitly noted that they had stopped reading scholarly journals due to their lack of relevance and instead sought information from nonjournal sources or trade publications.

If such a bias exists in these data, it suggests that the field of I-O psychology is in desperate need of changes to its journal publication system to better integrate science and practice in a mutually beneficial way. This was the foundation of strength that I-O psychology was built upon. Thus, in this commentary, we provide our perspective, from two practitioners and one academic, on changes that could be realistically made to journal policies within the I-O psychology community to (1) bridge the science-practice gap and (2) make research findings more accessible to practitioners and society more broadly. Some changes like these have already been made by a handful of forward-thinking I-O psychology editors, but much remains to be done to encourage practical, actionable research in I-O psychology so that it can be used to the benefit of organizations and society.

Bridging the science-practice gap with changes to journal policies

Journal editors have near-complete control, pending only publisher approval, to decide on the direction and content of their journals. As Aguinis and Vaschetto (Reference Aguinis and Vaschetto2011) asserted, editorial actions and decisions affect a broad range of stakeholders ranging from authors, reviewers, and editorial board members to professional associations, universities, and society. It is an extreme amount of power to direct the field that is held by a relatively small set of people. As such, we argue that changes to journal practices have the greatest potential to be realistically implemented and to also make a lasting impact on the science-practice gap. Our suggestions include increased involvement of practitioners in the editorial and review processes, revised reviewer guidelines to consider scientific contributions more broadly than theory expansion alone, and new requirements for authors to articulate practical contribution in their manuscripts and abstracts.

Actively include practitioners in the editorial/review process

As noted in the focal article, most prestigious I-O psychology journals are edited by business school academics (Highhouse et al., Reference Highhouse, Zickar and Melick2020). This is likely part of the reason for the shift toward a focus on management or organizational behavior topics in prominent I-O psychology journals over time. However, what was not noted in the focal article is that most journals, particularly those of high prestige, operate with editorial boards that are nearly devoid of practitioners. For example, all of the Journal of Applied Psychology’s editors and associate editors are academics, and out of 264 contributing editors, only one has a nonacademic affiliation. Similarly, Personnel Psychology has only one practitioner on the editorial board, and the only other practitioners officially affiliated with the journal serve on its book review advisory panel. Although we recognize that some academic scholars on these editorial boards likely engage in applied pursuits on a part-time basis, the lack of representation of practitioners in making recommendations and decisions on the content of journals from a field of study that was created and grew because of its practical value is staggering. Current journal editors can easily change this by inviting more practitioners to serve on editorial boards or as reviewers. For example, in his role as an associate editor at the International Journal of Selection and Assessment, the second author of this commentary makes it a standard practice to invite practitioners as reviewers, with a goal of ensuring that there is at least one practitioner and one academic reviewer for each article. By including both perspectives, reviewers can better address the needs of both science and practice in their feedback to authors.

To achieve this objective, editors must be able to find practitioner reviewers. We recognize that it can be a challenge for journal editors to identify and recruit practitioners who are willing and have the expertise to conduct manuscript reviews for scholarly journals, and we attribute this to a complex cyclical challenge. Specifically, reviewers tend to be sourced from the personal networks of editors, from the author lists of previously published papers relevant to papers submitted, and from editorial boards. Because editors tend to be academics, their personal networks tend to be academic. Because relatively few practitioners publish, practitioner names appear less often in author lists. And because most practitioners have limited or no reviewing history, they are less likely to be added to editorial boards. Consequently, journal editors need to consider alternative recruiting sources such as referrals from other practitioners who do review manuscripts, presenters at I-O psychology conferences, and local networking events.

To address the more systemic problem, we recommend taking steps to remove barriers to practitioner participation in publishing and reviewing. In general, practitioners are not incentivized to engage in these activities and may find little value in doing so. As a first step, if you are a practitioner, encourage your employer to allocate work time and financial support toward publication. This should not be considered organizational charity; publishing scientific research can bring a return on investment by promoting visibility of the business and building marketplace credibility. Cramer (Reference Cramer2007) found that practitioners who received support from their workplaces were better able to publish than those who did not receive support. We further encourage increased dialogue among editors, reviewers, and prospective practitioner-reviewers to increase awareness and the professional value of serving as a reviewer, author, or editor.

Changes in guidance for reviewers

We agree with Highhouse et al. (Reference Highhouse, Zickar and Melick2020) that there is an overreliance on theory in the most prestigious I-O psychology journals and that this trend has become overwhelming in recent years, as is reflected by journal policies emphasizing or requiring theoretical contributions (e.g., Cucina & Moriarty, Reference Cucina and Moriarty2015; Landers, Reference Landers and Landers2019). To address this issue, we echo Campbell and Wilmot’s (Reference Campbell, Wilmot, Ones, Anderson, Viswesvaran and Sinangil2018) call to change editorial and review policies for journals to remove the requirement to make a theoretical contribution or to build or test new theories. If the nontheoretical research has significant practical value and leads to interesting theoretical questions that can be further explored, this should be considered good science (Landers, Reference Landers and Landers2019).

We further recommend that editors provide guidance to reviewers to consider a practical contribution as part of their rating. The Journal of Business and Psychology is a good model of this practice, as one of the aims and scopes of the journal is the advancement of science-based practice; this journal was also among the top 10 relevant journals as rated by practitioners in the focal article. Another suggestion is for journals to include article types that highlight practitioner contributions as well as more traditional articles. One such example is Personnel Assessment and Decisions, which includes a Practitioner Demonstration Project subsection, while also emphasizing that articles in general should have practical relevance. Another is the International Journal of Selection and Assessment’s Information Exchange article type, which allows for the publication of practically useful but atheoretical research or even practitioner persuasive essays.

Requiring authors to articulate practical contributions

Editors can also help reduce the research-practice gap by requiring authors to clearly articulate the applied implications of their research within their manuscripts and abstracts. Such a change is precisely the reverse of a publication policy change made at Personnel Psychology over a decade ago to eliminate such a requirement, in hindsight a harmful decision for the health of our field. Additionally, requirements for practical contribution do not condemn basic research in I-O psychology; instead, they force authors to consider and articulate how their basic research could directly or indirectly lead to future findings of practical importance, outlining a path to practical value. If the original author of that research does not consider and articulate its current or eventual practical implications, it is unlikely anyone else will either.

Additionally, authors should be more directly encouraged to submit manuscripts involving meaningful research-practitioner partnerships, which will itself be encouraged by requiring articulation of practical implications. Reviewers should in response to such changes consider the trade-offs faced by field researchers between internal and external validity; some field research with messy designs simply cannot be done well in the lab, and this fact should not condemn messy field research. More meaningful research partnerships between academics and practitioners, and greater reviewer understanding of the challenges of practical research and value despite those challenges, can help to ensure that research has both clear practical value and scholarly value.

Improving practitioner access to and public accessibility of I-O psychology research

With better practices in place to produce more practically useful research, policy changes and initiatives must be made to improve practitioner access to that research and to build broader accessibility of that research, which ultimately benefits both academics and practitioners of I-O psychology. A broad cultural awareness of practical I-O psychology research would benefit the entire field through increased grant funding, university support, media appearances, and practical uptake of research-based practices. Our suggestions here include more practice-friendly research summaries, research “push” strategies, promotion of nonscholarly outlets, and planning for accessibility throughout the research process.

More practice-friendly research summaries

Results from the 2019 SIOP Practitioner Survey published in The Industrial-Organizational Psychologist showed that practitioners place great value on keeping current on their knowledge and expertise (Solberg & Porr, Reference Solberg and Porr2019). Every several years there is a SIOP preconference workshop that regularly sells all seats focusing on summarizing the last approximately five years of scholarly research and highlighting the applied implications. Despite its significant value, this workshop by design can only reach a small audience of well-funded practitioners. The SIOP Practitioner Survey results demonstrated a desire for more such opportunities. When practitioners voted on which initiatives they would like to see implemented, the plurality of votes was for SIOP to provide a “Relevant research update: Publishing a regular (e.g., annual) review of recent impactful I-O research and its implications for practice” (Solberg & Porr, Reference Solberg and Porr2019, Table 2). A yearly “state of the literature for practice” white paper would provide enormous value to the practitioner community, as well as providing important commentary on the practical value of all I-O psychology research published that year. Additionally, we recommend that SIOP more prominently promote to practitioners existing science popularization resources for I-O psychology, such as I-O at Work (https://www.ioatwork.com/).

“Push” instead of “pull” strategy to reach practitioners

In the focal article, it was noted that practitioners were most likely to perceive as relevant The Industrial Psychologist and Industrial and Organizational Psychology journals (Highhouse et al., Reference Highhouse, Zickar and Melick2020). One thing that is unique about these journals is that both publications are available as part of SIOP membership and made available with few barriers for no additional investment. Additionally, SIOP “pushes” notifications of the availability of new issues to SIOP members. The cost and convenience barriers are major problems for practitioners on a tight budget with little flexibility in work hours. Although most journals have mailing lists that send emails or other notifications upon the publication of new research, this requires practitioners to sort through a vast quantity of irrelevant research to find what they can actually use. Better systems for matching practitioners to practical research that would support their work are desperately needed.

Promote research in nonscholarly outlets

Practitioners in general have taken to broader media outlets both to share their own research and to consume new ideas, such as through blogs, press interviews, and popular press books and articles. Although some of these outlets (e.g., blogs) haven’t gained a lot of traction in terms of readership among an I-O practitioner audience (Solberg & Porr, Reference Solberg and Porr2019), such information is often shared with clients who can then use it to inform business decisions. In recent years, well-respected academic scholars (e.g., Steven Rogelberg, Adam Grant) have also shared I-O psychology research through best-selling business books and media appearances, translating journal findings into easily understandable and meaningful applications, consumable by academics from other disciplines, I-O psychology practitioners, human resources management practitioners, and the public alike. We believe this should be encouraged and rewarded across both academic and practitioner settings to help promote and demonstrate the relevance of research in I-O psychology to better improve our field and reduce the science-practice gap.

Plan for accessibility throughout the research process

Both academic and practitioner researchers should plan for how their research will support and influence academics, practitioners, and society at large, starting at the idea-generation stage. At all points in the research process, accessibility should remain central. For example, the #betterposter movement led by Mike Morrison attempts to drastically change the format of scientific posters to make them more digestible. Instead of cramming all possibly relevant information onto the poster, Morrison’s approach reimagines posters as a medium for communicating ideas, placing the main conclusions in large print, key charts and key details in smaller print, and the remainder in online source documents that are made available by scanning a Quick Response code. A few posters began to use this format at the SIOP 2019 conference, and there have been discussions of trialing the new poster format as a research project at SIOP 2021. To learn more about #betterposter, see Flaherty (Reference Flaherty2019) or this YouTube video (https://www.youtube.com/watch?v=1RwJbhkCA58).

Conclusion

To make scholarly research more useful and accessible for practitioners of I-O psychology and lay people alike, we as a field need to work to bring our diverse areas of expertise together to actually address the problem. The science-practice gap will not fix itself with time, and we should stop acting as though it will. Whether by journal editors, SIOP initiatives, or driven individuals, it will only be through active pursuit of meaningful changes to policy to enhance the relevance, value, access, and accessibility of research that we will ever bridge this gap.

References

Aguinis, H., & Vaschetto, S. J (2011). Editorial responsibility: Managing the publishing process to do good and do well. Management and Organization Review, 7(3), 407422. doi: 10.1111/j.1740-8784.2011.00223.x CrossRefGoogle Scholar
Campbell, J. P., & Wilmot, M. P. (2018). The functioning of theory in industrial, work and organizational psychology (IWOP). In Ones, D. S., Anderson, N., Viswesvaran, C., & Sinangil, H. K. (Eds.), The SAGE handbook of industrial, work & organizational psychology: Personnel psychology and employee performance (pp. 3–38). Sage Reference.Google Scholar
Cramer, D. P. (2007). Factors that facilitate practitioner publication: A grounded theory analysis [Unpublished doctoral dissertation]. Indiana University.Google Scholar
Cucina, J. M., & Moriarty, K. O. (2015). A historical look at theory in industrial-organizational psychology journals. Society for Industrial-Organizational Psychology, 53(1), 5770.Google Scholar
Highhouse, S., Zickar, M. J., & Melick, S. R. (2020). Prestige and relevance of the scholarly journals: Impressions of SIOP members. Industrial and Organizational Psychology: Perspectives on Science and Practice, 13(3), 273–290.Google Scholar
Flaherty, C. (2019, June 24), #betterposter. Inside Higher Ed. Retrieved February 22, 2020 from https://www.insidehighered.com/news/2019/06/24/theres-movement-better-scientific-posters-are-they-really-better Google Scholar
Landers, R. N. (2019). The existential threats to I-O psychology highlighted by rapid technological change. In Landers, R. N. (Ed.), Cambridge handbook of technology and employee behavior (pp. 321). Cambridge University Press.CrossRefGoogle Scholar
Solberg, E., & Porr, B. (2019). What do practitioners want? Survey practitioner survey results revealed! https://www.siop.org/Research-Publications/Items-of-Interest/ArticleID/3179/ArtMID/19366 Google Scholar