The impact of open science on practitioners

The goal of I-O psychology is evidence-based practice; however, I-O consultants and human resource (HR) professionals often question the usefulness of scholarly research (Rynes et al., Reference Rynes, Colbert and O’Boyle2018). Recent debates on the “replication crisis” in psychology have cast further doubt on the credibility of researchers. The growing concerns about research misconduct and questionable research practices (QRPs; e.g., p-hacking, HARKing) have galvanized the open science movement or “psychology’s renaissance” (Nelson et al., Reference Nelson, Simmons and Simonsohn2018).

The open science movement offers potential benefits to practitioners. While there is variation in the adoption of open science practices across academic disciplines and even within the subfields of psychology, open science practices typically include preregistration, open data, and open materials (Banks et al., Reference Banks, Field, Oswald, O’Boyle, Landis, Rupp and Rogelberg2019). From the perspective of consumers of research, preregistration of design and analysis plans can reduce unintentional QRPs (Kupferschmidt, Reference Kupferschmidt2018), thereby increasing the trustworthiness of scientific knowledge. Open datasets which are publicly available for download can allow practitioners to verify, question, and build on the results. Namely, consultants can use open data to establish “proof of concept” to get buy-in from stakeholders or pilot extensions of the work (especially with big data; Guzzo et al., Reference Guzzo, Schneider and Nalbantian2022). Open study materials that are provided at no cost further facilitate direct and conceptual replications. This enhances the internal and external validity of research findings which can give practitioners more confidence in the generalizability of evidence-based recommendations. Open materials can also provide professionals with validated measures to use with organizational samples for benchmarking. Taken together, the adoption of open science practices holds promise for evidence-based management.

Nevertheless, the open science movement is relatively young and there is ongoing debate on best practices (Banks et al., Reference Banks, Field, Oswald, O’Boyle, Landis, Rupp and Rogelberg2019). Within psychology, many have only a cursory knowledge of the issues. As a result, there is uncertainty about how to educate students on the replication crisis and how to train the next generation of researchers to implement open science practices (Chopik et al., Reference Chopik, Bremner, Defever and Keller2018). Of concern, the conversations on open science occur primarily among academic communities and within scholarly circles. As such, the primary agents of culture change are authors, journal editors and reviewers, and funding agencies. The academic and scientific community have yet to include the diverse voices of HR professionals, I-O consultants, and policy makers into the discussion.

Widening the scientist–practitioner gap

The one-sided discourse on open science is problematic because there is already a disconnect between the knowledge that researchers produce and the knowledge that practitioners implement. Historically, the reasons for the research-practice gap include financial burden (e.g., articles behind paywalls), relevance of topics, time constraints, and technical and stylistic complexity (Banks et al., Reference Banks, Pollack, Bochantin, Kirkman, Whelpley and O’Boyle2016). Very few HR professionals and consultants read research-oriented journals (Rynes et al., Reference Rynes, Colbert and Brown2002). Instead, they draw on other sources such as professional peers and mainstream media outlets for information. These sources can misrepresent the replication crisis in psychology and intensify distrust of research.

In the focal article, Guzzo et al. (Reference Guzzo, Schneider and Nalbantian2022) discuss the difficulties that practitioner–researchers face in the publication process. Managers and HR professionals who are consumers of research face different challenges. While the greater transparency of research increases the rigor of research (e.g., studies that are better planned, fewer false positives), it also increases the complexity of knowledge transfer. The implementation of open science practices increases the length and difficulty of journal articles (e.g., detailed supplemental materials are more common). Greater openness to “messy” findings (Aguinis et al., Reference Aguinis, Banks, Rogelberg and Cascio2020) and to studies that “don’t work” (e.g., null effects may be more likely to be published with registered reports and results-blind reviews) may make it difficult to distill what is relevant and useful. While openness to imperfect findings may reduce the number of effects in file drawers and be a boon to the study of real-world problems in organizational research, open science initiatives are only useful to the extent that readers have the background to vet the quality of research. Practitioners who do not have the time and training to read dense journal articles and digest methodological details may be more “lost in translation” than ever before (Shapiro et al., Reference Shapiro, Kirkman and Courtney2007). So, while the open science movement has made great strides in increasing access (e.g., availability of articles and data), it still lags in inclusion (e.g., participation is limited to those with expertise in research methods).

Bridging the divide

The point of this commentary is not to reject or cast doubt on the value of open science initiatives. The key tenets of rigor, transparency, and reproducibility are critical for quality science. However, the adoption of open science initiatives without consideration of the consumers of research can magnify the research-practice gap. The field should hold the door open to all stakeholders, including practitioners who are essential to the application of evidence-based I-O psychology. To abate the research-practice gap in open science, I propose three approaches drawing on Aguinis et al.’s (Reference Aguinis, Banks, Rogelberg and Cascio2020) stages of bridge-building: knowledge production, transfer, and training.