The goal of focal articles in Industrial and Organizational Psychology: Perspectives on Science and Practice is to present new ideas or different takes on existing ideas and stimulate a conversation in the form of comment articles that extend the arguments in the focal article or that present new ideas stimulated by those articles. The two focal articles in this issue stimulated a wide range of reactions and a good deal of constructive input.

Most future industrial and organizational (I-O) psychology practitioners and researchers initially enroll in an introductory I-O psychology course during their junior or senior year of undergraduate studies, making introductory textbooks their first in-depth exposure to the field and an important knowledge base. We reviewed and analyzed the 6,654 unique items (e.g., journal articles, book chapters) published in 1,682 unique sources (e.g., scholarly journals, edited books, popular press publications) and authored by 8,603 unique individuals cited in six popular I-O psychology textbooks. Results showed that 39% of the top-cited sources are not traditional academic peer-reviewed journals, 77% of the top-cited articles were published in cross-disciplinary journals, and 58% of the top-cited authors are affiliated with business schools and not psychology departments. These results suggest that the science–practice divide in I-O psychology may develop later—perhaps after graduates obtain employment as either practitioners or researchers. Also, results suggest I-O psychology is closer to business and management than social psychology and psychology in general. We discuss additional implications for the science–practice divide, how to define and measure scholarly impact, and the future of I-O psychology as a field, including the movement of I-O psychologists to business schools and the sustainability of I-O psychology programs in psychology departments.

Over the last 5 years, there has been a surge of interest and activity among large groups of practitioners and scientists in industrial and organizational (I-O) psychology, who have joined together with a broad range of agencies and organizations, in small to very large collaborative teams, to examine some of the grand challenges and problems facing our field. For example, a consortium of university professors and their counterparts in five Fortune 100 firms, two global human resources (HR) consultancies, and NASA have been working together to examine the practical and scientific challenges in determining how to optimize the selection and on-boarding of a new generation of workers charged with building the first colony on Mars. One I-O practitioner involved with this initiative said, “Our project represents one of the most seamless and productive integration efforts involving the practice and scientific community, perhaps in the history of our profession.”

Abraham Lincoln was fond of saying “killing the dog does not cure the bite” when referring to problems and their persnickety pervasiveness. When thinking about the problems facing the industrial and organizational (I-O) psychology profession, there is no greater source of frustration than the gap between a scientist's findings and the application of those findings to practice. In recent years, organizations such as the White House Behavioral Sciences unit, the Society for Human Resource Management (SHRM) Foundation in partnership with The Economist Intelligence Unit, and many others have explored the gap between research and practice and have highlighted every major derailer, from delays associated with peer-reviewed publication cycles to a lacking infrastructure for bringing science to practitioners. In 2014, the SHRM Foundation even went so far as to implement a strategy based on driving research directly to practitioners through executive round table forums. Despite the best efforts to identify strategies for closing the gap, many organizations have failed to find the optimal means for bringing I-O psychology research to the masses of human resource (HR) practitioners and, in many cases, even I-O psychology practitioners dealing with significant organizational issues.

I will be making some highly personalized comments on the Aguinis et al. (2017) focal article concerning rigor versus relevance, renaming/rebranding industrial and organizational (I-O) psychology, and I-O psychology versus business school organizational behavior (OB). Before commenting, however, I feel compelled to briefly frame my remarks from the perspective of my 50-year academic career. For example, I think it is important to note that I go back to the early 1960s at the University of Iowa, College of Business. I was studying for my PhD in the just emerging field of management and organizations (nothing was offered called organizational behavior or strategic management). However, and very unusual for the times for management majors, I also took a minor in the psychology department concentrating on social and I-O psychology. Also, after receiving my PhD in 1965, for my 2-year military obligation, after infantry officer training I was assigned to West Point and taught cadets the required psychology course and military leadership. This background had a formative and lasting impact on my thinking about OB and I-O psychology.

Aguinis et al. (2017) highlighted the gender disparity in authorship of publications within the field of industrial and organizational (I-O) psychology. We agree with the authors that this is a troubling finding and think that this gender disparity within our field is the most critical implication of the focal article. I-O psychologists are specifically trained to address employment issues, including gender disparities at work. To see such disconcerting findings in our area of expertise is akin to a sports team losing to a competitor when they have the home court advantage. Namely, we are left feeling deflated and asking ourselves, “What went wrong?”

The focal article by Aguinis et al. (2017) offers a rich brew of data and explication regarding the devilishly complicated concept of impact of sources (both science-based and practice-based), of industrial and organizational (I-O) psychology articles, and of authors of articles in the discipline of I-O psychology. The criteria developed to assess the impact of these are simple frequency counts indexing the number of times items within each domain are cited by leading introductory I-O psychology textbooks, which in turn leads to ranked lists. These are then used as a basis for answering numerous highly significant questions about the discipline of I-O psychology, including its scholarly and practice-based underpinnings, and the future prospects of the field.

It is encouraging to see the diversity of sources cited in the top introductory textbooks in industrial-organizational (I-O) psychology. It is not surprising that the Journal of Applied Psychology (JAP) and Personnel Psychology are off the charts in terms of the two most-cited sources in these six textbooks. However, I would not have expected that in these books, “77% of the top-cited articles were published in cross-disciplinary journals” (Aguinis et al., 2017, p. 507). I wish to build on Aguinis et al. (2017), with a focus on the relationship between I-O psychology and human resource development (HRD). I am the author of an HRD textbook (Werner, 2017a). The journal I co-edit, Human Resource Development Quarterly, is one of those “cross-disciplinary journals” on Aguinis et al.’s list of most-cited sources (56th out of 110, 40th in academic journals), with 16 citations. Finally, my article on legal issues in performance appraisal was cited in five of the six textbooks (Werner & Bolino, 1997).

In their focal article, Aguinis et al. (2017) conducted an empirical analysis of the most frequently cited sources, articles, and authors in industrial and organizational (I-O) psychology textbooks. The authors conclude that their “results are encouraging regarding the scientist–practitioner model” (p. 545). We disagree. Although we applaud the effort that went into this research, we are concerned that the method used in the article, focusing on textbook citations, creates yet another “researcher-centric” index that will do nothing to address the research–practice gap. The problematic “researcher-centric” perspective manifests itself in several ways in the focal article, which we elaborate below.

Academics sometimes forget that the purpose of a university is to educate: our students, our local communities, each other, and the world. Although each university is unique in its constituency, all share the charge to generate knowledge for the protection and benefit of the public good. The goal of an academic should be to beneficially impact society, broadly defined, with scholarly activity. As editor and columnist for The Industrial-Organizational Psychologist, one publication highlighted by the focal article, we applaud the efforts of Aguinis et al. (2017) to put forth alternative approaches to defining impact. Like them, we are concerned that many of the measures of “impact” we currently use do not capture this charge.

In their focal article, Aguinis et al. (2017) provided a bibliometric analysis of our six industrial and organizational (I-O) psychology textbooks, noting among other things the sources, articles, and authors we collectively cited the most. Their analysis provides information about what we cited but not why. In this commentary on their article, our goal is to provide some insights into our process in deciding what sources to include and what not to include in our textbooks. Although each of us has our own way of deciding on the content of our books, there is enough commonality that we decided to write this commentary together.

In their focal article, Aguinis et al. (2017) categorized the 6,654 unique citations, summed across the six introductory industrial and organizational (I-O) psychology texts, in various ways. They then suggested how such data could be used to (a) infer the “state” of the scientist–practitioner divide; (b) document the extent of the movement of I-O psychologists to management schools; (c) evaluate the future prospects of I-O psychology as a field; and (d) provide guidance in how to define, measure, and reward “scholarly impact” (quotation marks added). This crosses the line from interesting to very counterproductive.

Aguinis et al. (2017) contribute interesting analyses of cited sources in contemporary undergraduate industrial-organizational (I-O) psychology textbooks and continue their ongoing investigation into the long-term viability of I-O psychology as a unique discipline (see Aguinis, Bradley, & Brodersen, 2014). These analyses, conducted by authors who are members of business schools, attempt to answer questions related to the nature of work conducted by I-O psychologists, comparing the quality and importance of work conducted by faculty in business schools with that conducted by faculty in psychology departments. One of their general themes is that members of business schools are conducting important research that is influencing the future of I-O psychology by overtaking undergraduate textbooks. As such, the article has the feel of a conquering hero taunting its vanquished foe.

Aguinis et al.’s (2017) analysis of the “most frequently cited sources, articles, and authors in industrial-organizational psychology textbooks” is a commendable piece of scholarship. Certainly, they have applied themselves to an important question and articulated a meaningful set of answers. We have no doubt too that for many readers the insights and answers they provide will be informative, compelling, and even reassuring—if only because they reinforce a view of the world with which they are familiar and by which they are comforted, even if that familiarity and comfort are framed in terms of a set of knotty professional concerns (Morton, Haslam, Postmes, & Ryan, 2006).

Aguinis et al. (2017) address an issue of upmost importance for the field of industrial-organizational (I-O) psychology: recruitment. The ability to attract and retain talented individuals is a principle determinant of success in a knowledge-driven economy (Yu & Cable, 2012). The focal article notes that future practitioners and researchers are commonly exposed to the field of I-O psychology for the first time via introductory courses taken during their undergraduate education. A study by Rose et al. (2014) likewise suggests that introductory courses are among the most popular channels through which business and human resource professionals learn about I-O psychology. Consequently, the information communicated in these courses not only shapes the beliefs and behaviors of those who might one day produce/provide the goods/services of I-O psychology, but also those who might consume them. Introductory courses are, therefore, both an important recruitment source as well as an important marketing channel. Aguinis et al. provide a much-needed content analysis of the information communicated to students through introductory textbooks and offer insight into the ways in which this information may affect the future of I-O psychology. Building from their analysis of content, this commentary offers an approach to program evaluation that utilizes the principles of brand management to better understand how the messages communicated to students impact their beliefs about the field. Moving from analysis to evaluation is a logical next step in making a desired future for I-O psychology.

The rate of technological change is quickly outpacing today's methods for understanding how new advancements are applied within industrial-organizational (I-O) psychology. To further complicate matters, specific attempts to explain observed differences or measurement equivalence across devices are often atheoretical or fail to explain why a technology should (or should not) affect the measured construct. As a typical example, understanding how technology influences construct measurement in personnel testing and assessment is critical for explaining or predicting other practical issues such as accessibility, security, and scoring. Therefore, theory development is needed to guide research hypotheses, manage expectations, and address these issues at this intersection of technology and I-O psychology. This article is an extension of a Society for Industrial and Organizational Psychology (SIOP) 2016 panel session, which (re)introduces conceptual frameworks that can help explain how and why measurement equivalence or nonequivalence is observed in the context of selection and assessment. We outline three potential conceptual frameworks as candidates for further research, evaluation, and application, and argue for a similar conceptual approach for explaining how technology may influence other psychological phenomena.

Modern technology and technological advances offer a variety of benefits and challenges for assessment, data collection, communication, and other research- and practice-related endeavors. The focal article written by Morelli, Potosky, Arthur, and Tippins (2017) offers a segue into discussions about some of these issues. Although the authors offer some unique insights, we believe their view is incomplete, as it is potentially limited by their focus on testing and assessment. Below, we outline a few key points we hope will advance the conversation. Our commentary is largely grounded in the field of human–computer interaction (HCI), which is an interdisciplinary field that integrates expertise from computer science, psychology (and other behavioral sciences), and many other fields. Whereas psychology tends to place the human user at the forefront of discussions concerning technology, HCI expands beyond just the user's psychology, focusing on the design of interfaces that allow users to interact with computing technology in new ways (Card, Moran, & Newell, 1983).

Morelli, Potosky, Arthur, and Tippins (2017) make a timely and appropriate call for authors to create conceptual models of technology in industrial-organizational (I-O) psychology. We agree with their call, but we believe that Morelli et al. overlooked the contributions of related fields that conduct research on technology in the workplace that are already consistent with their call. For this reason, we briefly detail other fields that commonly study the dynamics of technology and its influence on the workplace, followed by a discussion regarding the place of I-O psychology in the broader scheme of technology research. This discussion can aid future authors in conceptualizing appropriate contributions to the study of technology in I-O psychology as well as identifying whether these contributions benefit other fields. Perhaps more importantly, this discussion can help identify where I-O psychology fits in the broader scheme of technology research and which associated fields may be most readily available to aid in the creation of new models—two questions that currently seem unanswered.

In industrial-organizational (I-O) psychology, much like in the organizational sciences more broadly (Hambrick, 2007), we have a bit of an addiction to theoretical models. It is commonly assumed that developing new theory is the most valuable way to solve pressing research problems and to drive our field forward (Mathieu, 2016). However, this assumption is untested, and there is growing awareness among organizational scientists that this hardline approach, which is unusual among both the natural sciences and other social sciences, may even be damaging the reputation and influence of our field (Antonakis, 2017; Ones, Kaiser, Chamorro-Premuzic, & Svensson, 2017). As Hambrick (2007) describes, the requirement for theory first “takes an array of subtle, but significant, tolls on our field” (p. 1348). As we will describe in this article, Morelli, Potosky, Arthur, and Tippins’ (2017) suggestions, if taken at face value, will likely create such tolls by encouraging the creation of new theories of dubious value. To be clear, we agree with Morelli et al. that better theory is needed for technology's impact on I-O psychology broadly and talent assessment in particular. We disagree, however, that creating new technology theories using the approaches that I-O psychology typically employs is likely to accomplish this broader goal. Rather, it will ultimately only isolate research on I-O technologies even further from both mainstream I-O research and technology research. Given that we are already quite isolated, this would be a disastrous path.

We wonder whether theory alone can solve problems and answer questions faced by practitioners working on the front lines of assessment innovation. Stated another way, to what degree can current theories influence the application of our work to new technology when it comes available? We are speaking as practitioners working in selection, the area in which technology has been studied most commonly in industrial-organizational (I-O) psychology (e.g., King, Ryan, Kantrowitz, Grelle, & Dainis, 2015). More specifically, we focus on the impact of mobile technology on our selection systems. We are excited for the focal article (Morelli, Potosky, Arthur, & Tippins, 2017) on theory development relative to technological advancement because much of the work we do in this area has not been discussed significantly in the literature. Our goal in this commentary is to review what we have learned about the implications of technology from our experience building and validating innovative prehire assessments.