Skip to main content Accessibility help
×
Home

Information:

  • Access

Actions:

      • Send article to Kindle

        To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

        Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        What is the actual impact of measuring academic notions of impact?
        Available formats
        ×

        Send article to Dropbox

        To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

        What is the actual impact of measuring academic notions of impact?
        Available formats
        ×

        Send article to Google Drive

        To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

        What is the actual impact of measuring academic notions of impact?
        Available formats
        ×
Export citation

To most people, the word ‘impact’ conjures up images of the forceful collision of one object into another, the landing of a blow, perhaps, or the crashing together of two cars moving at speed. In academia, impact has come to mean something rather different, although frequently no less concerning for those who struggle with it on a daily basis: the effect or influence of empirical work beyond academia. However, when we look at the ways in which impact is assessed and the systemic pressures placed on universities, journal editors, and researchers by this assessment, it is often something more like a car crash that comes to mind. Given the troubled relationship many academics have with the notion, it is worth stopping and asking ourselves just what is the actual impact of measuring impact? This editorial offers an explainer for the academic notion of impact and asks what it looks like for the Educational and Developmental Psychologist and other growing journals going forward.

What is impact?

As a researcher and academic interested in school belonging, the real impact of my work is found in the implementation by teachers of effective strategies and approaches that foster opportunities for their students to belong, in school leaders offering supportive climates for their staff, in communities working alongside schools to address a changing social landscape that impinges on feelings of belonging towards others and places, and, ultimately, in the sense of belonging felt by students towards their schools (Allen, Kern, Vella-Brodrick, Waters, & Hattie, Reference Allen, Kern, Vella-Brodrick, Waters and Hattie2018). Allen, Kern, Vella-Brodrick, & Waters, Reference Allen, Kern, Vella-Brodrick and Waters2017; Allen, Vella-Brodrick, & Waters, Reference Allen, Vella-Brodrick and Waters2016). However, to date, measures of impact do not directly assess the real-world implications of my research, but instead inform me if my publications are being cited by other researchers (not necessarily my central audience).

Traditional Journal Impact Factors (JIFS) are calculated by Clarivate Analytics using an extremely simple metric, based on nothing more than the number of times selected articles are cited. The more often a journal is cited, the higher its impact factor; the higher the impact factor, the more prestigious the journal is presumed to be. A number of other models incorporating a wider variety of metrics are also now used to evaluate journal rankings but all take the number of citations journals receive as their foundation. For example, SCImago Journal Rank (Scopus, Elsevier) evaluates scholarly journals based on number of citations, but weights these citations based on the ranking of the journals they are found in. The citations issued by more “prestigious” journals are thus of greater worth than citations from less important journals. The SNIP (Source Normalized Impact per Paper) and CiteScore ranking are other metrics that also draw on citation scores. Despite their widespread use, some scholarly work suggests that journal impact factors are ambiguous and unreliable (e.g., Casadevall & Fang, Reference Casadevall and Fang2015; Campanario, Carretero, Marangon, Molina, & Ros, Reference Campanario, Carretero, Marangon, Molina and Ros2011; Greenwood, Reference Greenwood2007).

For individual articles, Altmetric scores are becoming increasingly important. Altmetric scores measure how much attention an article receives online and are thus able to capture the degree of broader global interest in specific pieces of research rather than just the interest in the research shown by other researchers. These scores are inherently more inclusive, in that they widen the base of significant interested parties to take in all internet users rather than just other academics. As more people use technology as their primary vehicle for receiving information, Altmetric scores may, with time, become a reliable way to measure public engagement with research. Nevertheless, the actual impact of research remains elusive.

Why we need to talk about impact

Journal impact factors and other measures that are used to rank journals have an enormous influence on where researchers publish, on what they publish in order to reach their impact goals, and, ultimately, on who reads the research that they publish. Restrictions on where researchers can publish vary between universities and even countries, with some institutions being more inflexible than others. Publications in the more prestigious journals lead to rewards, incentives, and promotions. Never before has there been a more obvious systemic influence on researchers working in modern universities, a situation that can be compared to the exertion of overt political control over academia or even to past eras in which religious demands constrained research. This systemic hurdle has, itself, an enormous ‘impact’ (in the classical sense of the word), in that it arguably serves as a mechanism for controlling what information is generated by research and consequently disseminated to the general public.

As researchers, practitioners, and academics it is important that we discuss the notions of impact that have become prevalent in our institutions. Is impact measured effectively? And if it is (or perhaps is not), what are the consequences of measuring impact? While understanding the actual and real notable influence of research is important, it is wise to understand that high-quality and high-impact work can also be found outside of the more prestigious journals.

Towards real high-impact

The current issue of the Educational and Developmental Psychologists presents work with tremendous potential for applied and practical impact. With this in mind, how can we ensure that the articles in this current edition, Volume 37, Issue 2, have tangibly high impact? (See Alegre & Benson, Reference Alegre and Benson2019; Alonso & Little, Reference Alonso and Little2019; Declercq, Marle, & Pochon, Reference Declercq, Marle and Pochon2019; Jimeno, Reference Jimeno2019; Hemmings, Kay, & Sharp, Reference Hemmings, Kay and Sharp2019; Missenden & Campbell, Reference Missenden and Campbell2019; Wheldall, et al., Reference Wheldall, Bell, Wheldall, Madelaine, Reynolds and Arakelian2019). While citing is certainly to be encouraged, what is more important is that useful information is actually disseminated to those who can make valuable use of it. Circulate, share, and distribute; these are the best ways to ensure that research on educational and developmental psychology has real impact at the sharp end. Is there an interesting finding? An interesting quote? Then post about it on Twitter, Instagram, Linkedin or Facebook. Print the article out to hand around to interested parties in hard copy. Just make sure that the work researchers do gets into the hands of those who can use it.

Sharing information is the first step to take towards making a real impact. While Altmetric scores have sought to address the changing ways in which we measure impact, publication metrics are ultimately meaningless unless the research we produce leads to positive outcomes in the lives of others. To date, there is no genuine and reliable metric to measure this kind of real impact and perhaps there never will be. Or maybe the measure of impact should be as fluid and diverse as the actual research which an impact metric seeks to assess? Replication, evaluation, and validation then become alternative and yet, necessary ways in which impact can be measured. Although not an entirely new concept.

While the journal impact factor is undoubtedly here to stay and will continue to influence researcher behaviour, we should not confuse journal impact with journal importance. For professional journals such as the Educational and Developmental Psychologist, this creates an important distinction. Professional journals fill a scholarly gap that journals cannot meet by mere impact factor alone.

References

Allen, K. A., Kern, P., Vella-Brodrick, D., Waters, L., & Hattie, J. (2018). What Schools Need to Know about Belonging: A meta-analysis. Educational Psychology Review, 30(1), 134. doi: 10.1007/s10648-016-9389-8
Allen, K. A., Kern, P., Vella-Brodrick, D., & Waters, L. (2017). School Values: A comparison of academic motivation, mental health promotion, and school belonging with student achievement. Educational and Developmental Psychologist, 34(1), 3147.
Allen, K. A., Vella-Brodrick, D., & Waters, L. (2016). Fostering School Belonging in Secondary Schools Using a Socio-Ecological Framework. The Educational and Developmental Psychologist, 33(1), 97121. doi:10.1017/edp.2016.5
Alegre, A., & Benson, M. (2019). Family warmth, self-perception, social competence, and friendships in emerging adulthood. The Educational and Developmental Psychologist, 17. doi: 10.1017/edp.2019.10
Alonso, J., & Little, E. (2019). Parent help-seeking behaviour: Examining the impact of parent beliefs on professional help-seeking for child emotional and behavioural difficulties. The Educational and Developmental Psychologist, 18. doi: 10.1017/edp.2019.8
Campanario, J.M., Carretero, J., Marangon, V., Molina, A., & Ros, G. (2011). Effect on the journal impact factor of the number and document type of citing records: a wide-scale study. Scientometrics, 87(1), 7584. doi: 10.1007/s11192-010-0333-2
Casadevall, A., & Fang, F. C. (2015). Impacted Science: Impact Is Not Importance. American Society for microbiology, 6 (5) e0159315; doi: 10.1128/mBio.01593-15
Declercq, C., Marle, P., & Pochon, R. (2019). Emotion word comprehension in children aged 4–7 years. The Educational and Developmental Psychologist, 16; doi: 10.1017/edp.2019.17
Greenwood, D. C. (2007). Reliability of journal impact factor rankings. BMC medical research methodology, 7, 48. doi: 10.1186/1471-2288-7-48
Hemmings, B., Kay, R., & Sharp, J. (2019). The relationship between academic trait boredom, Learning approach and university achievement. The Educational and Developmental Psychologist, 110; doi: 10.1017/edp.2019.11
Jimeno, M. (2019). Improving the quality of life of children and adolescents with autism spectrum disorders through athletic-based therapy programs. The Educational and Developmental Psychologist, 17; doi: 10.1017/edp.2019.7
Missenden, N., & Campbell, M. (2019). Secondary school teachers’ ability to recognise and refer students with differing levels of anxiety. The Educational and Developmental Psychologist, 19; doi: 10.1017/edp.2019.12
Wheldall, K., Bell, N., Wheldall, R., Madelaine, A., Reynolds, M., & Arakelian, S. (2019). A small group intervention for older primary school-aged low-progress readers: Further evidence for efficacy. The Educational and Developmental Psychologist, 16; doi: 10.1017/edp.2019.15