Summing up our findings

This book has drawn on a wide range of qualitative data to explore how the UK's approach to research impact is playing out. It started by outlining the policy changes that had informed, and culminated in, the current approach to research impact (Chapter 2) and by outlining the various arguments for, and concerns about, the consequences of this approach (Chapters 1 and 3). In Chapter 3, we grouped issues raised in existing literature into ten key concerns. These ranged from the fundamental (is it possible to meaningfully demonstrate, measure and attribute research impact? What are the theoretical foundations of the UK's approach to research impact and how do these accord with relevant scholarly literature? Does the UK's approach to research impact restrict academic freedom/autonomy and/or misleadingly assume research impact to be always positive?) to the more prosaic and pragmatic (do incentives for academics to demonstrate research impact risk activities which will further overload external audiences with information? Is the seemingly arbitrary time limit appropriate or restrictive? What are the costs of assessing research impact and do these represent value for money? Does the current approach reify individual working and traditional elites?). Much of the literature informing this chapter sketched out theoretical concerns or identified issues in how research impact was affecting specific areas of scholarly work. This book builds on these existing contributions by taking an empirically grounded, in-depth exploration of how relevant groups (particularly academics working in the UK but also impact assessors, funders and academics working elsewhere) view the UK experiment with research impact. While our own social science grounding and outlook has inevitably informed the focus of our data and analysis (including, perhaps most notably, a recurrent focus on policy and practice as key ‘targets’ of research impact efforts), collectively our data map across a wide range of academic disciplines.

Our findings, like the existing literature, paint a complex and varied picture. There is no doubt that contrasting views exist about the UK's approach to research impact, within and beyond the UK, and that these are often strongly held.

As Chapter 2 showed, the current policy interest in research impact can be traced back to efforts to ensure that academic research provides a return on investment to the public purse. This agenda has risen alongside, and often rather separately from, a wider tradition of public engagement with, and in, academic research. The extent to which public engagement is seen as part of, or separate from, research impact appears to vary by discipline, as Chapter 4 discussed. This chapter explores the overlaps and elisions between these two fields of practice in more detail, comparing their articulation in guidance from research funders and regulators in the UK with how researchers from different disciplines describe the experiences, pressures and opportunities of each. We conclude with a brief reflection on continuing shifts in these unstable fields, as, in the UK, 2017's industrial strategy reorients research impact towards economic competitiveness.

Definitions: the relationship between impact and public engagement

It is not always clear or consistent what different actors seek to signify when they use the terms research impact or ‘public engagement’. Harmonising definitions of research impact between funders and evaluators of research, particularly with regards to public engagement, is one of the stated goals of REF2021 (HEFCE et al, 2017). The terms include distinct ‘targets’ (the public and, e.g., policy, practitioners or industry), and two ‘processes’ (impact versus engagement). These are contested terms; the notion of ‘the public’ (singular, coherent) is, in contemporary social science, almost always pluralised to recognise the dynamic and relational nature of societal groupings (Felt and Cochler, 2010; Newman and Clarke, 2009). There are other resonances and disagreements here: engagement implying a two-way process, impact seeming more didactic. However, the combinations that have come to dominate contemporary universities are for policy to be ‘impacted’ and ‘the’ public to be engaged.

The relative paucity of definitions of the term research impact in official guidance has been commented on before (Terama et al, 2016). By contrast, the work of the National Coordinating Centre for Public Engagement, an organisation funded by a consortium of HEFCE, UK Research Councils and the Wellcome Trust, has gone some distance to clarifying and harmonising definitions of public engagement. They have published a series of outputs which define and elaborate a considered and thorough definition.

Previous chapters have offered a critical review of the research impact agenda, situating it within international efforts to improve research utilisation for key groups including academics, research funders, ‘knowledge brokers’ and research users. This chapter considers where public intellectualism rests within these concerns. Specifically, the chapter explores the multiple public spheres that make up space between the academy and politics by focusing on: (i) three established academics working within the social sciences who engage outwardly in ways that might be described as ‘public intellectualism’; and (ii) three established academics who work closely with external policy audiences on a range of recognised public policy ‘problems’, who we refer to as ‘academic interlocutors’. These in-depth interview accounts from leading academics working in Australia, Canada, the USA and the UK allow us to continue the work of Chapter 5 in exploring the kinds of externally facing practices that are facilitated through the UK’s approach to impact, compared to the kinds of practices being supported in other contexts with an established interest in knowledge exchange and research utilisation. Since the literature on public intellectualism is distinct from much of the literature on which this book has drawn so far, this chapter begins by briefly reviewing some of the key contributions to this literature. This allows us to situate the analysis in this chapter within these broader discussions, including by considering the elementary question of whether (and if so, how) notions of public intellectualism and impact are perceived to be complementary. The international perspectives included in the data in this chapter allow us to reflect on how developments in the UK are perceived by externally engaged academics elsewhere.

Which public and what intellectual?

In the opening sentence of his seminal essay on the relationship between social scientific inquiry and political positioning, the late Howard Becker maintained that researchers will routinely ‘find themselves caught in a crossfire. Some urge them not to take sides, to be neutral and do research that is technically correct and value free. Others tell them their work is shallow and useless if it does not express a deep commitment to a value position’ (Becker, 1967: 239).

Introduction: key concerns with, and critiques of, the UK’s impact agenda

As Chapter 2 began to outline, the announcement of REF's inclusion of impact assessment ignited intense debate among academics. Statements outlining how research impact would be retrospectively assessed via REF appear to have generated particularly widespread concerns. In contrast, responses to the adoption of pathways to impact statements by UK Research Councils and the UKRI have, since Braben et al's (2009) Times Higher letter, been comparatively muted. Sociologist John Holmwood has questioned this, since it is in shaping new research that such changes have the potential to have their greatest repercussions for academic activity (though of course, advance knowledge about what REF is trying to measure and reward may also shape research activity):

Although the majority of voices commenting on the UK's approach to research impact are critical of specific aspects, this does not mean that they disagree with the basic, underlying assumption: that science should be, in some way, beneficial to society. Indeed, the idea that academics should be subject to something like a ‘social contract’ (Martin, 2011), in which they receive public funds and, in return, provide society with useful innovations, seems to be broadly supported, with some academics themselves articulating a moral or ethical obligation to work towards achieving (socially beneficial) change. For example, Eynon (2012: 2) reflects that researchers have ‘always wanted to make a difference’, while Smith's research on public health researchers cites multiple examples of academics expressing a desire to achieve impact (see Smith, 2013; see also Chapter 6).

The issues with the UK's research impact agenda that we have identified in existing literature rarely challenge the fundamental premise that academic research and other intellectual work should be socially beneficial. Rather, concerns tend to focus on the potential consequences of current efforts to incentivise, measure and reward research impact in the UK.

While news of impact as a new component of research performance evaluation appeared, in the late 2000s, to provoke much in the way of concern among large swathes of the UK's academic community (see Chapter 3), as we have also seen (particularly in Chapters 6 and 7), research impact, as featured in the UK's REF and research council funders, was not all bad news to all people. Nor, as Chapter 8 highlighted, was it a completely ‘new’ feature of university life – certainly, as will be argued in this chapter, in the context of the way it was responded to and ultimately co-opted by institutions as an instrument of self-promotion.

This chapter builds on some of the concerns outlined across the rest of the book and considers how academics’ participation within the REF's impact audit regime produces contradictions that destabilise its rationalisation as a vehicle for enhanced scientific transparency and accountability. It discusses how the impact agenda has produced a variety of illusions and/or misassumptions related to transformational change on academic praxis. It also points to how the centralising and prioritising of impact as a performance expectation of academics is a false corrective that disregards and fails to acknowledge much of what many academics already do. This chapter is the book's most critical take on the research impact agenda, ultimately contending that, as a catalyst for cultural and behavioural change (leitmotifs of the era of new public management), the impact agenda no more ameliorates the nexus between science and society than it inauthenticates the potential for trustworthy, meaningful and sustainable relationships between scientists and their public patrons. Finally, it deals with the mythologies that have sprung from and courted an impact agenda for academic research in the UK.

Telling (one-sided) tales

In this chapter, we argue that the UK's approach to research impact specifically within the REF (which is the aspect of the impact agenda that appears to have prompted the most critique across the other chapters of this book) has affected and accentuated (and prospectively will continue to affect and accentuate) a cleft between academics and their public communities. This is because this approach to research impact ‘demands’ that academics invest resources in telling a good tale of themselves, and of themselves only at the expense and exclusion of their ‘public partners’, unless where called upon as corroborators.

One of the most significant changes to have taken place during our collective time working at UK universities has been the rise of the ‘impact agenda’. Fifteen years ago, few academics had heard of, or were using, the term ‘research impact’. Back then, the contribution of academic work beyond universities was often only of concern to applied disciplines or to individual academics with an inclination towards achieving external engagement and influence. Institutional support for undertaking externally facing work was variable and funding opportunities generally limited. In that context, external engagement and impact were rarely core to academic job descriptions or promotion criteria.

All of this has changed. Working as an academic in UK universities today means that expectations and opportunities to engage with external audiences are higher, and more widespread, than they have ever previously been. Multiple external funding schemes relating to knowledge exchange activities, public engagement and research impact have emerged, and most of the large grant schemes require applicants to provide detailed information about the likely nonacademic beneficiaries, the likely impacts and/or public engagement plans. Most UK universities have introduced internal mechanisms to support research impact, including internal funding opportunities (the extent to which these initiatives can narrow the scope of engagement as part of strategic efforts to meet new performance measures is an issue we consider in this book). Many UK universities have also invested in new roles and centres that focus specifically on knowledge exchange and impact, with a view to both achieving grant income and performing well in the Research Excellence Framework (REF), which informs core research funding allocations to UK universities and which includes a focus on research impact assessment. Indeed, there now exists something of an ‘impact industry’ in the UK. The implications of this agenda affect not only research income but also prestige (Williams and Grant, 2018). It is increasingly hard to imagine university managers conceiving of permanent academic posts that do not include some expectations of outward-facing work, in the pursuit of both grant income and highly sought-after REF impact case studies.

For academics with a commitment to effecting ‘real-world’ change, and working collaboratively with members of the public, policymakers, activists, businesses or practitioners, some of these changes feel very welcome.

As we have seen, for some academics, commitments to research impact represent a disturbing set of practices designed to draw boundaries around what constitutes ‘legitimate’ academic work in ways that reify apolitical, policy-driven research while silencing more critical voices (e.g. Slater, 2012). This, in turn, has fuelled concerns that efforts to promote research impact in the UK are, in effect, directing researchers towards the production of ‘policy-based evidence’ (Cohen, 2000; Hammersley, 2005; Slater, 2012). These concerns have, however, been less evident in public health than many other areas of academia, with senior public health researchers often arguing (as we saw in Chapter 4 and 7) that a stronger relationship between research and policy is desirable and that this is likely to be achieved through closer working relationships between researchers and policymakers (e.g. Hunter, 2009; Whitehead et al, 2004; Wimbush et al, 2005). However, the failure to achieve key public health goals in the UK, such as a substantial reduction in health inequalities, despite efforts by researchers and policymakers to work more closely together, has prompted questions about this relationship and about the quality, focus and/or utility of available research (Mackenbach, 2011).

In this chapter, we begin by briefly reviewing two key studies exploring the relationship between evidence and policy for the longstanding public health concern with the cross-cutting challenge of reducing health inequalities in the UK, both of which included policymakers as research participants. These papers suggest that policymakers concerned with public health and health inequalities in the UK put forward relatively consistent proposals for improving the use of research in policy across both studies, despite a publication gap of nearly a decade. Next, the chapter compares the views of public health academics interviewed in 2003– 7, the run-up to the RAE in 2008 (before the emergence of research impact) with the views of academics working in the same field in 2011– 15, who experienced REF2014 and the first attempt to assess impact case studies (additionally making brief reference to the perspective of interviewees based in policy organisations). This comparison shows that, in this relatively applied research field, an initial scepticism about the likely influence of efforts to promote and reward research impact gave way to a diversity of views, stretching from enthusiastic support (for academics who feel far more enabled to undertake external-facing work than they once did) to pragmatic acceptance.

The understanding of any academic phenomenon would be incomplete without an exploration of its disciplinary character. Yet much of the existing published literature assessing the UK's approach to research impact focuses on single disciplines or areas of research (e.g. Dunlop, 2018; Greenhalgh and Fahy, 2015; Haux, 2018; Pettigrew, 2011; Watermeyer, 2014; Smith and Stewart, 2017), institutional research groups (Biri et al, 2014) or particular professions (e.g. Kelly et al, 2016; Marcella et al, 2018). In particular, there is a high number of studies relating to the research impact agenda that focus on health (Haux, 2018). Such a strong disciplinary bent in the existing analyses fails to capture, or allow comparative consideration of, the diverse ways in which impact and impact initiatives are experiences across different disciplines. This chapter starts to address this gap by focusing specifically on the role that disciplinary training, preferences and context appear to play in perceptions and experiences of the UK's impact agenda.

The limited existing literature that has explored experiences and perceptions of the UK's impact agenda across disciplines has reached mixed conclusions. A recent survey, which focused on research-active staff at a single UK institution (UWE), found only nuanced differences between academics based in different disciplines, concluding that ‘much can be shared and learned by and amongst researchers in different settings regarding impact best practice and its evidencing’ (Wilkinson, 2019: 83). In contrast, an interview-based study with academics in the UK and Australia found that disciplinary orientation can ‘dictate the type of activity associated with [impact]’ (Chubb, 2017: 22). For example, Chubb found that participants working in arts and the humanities emphasised the ‘public good’, a sense of the general value of research work, public engagement activities and cultural enrichment, while participants working in life and earth sciences emphasised policy impacts, practical improvements and working with industry, participants based in physical science, maths and engineering emphasised society, applied/useful research and an economic contributions, and, finally, participants in the social sciences stressed policy impacts, the direct use of research, societal change and broader engagement.

Another interview-based study with academics working in seven different disciplines in the UK found that ‘norms and traditions shared within disciplines, fields and modes of enquiry frame what can count as worthwhile “impacts” and as desirable or acceptable impact-oriented activities for individuals and groups’ (Oancea, 2013: 248).

As we have seen, there appears to be a basic assumption within the UK's approach to research impact (see Chapters 1, 2 and 3) that the incentives for the engagement between researchers and research users lead to an increase in research impact. Consequently, recent decades have witnessed the expansion of funding initiatives aimed at supporting and incentivising knowledge translation. One of the key UK research council strategies to address concerns about research utility and impact has been to fund knowledge broker initiatives aimed at bridging the gap between research and application and increasing engagement between academic and non-academic audiences. These organisations and posts are predominantly located at universities and have employed mainly academics, although their remit goes beyond what would traditionally be considered academic work and includes developing practices aimed at supporting evidence-based policy and practice.

This approach to increasing evidence uptake in policy and practice is, at least to a degree, reflected in the recommendations from research use literature (see Choi, 2005; Oliver et al, 2014; Cairney and Oliver, 2018). This strand of scholarship argues that one of the barriers to research uptake is the academic epistemic culture, which incentivises research that is decontextualised, theoretical and not necessarily relevant to practical problems (D’Este et al, 2018). From this perspective, the impact agenda might be viewed as offering a crucial opportunity to diminish key barriers to research use by promoting the activities that help ensure there is less of a gap between user preferences and research practices; in effect, promoting new priorities, incentives and values in UK academia. How the impact agenda is actually being experienced by those at the ‘vanguard’ of these changes is, however, underexplored.

In this chapter, we start to explore this gap by focusing on academics working in two specially designated and funded university-based knowledge exchange organisations that were focusing on policy and practice audiences (see Chapter 1 for a more detailed overview of the research data). These organisations have both been involved in work to promote knowledge exchange and achieve research impact for over a decade. We view these organisations as positioned at the forefront of the wider changes that the impact agenda is encouraging across academic institutions in the UK.

In this chapter we consider early and evolving rationalisations, the conceptual underpinnings and historical antecedents of an impact agenda in the UK. We note also the salience and exportability of the UK's impact agenda as a policy with growing appeal and traction across the international higher education community.

The emergence of research impact as an idea

The emphasis on research impact has been increasing steadily in the UK since the late 1990s (Cabinet Office, 1999), intensifying in a context of growing frustration that, despite apparently mutual political and academic interest in strengthening the links between research and policy, the actual use of evidence in policy remained limited (e.g. Katikireddi et al, 2011; Husband, 2016; Naughton, 2005). Academic critiques of policy often position responsibility for this dissonance on the policy side, noting the availability of relevant research that appears to have been ignored. Yet, the persistent sense of a large gap between research and policy has also served to intensify longstanding policy concerns about the lack of ‘return on investment’ from publicly funded research in UK universities.

As Chapter 1 outlines, one means of addressing this concern is that research impact now forms a significant section of grant application processes for all the major UK funding councils. This means that academics are required to set out how the work they will undertake will benefit non-academic audiences and what they will do to help achieve this in advance of undertaking the research, in the knowledge that this part of the application informs reviewer assessment. Meanwhile, in the run-up to 2014, it was decided that the system for nationally appraising university research, REF, would award 20 per cent of overall scores to institutions on the basis of impact case studies. This is rising to 25 per cent in the upcoming REF (REF2021, 2017). Hence, obtaining core research funding (largely distributed on the basis of REF scores) and project-specific research funding from UK research councils are now both strongly dependent on academics’ abilities to respond adequately to questions about the broader (non-academic) value of their work. These changes represent the contemporary backdrop for this book.