Introduction

The What Works Centres as a whole make use of a wide variety of data to understand the world and the impact that policy or other interventions can have on it. However, in the majority of cases, this leaves ‘what works’ research, evaluations and synthesis focused on things that are easy to measure, or indeed, which are already measured through administrative datasets. For example, grades achieved, days spent in care, number of crimes in a particular area. This approach is rational and sensible, especially given constrained resources. Primary data collection – measuring something that is not already captured in administrative datasets – can double the cost of an evaluation.

This creates a trade-off for centres: do they measure what's easy, and miss out on the nuances of participant experiences, or do they measure what really matters, but find themselves able to run fewer studies overall. Most centres choose the former, and it is easy to see why. However, this runs the risk of missing out on important factors. If an intervention in education improves grades in maths through hot-housing (an intensive learning episode), but leaves students with low morale and wellbeing, it could damage their relationship with learning for the rest of their life. Is this intervention really a success? Successive governments have made a priority of adult learning, reflecting in part the failure of the mainstream education system. This work is made harder by the disaffection with learning felt by many adults – suggesting that a short-term boost to grades for some may have a longer, and more negative, impact for others (Hume et al, 2018). It is incumbent on the What Works Network to try and measure these longerterm, and more nuanced, outcomes.

Although the British government took steps to measure wellbeing as a national statistic, its lack of administrative records on the same means that wellbeing is too often a ‘soft’ or qualitative measure collected as an afterthought in what works type research. In this chapter we argue for a more holistic approach to administrative data collection, and for the What Works Network to be more ambitious in its data collection in the future.