Layard (2005), in what I believe is an important book, maps out the ‘new science’ of happiness, a state associated with, but not the same as, good health. Ultimately, policies for neighbourhoods should be aiming at happy neighbourhoods as well as healthy ones. Step outside the front door and there are things to be happy about rather than things that are troubling or stressful. These may be a job to go to; a place for children to play and teenagers to hang out; neighbours that we trust and spend some time with; contact with nature and trees; an environment that is looked after, reflects well on us and is free from hazards; shops and services that are reachable; participation in decisions that affect us and our neighbourhood; and, above all, a place that we want to be. This does not mean that the neighbourhood necessarily makes a great difference for many of us, but that the more it constrains, stresses or disables us the more likely it is that our health will be damaged. Neighbourhoods, therefore, are a legitimate object of policy. They are systems that among their many inputs may sometimes need policy interventions.

Amin (2002) suggests that we think of places as nodes in networks of relationships that situate practices in space. I have gone further than this to argue that residential places are systems. Amin argues that practices may or may not reinforce local connections so that the place itself has agency. He plays down any idea that places have a special agency in themselves. As nodes of situated practices they are less obviously agents than firms, governments, social movements or other types of organisation. Practices in a place will often have much more significant relationships with networks beyond that place than with the place itself, so ‘places are more than what they contain, and what happens in them is more than the sum of localised practices and powers, and actions at other ‘spatial scales’’ (Amin, 2002, p 395). However, Amin does not deny the significance of proximity and particularity in space, both because levels of government are organised territorially, and because these characteristics engender political and social movements and local action of various kinds. But he reminds us that this action, although situated locally, engages with relational networks beyond any single place.

Lupton (2003, p 1) comments that, ‘Rarely has the neighbourhood enjoyed as high a profile in public policy as it does today’. She continues with a list of the many area-based initiatives (ABIs) introduced by Labour governments since 1997. In England, the focus on neighbourhoods in strategies to tackle deprivation has mainly been concerned with improving the delivery of public services in deprived areas so as to achieve measurable improvements in key outcomes across health, employment, education, housing and crime. Similar approaches exist in Scotland, Wales and Northern Ireland, but currently with less emphasis on performance assessment and management. While there are substantial additional resources for these programmes, they also emphasise news ways of working that promote partnerships and community engagement.

Neighbourhood renewal: its targets and performance

England's National Strategy for Neighbourhood Renewal was launched in 2001. It is one of the most significant of a wide range of social policy initiatives launched by the New Labour government after the Conservatives were defeated in 1997, and which has included both ABIs and national programmes such as the drive to eliminate child poverty (HM Treasury, 2005). The programme was shaped by a series of reports from 18 Policy Action Teams, which concluded that past initiatives had failed sufficiently to tackle the problems of depressed local economies and the safety and stability of local communities, or to improve public services for the most disadvantaged residents and involve residents in decisions affecting them (SEU, 2001). They also crucially lacked effective local leadership, partnership working and information.

These last issues reflect the wider shift in public policy that has been occurring under New Labour, and indeed began earlier in the 1990s under the Conservatives. This has been away from government through blueprint and direct service delivery dominated by public sector professionals, and towards governance through networks that bring different public, private and voluntary/community actors together to agree, plan and deliver programmes that often cross organisational boundaries (Stoker, 1998; Kooiman, 2003). This is an agenda that is very much about achieving the delivery of measurable outcomes (hence the emphasis on information), but not prescribing in detail organisational arrangements on the ground. The emphasis is on working differently, and tackling joined-up problems with joined-up solutions (Cabinet Office, 1999). Local actors are given space and incentives to realise the policy intent from the centre by developing the concrete organisational arrangements locally.

To care for places, and to realise the potential of people, are intrinsically good things to do. This book is an investigation of how well we are doing them when it comes to renewing disadvantaged neighbourhoods and reducing preventable causes of illness and death. By renewing neighbourhoods it is possible to tackle inequalities in health that have something to do with where people live. With the arrival in 2001 of England's National Strategy for Neighbourhood Renewal we can investigate whether this possibility is starting to be realised, given that this marked ‘a much firmer statement of the desirability of a more spatially equal society than governments have ever voiced before’ (Kintrea and Morgan, 2005, p 13). If, as Marmot (2004, p 9) remarks, ‘health can be used as a marker of a successful society’, then health outcomes across society are both an object of, and a judgement on, such a strategy.

Neighbourhood renewal and tackling health inequalities are interventions in ‘wicked problems’; issues that are ‘hard to treat’ not just because they are complicated but because they are complex. In the audit culture of contemporary public policy, the delivery of programmes to tackle these issues is performance managed. This needs to embrace the whole system in which the issues are embedded and assess how far inequality gaps are closing. The major test of these programmes is whether they improve conditions in the most disadvantaged areas at a faster rate than conditions improve for the population as a whole. These challenges of auditing change in whole systems and narrowing gaps as measured by ‘floor targets’ present a research agenda that currently is catching up with policy innovation rather than leading it.

As chapters in the book argue, complexity theory offers a conceptual framework for making more progress on this research front. The theory is worth serious attention among social scientists and policy makers, and in some quarters is getting it. Complexity is ‘out there’ as a real phenomenon and there is something extraordinary about the apparent growth of complexity across time as if ‘built into the structure of the world’ (Gregersen, 2003, p 228). For a while it was Ragin's (2000) book on Qualitative Comparative Analysis (QCA) that gave me a framework for moving on from complexity theory to actual empirical and policy implications, and Ragin remains a big influence.

The Black report, published in 1980, is often cited as the key reference point for when health inequality was placed on the national policy agenda in the UK (Black et al, 1980). Its recommendations, however, were rejected as too costly by the then Conservative government which the previous year had replaced the Labour administration that had commissioned the report. Health inequality, though, would not go away as an issue, with continuing pressure from public health movements such as the WHO's ‘Health for All’ campaign and Healthy Cities Project (Ashton and Seymour, 1988). Nevertheless, public health policy under the Conservatives concentrated on specific disease prevention strategies rather than health inequalities or the wider determinants of health beyond individual lifestyles. Targets for reducing both major diseases and individual risk factors were set out in England's Health of the Nation strategy (DH, 1992). The strategy acknowledged ‘health variations’ that were linked to socioeconomic factors but essentially dismissed the issue as not well understood.

Labour returned to power in 1997 and, in social policy, developed alongside a number of new national measures a swathe of ABIs aimed at tackling ‘social exclusion’, including the NR strategy and HAZs (SEU, 2001; Barnes, M. et al, 2005). The new government commissioned an update of the evidence on health inequality from a working group led by Professor Donald Acheson, which reported in 1998 (Acheson, 1998). This recommended evaluating all public policies for their impact on health inequalities, prioritising the health of families with children because of the significance of early-life influences on health, and taking further measures to tackle health inequalities and improve the living standards of poorer households. The report was followed by a new national strategy for England, Saving Lives: Our Healthier Nation (Secretary of State for Health, 1998) and in Scotland and Wales, which from 1998 had devolved governments with responsibility for health policy, by Towards a Healthier Scotland ( Secretary of State for Scotland, 1999) and Better Health: Better Wales (Secretary of State for Wales, 1998). Targets were set in these strategies for reducing death rates from cancer, CHD and stroke, accidents and mental illness. The strategies also recognised the importance of tackling social, economic and environmental factors, and of partnership working between the NHS, local government and the voluntary and community sectors.

It is often argued that communities of propinquity, neighbourhood communities, have declined as people's work and consumption patterns have become wider and more fragmented (Gilleard and Higgs, 2005). This transition has been described in various ways, such as from Fordism to post-Fordism, from ‘organised’ to ‘disorganised’ capitalism, or from modernity to late modernity or post-modernity (Lash and Urry, 1987). In the UK, this new phase space is post-industrial. But although the occupational basis for a neighbourhood identity such as that of a pit village or shipyard community may have largely disappeared, neighbourhoods still often have a material basis in consumption terms. The collective consumption of shared space also takes on a special meaning when it is centred on the home, a place of emotional, financial and labour investment, and where a sense of personal identity and value may be very influenced by how much the neighbourhood is in demand as a place to live. The neighbourhoods of consumer society may be different from the neighbourhoods of industrial society, and part of people's wider and more numerous networks in a more complex society, but they remain meaningful systems, and systems that have effects.

Neighbourhood effect studies

Community studies based largely on qualitative methods have a long history of exploring the distinctiveness of ‘places’ and testing general theories of social change by ‘grounding’ them in specific places (Crow, 2002). In recent years, there has been an expansion of quantitative investigations of whether variations in outcomes such as educational attainment, voting, health status, children's behaviour and domestic violence can to some extent be explained by variation between small-scale geographical contexts once differences between individuals are controlled for statistically. This statistical modelling separates ‘individual’ from ‘neighbourhood’ effects, and in general suggests a much larger effect on health of individual-level variation than neighbourhood-level variation (see Kawachi and Berkman, 2003, for a comprehensive review). Even so, spatial strategies such as ABIs may be able to achieve an appreciable global health gain if they can achieve an improvement in health across large numbers of people.

The evidence about only a small compositional neighbourhood effect is not conclusive. A study by Leyland (2005), for example, suggests that it is area deprivation rather than individual occupational class that is associated with socioeconomic gradients in CVD conditions.

An introduction to complexity theory

This book argues that improving health in neighbourhoods can be informed by using complexity theory. The reason is that neighbourhoods and their ‘states’ are outcomes of complex causation.

A relationship whereby A causes B in a linear and mechanical fashion is simple (echoing the last chapter's point about ‘tame’ problems). Many such relationships operating together is complicated. Complexity arises when there is interaction between many elements, such as the relationship between A and B depending on interactions with C, D or E (a ‘wicked issue’ in policy terms). When this happens, emergent and difficult-to-predict properties can arise from the interactions.

While traditional quantitative research in the social sciences is concerned with analysing relationships between variables, such as the relationship between income inequality and rates of premature mortality, complexity theory focuses on cases as empirical and actual domains. The interest is in the ‘states’ of these cases; not so much how and why variables change but how and why cases change. There is a direct concern with context in this respect, which variables-oriented quantitative research tends to regard as a difficulty to be controlled for, rather than part of explanations. The nature of cases as configurations of compositional and contextual attributes that combine to produce emergent outcomes can be described and investigated in either qualitative or quantitative terms (Byrne, 2002).

Cases, whether individuals, households, neighbourhoods, organisations or economies, are conceptualised within this theoretical framework as complex adaptive systems with self-organising behaviour. They are open to their environment and therefore affected by it, and have characteristics that are often fuzzy, not least their boundaries. Fuzzy in this context means that cases are not necessarily crisply defined in terms of having or not having key attributes (key attributes being those that are causally linked with outcomes of interest). A case may be in one attribute category say 80% and in another say 30% (Ragin, 2000). Examples of neighbourhood attributes in this respect could be segregated/mixed, safe/unsafe or green/barren. The important thing about the extent of ‘in or outness’ is its causal significance in terms of changes in state.

Feedbacks and interactions: going upstream

Qualitative change in a system can arise from internal agency, an external intervention or more typically a mixture of both. At any given time, however, environmental parameters present limits to what is possible in a given phase space. Within these limits a variety of outcomes are possible, with local agency making a difference. The limits can also change with a phase transition. Take the example of a social housing estate in England. This housing tenure is nationally regulated in ways that create a defined space of possibilities, with limits defined by access to the tenure depending on housing need. Within these limits the range of possibilities for the local system state depend on the amount of local demand for social housing, the demographic composition of this demand, and the local response. These conditions can mean that a social housing estate is a settled, stable and popular place to live or an unstable and unpopular neighbourhood. In England tenants have the right to buy their rented properties and, as already discussed, many regeneration initiatives aim to diversify the tenure of social housing estates with more privately owned housing. At a certain point this extension of owner-occupation may change the parameters governing the state of the area and a phase transition to another state may occur; one in which new parameters such as mortgage interest rates take the place of previous parameters such as rent and subsidy policies. One consequence of such a change is that the neighbourhood may become more settled, with less turnover, but social housing opportunities will have been reduced.

Interactions between the agents in a system and with their environment have the potential to produce non-linear change or transitions to different qualitative states. This is what is meant by emergence: the outputs of a system are not proportional to the inputs – as with linear change – but are new states arising from causal combinations creating a qualitative change. Relations between agents and with their environment are therefore important in understanding the emergent potential of a system – the positions in the phase space it could occupy – especially how feedbacks have either a stabilising or transformative effect. Negative feedback maintains a stable state during which there is little or only incremental change in response to an input.