3.1 Introduction

Conservationists have long had to deal with a number of prominent, recurring issues, such as habitat loss and fragmentation, pollution, invasive species and wildlife harvesting, to name a few. On top of these well-known challenges, others have emerged. Over the last half century, these have included the impact of halogenated pesticides and defoliants, acid rain from coal-fired electricity generation, ecological impacts of biofuel production and atmospheric releases of ozone-depleting chemicals. In more recent times, concerns have emerged around microplastics and exploitation of the Arctic, although some changes also bring opportunities for conservation, such as using mobile phones to collect data. New and emerging issues tend to make policy and practice more difficult. They add to an already challenging agenda, and often require a response when knowledge of the problem is limited.

Emerging from the relatively new field of ‘futures’ studies, horizon scanning is still developing as a method. By crowd sourcing information and drawing on communities of practice to sort, verify and analyse that information, horizon scanning offers an efficient way to look for early indications of poorly recognised threats and opportunities (Sutherland & Woodroof, Reference Sutherland and Woodroof2009; van Rij, Reference van Rij2010). It aims to minimise surprises by foreseeing these threats and opportunities, enabling policy-makers and researchers to respond quickly to developing problems. Horizon scanning is an approach primarily used to retrieve, sort and organise information from different sectors that is relevant to the question at hand, in a similar process to intelligence gathering. It can also include varying degrees of analysis, interpretation and prioritisation, but deciding which issues to act on, and how to act on them, typically takes place after the horizon scanning, and is assisted by other ‘futures’ tools, such as visioning, causal layered analysis, scenario planning and backcasting (e.g. Glenn & Gordon, Reference Glenn and Gordon2009; Inayatullah, Reference Inayatullah and Gutierrez Junquera2013; Cook et al., Reference Cook, Inayatullah and Burgman2014a). Recent frameworks have also been developed to link different futures tools, such as horizon scanning and scenario planning, together (Rowe et al., Reference Rowe, Wright and Derbyshire2017).

Horizon scanning outputs come in a wide range of forms. Some broadly describe a single trend that cuts across different parts of society, such as the rise of big data, or the future of a general area of interest, such as ‘Environmental Sustainability and Competitiveness’ (Policy Horizons Canada, 2011). These outputs are usually aligned with more general foresight programmes. Other exercises look at a set of more specific potential threats, such as invasive species that may arrive in the UK and threaten biodiversity (Roy et al., Reference Roy, Peyton and Aldridge2014), and compare them in an approach similar to risk assessment. For the last 10 years, conservation scientists have run annual horizon scans to identify emerging issues with the potential to impact global conservation (e.g. Sutherland et al., Reference Sutherland, Butchart and Connor2018). A similar approach has also been used to identify important scientific questions that, if answered, would help guide conservation practice and policy (e.g. Sutherland et al., Reference Sutherland, Adams and Aronson2009).

As with any policy advisory work, there is always a risk that useful information is gathered but not followed up, as decisions are often driven by other, usually non-scientific, factors. This risk may be higher with unsolicited (grassroots scans produced by a community of practitioners, researchers or academics) rather than solicited scans (called for by policy- and decision-makers). It can be unclear where the responsibility lies for integrating outputs into policy-making, and uptake depends on the organisational culture at the time (Delaney & Osborne, Reference Delaney and Osborne2013). Schultz (Reference Schultz2006) pointed to a conceptual contradiction between evidence-based policy and horizon scanning, where the latter searches for issues that may not be fully supported by a definitive body of evidence. A more optimistic perspective is that horizon scanning needs to be embedded in a broader strategic foresight framework, to increase the likelihood that findings are translated into practice (e.g. van Rij, Reference van Rij2010; Cook et al., Reference Cook, Inayatullah and Burgman2014a). As mentioned above, horizon scanning identifies emerging and novel threats and opportunities as a first step, but other foresight tools serve different purposes along the pathway to adopting appropriate policy. These other foresight tools are not explicitly covered in this chapter, but we provide an example, The Antarctic Science Scan and Roadmap Challenges Exercise, of a hybrid horizon scanning activity where an accompanying road map was also produced to outline actionable recommendations (Box 3.2).

In this chapter, we introduce both general and specific approaches to horizon scanning, outline some ways of achieving and measuring impact and explore how horizon scanning may evolve in the future.

3.2 Approaches to horizon scanning

‘Exploratory horizon scanning’ identifies novel issues by searching for the first ‘signals’ of change across a wide range of sources (such as an early scientific paper describing a potentially impactful new technology). ‘Issue-centred scanning’ monitors issues that have already been identified by searching for additional signals that confirm or deny that the issue is truly emerging (Amanatidou et al., Reference Amanatidou, Butter and Carabias2012). Signals can be organised into clusters (multiple pieces of information) that can either contribute to the evidence base around pre-identified issues, or form a long list of novel issues that are potentially emerging (Figure 3.1). The long list of issues can be further analysed and prioritised into a shortlist using methods detailed below. Some horizon scanning exercises take further steps to make the output more useful for the end user, for example, by assessing the policy relevance of the issues or the feasibility of addressing them, and by identifying those that warrant ongoing monitoring (Sutherland et al., Reference Sutherland, Allison and Aveling2012).

Figure 3.1 General framework for horizon scanning, reflecting the key steps in the procedure (ovals), inputs and products (rounded rectangles), key outputs (rectangles), actors and end users (triangles), and activities and methods (floating text).

Process adapted from Amanatidou et al. (Reference Amanatidou, Butter and Carabias2012).

There is a range of different ways to carry out horizon scanning; we introduce the main stages and provide some specific examples in the boxed texts and Table 3.1. Because our definition of horizon scanning concentrates largely on information retrieval, sorting and, to some extent, analysis and prioritisation, we focus here on methods that facilitate these activities.

3.2.1 Scoping

Like any major project, horizon scans need to be scoped and clear guidelines developed to assist scanners. A comprehensive scoping exercise addresses the following questions.

• What is the guiding question that defines what you want to know?

• How broadly or narrowly defined is the field of interest?

• What are the key drivers of change and activities in the field? It is common to organise thinking around a STEEP (Social, Technological, Economic, Environmental and Political factors) framework.

• What is the spatial scope? For instance, are you seeking issues with global or more localised impact?

• How far into the future should scanners be projecting?

• Who should be involved?

• Who are the potential end users?

Many of these considerations will be constrained by the resources available and the needs of the end user, but tools such as stakeholder analysis (Reed et al., Reference Reed, Graves and Dandy2009), domain mapping (Lesley et al., Reference Lesley, Floyd and Oermann2002) and issues trees (Government Office for Science, 2017) can be useful. Scoping exercises may also involve some pilot scanning to get a feel for how well-defined the task is. For example, preliminary scanning in a US Forest Service project that aimed to identify emerging issues that could affect forests and forestry in the future revealed that ‘natural resources and the environment’ was too broad a topic for their exercise. Instead, it was narrowed to ‘forests’ (Hines et al., Reference Hines, Bengston and Dockry2018).

Horizon scans that rely heavily on people rather than computers to do the scanning reflect the biases of those participants. A well-structured procedure for obtaining judgements from participants (e.g. Figure 3.2) will go a long way to mitigate psychological biases (Burgman, Reference Burgman2015b), but in order to capture a broad array of perspectives, involving a diverse group of people to identify and prioritise candidate issues is critical. A cognitively diverse group – comprising individuals who think differently – is thought to maximise collective wisdom and objectivity (Page, Reference Page2008). A good proxy for cognitive diversity is demographic diversity. Achieving demographic diversity can be challenging in practice. For example, there may be language barriers to overcome, and people with certain occupations (e.g. scholars) may be over-represented in horizon scans conducted by researchers. Inviting contributions from further afield, both geographically and from outside immediate peer circles, broadens the scope of issues considered. This might be achieved by putting out an open call for issues online and advertising it through relevant websites and email lists (e.g. Esmail et al., Reference Esmail, Wintle and Sas-Rolfes2019), or posting a call for ideas on social media.

Figure 3.2 The Delphi-style horizon-scanning approach often used in conservation (Sutherland et al., Reference Sutherland, Fleishman and Mascia2011).

Figure reproduced from Wintle et al. (Reference Wintle, Boehm and Rhodes2017), published under the Creative Commons Attribution 4.0 Licence.

3.3 Making a difference with horizon scanning

Gauging the extent to which horizon-scanning outputs inform policy, future research directions and resource investments is not always straightforward and no-one has yet tested the effectiveness of this process. In instances where the primary decision-making organisation uses horizon scanning internally to assist with deliberations (e.g. scans to set priorities for a government agency), actions can be mapped directly against outcomes. In these cases, implementing the actions indicates impact. In other cases, scans can be driven by a community outside of government to set agreed future directions that can then be used to persuade external resource allocators. Even in cases where policy appears to reflect issues flagged in a horizon scan, it is difficult to trace direct influence, as inputs from multiple sources are often blended in final policy decisions without attribution. It also may take years for real-world impact to be realised. Nevertheless, there are ways in which uptake of horizon-scanning output can be encouraged.

As a starting point, horizon scanning outputs can be matched to the organisations they are most relevant to. For example, policy-makers and practitioners can come together in a follow-up workshop to assess the importance of previously identified horizon-scanning issues for their organisation (Sutherland et al., Reference Sutherland, Allison and Aveling2012, Reference Sutherland, Fleishman and Clout2019). Or, the end user (e.g. policy-makers and practitioners) can be engaged in the horizon scan from the outset, as in a recent scan of research priorities for protected areas (Dudley et al., Reference Dudley, Hockings and Stolton2018). Similarly, horizon-scanning networks involving representatives from a range of government agencies, such as the Australasian Joint Agencies Scanning Network, or the UK Human Animal Infections and Risk Surveillance group, provide an ongoing forum for sharing information on new and emerging issues that potentially impact different departments and organisations. Regular meetings and reports are used to deliver this information to policy-makers in a timely way (Delaney & Osborne, Reference Delaney and Osborne2013).

In-depth follow-up analyses of horizon-scanning issues may also help policy-makers decide which to target first. A formal risk analysis of likelihood and consequences might be most appropriate for horizon-scanning outputs that compare similarly well-defined issues, for example, comparing one invasive species with another (e.g. Roy et al., Reference Roy, Peyton and Aldridge2014). It may be more challenging if some of the issues in the candidate set are more coarse-grained than others (e.g. comparing ocean warming with a specific emerging fungal disease in some snakes). Nonetheless, risk-based prioritisation at least offers a framework for comparing and forecasting issues (Brookes et al., Reference Brookes, Hernandez-Jover and Black2014) and for formally considering the strength of evidence for each (Garnett et al., Reference Garnett, Lickorish and Rocks2016).

Simply making horizon-scanning outputs known and available to policy-makers can encourage uptake. For example, issues identified in the annual global conservation scans (Box 3.1; Sutherland et al., Reference Sutherland, Butchart and Connor2018) have previously helped inform the UK’s Natural Environment Research Council ‘Forward Look’ strategic planning, but when a decision-maker does not already have a use in mind, it may be unclear what to do with horizon-scanning information without more context and guidance. Detecting signals and potential issues is only the first step towards making a difference: further intelligence about drivers is then needed to make sense of that information. For example, incorporating available data and modelling on air traffic movements with disease surveillance data might have helped anticipate the emergence of West Nile Virus in the United States in 1999 (Garmendia et al., Reference Garmendia, Van Kruiningen and French2001; Brookes et al., Reference Brookes, Hernandez-Jover and Black2014). It is the combination of horizon scanning, intelligence analysis (which provides context for the scanning output) and forecasting the chances of events unfolding that is particularly helpful in translating scanning outputs for policy-making. This can be embedded in a workflow, parts of which can be automated, such as compiling the context, narrative and structure into a digestible report on an important emerging issue (e.g. Box 3.3). When forecasting and open-analysis communities are already in place, this workflow can be delivered efficiently (Grossel et al., Reference Grossel, Lyon, Nunn, Robinson, Burgman, Nunn and Walshe2017).

Horizon scanning that occurs within organisations is evolving into a more effective tool than it was in its infancy. To facilitate the spread of best practice and reduce duplication, the UK has seen greater integration of horizon-scanning activities between different government departments, mainly in response to the Day Review (Reference Day2013). The review recommended that horizon scans: (i) look beyond short-term agendas and parliamentary terms, (ii) focus on specific areas rather than broad topics in order to get more traction, (iii) are championed by those who use them in strategic decision-making, (iv) produce shorter outputs that are more likely to get the attention of senior decision-makers and (v) draw on inputs and existing analyses sourced from a ‘wide range of external institutions, academia, industry specialists and foreign governments’. The extent to which all these recommendations have been implemented is unclear, but they represent a clear set of guidelines to follow.

There are a range of other useful frameworks that can be used for translating scanning outputs including roadmapping the steps towards acting on different horizon-scanning issues, for example, by assessing the feasibility and estimating how long it would take to develop technologies needed to address particular research gaps (Box 3.2; Kennicutt et al., Reference Kennicutt, Chown and Cassano2015). The Antarctic science scan and roadmap has since been used to set National Antarctic Program goals, judge the effectiveness and relevance of past investments, and guide investment of other national programmes (National Academies of Sciences Engineering and Medicine, 2015; www.nsf.gov/funding/pgm_summ.jsp?pims_id=505320&org=OPP&from=home).

3.4 Future directions

We have discussed some of the pros and cons of different approaches to horizon scanning. If using a manual approach, structured methods are essential for mitigating the social and psychological biases that human horizon scanners are prone to, especially when forecasting complex and uncertain futures (Hanea et al., Reference Hanea, McBride and Burgman2017). Although historically it has been criticised for confusing opinion with systematic prediction (Sackman, Reference Sackman1975), an iterative Delphi-style approach offers the advantage of drawing on the collective wisdom of a group, while affording individuals the opportunity to give private, anonymous judgements and revise them in light of information and reasoning provided by others. Compared with other elicitation approaches, such as traditional meetings, the Delphi method has also been found to improve forecasts and group judgements (Rowe & Wright, Reference Rowe, Wright and Armstrong2001). Manual approaches could be further improved by making the search for issues more systematic. Semi-automated tools and AI will increasingly enable searches uninfluenced by the biases of the manual searcher. For example, the Dutch ‘Metafore’ horizon-scanning approach (De Spiegeleire et al., Reference De Spiegeleire, van Duijne, Chivot, Daim, Chiavetta, Porter and Saritas2016), developed in The Hague Centre for Strategic Studies, already uses some automated approaches to systematically collect, parse, visualise and analyse a large ‘futures’ database to complement their manual scanning.

Future horizon scanning and intelligence gathering may also see more open-analysis, ‘citizen science’ tools becoming adopted. While organisations are increasingly scanning open-source material (including news and social media), analyses typically remain internal (Grossel et al., Reference Grossel, Lyon, Nunn, Robinson, Burgman, Nunn and Walshe2017). This means the analyses are generally not available to external users in an unfiltered form or in a timely way, which is particularly important for risks such as disease spread. Governments may opt for confidentiality for both security and political reasons. For instance, negative public perceptions about a suspected emerging herpes virus in oysters might affect trade, which might delay the disclosure of this information by authorities, in turn delaying risk mitigation actions (Grossel et al., Reference Grossel, Lyon, Nunn, Robinson, Burgman, Nunn and Walshe2017). Intelligence tools (e.g. Box 3.3) that draw on a community of users to openly analyse news and information on potentially emerging issues offer more timely and transparent synthesis of information, which encourages more responsive decision-making. Examples of this can be seen in citizen science, for example where citizen volunteers have helped analyse satellite-based information in the wake of natural disasters to help emergency responders to rapidly assess the damage (Yore, Reference Yore2017). In conservation science, involving a broader community of people in a participatory process like open-analysis may also increase public support for science and the environment (Dickinson & Bonney, Reference Dickinson and Bonney2012). More open-source and open-analysis scanning tools in the future will also likely be complemented with better information visualisation and GIS (e.g. including maps that indicate where a relevant incident has taken place) (Dickinson et al., Reference Dickinson, Shirk and Bonter2012), not only for identifying novel issues and monitoring issues that are already emerging, but also for locating and efficiently communicating this information.

Advanced text analytics, including text mining, will also provide a more comprehensive and systematic approach to future horizon scans. Indeed, some horizon-scanning centres, such as Singapore’s Risk Assessment and Horizon Scanning programme, already use sentiment analysis – a way of computationally categorising subjective opinions expressed in text (e.g. positive, negative or neutral) – to uncover themes in content retrieved by their analysts. Even more sophisticated text analytics are becoming available, for example, to explore areas of disagreement, conflict or debate in the text of scientific literature to help track developments in science and technology (Babko-Malaya et al., Reference Babko-Malaya, Meyers and Pustejovsky2013). They can also be used to detect language expressing excitement about a new idea, and other indicators of emergence, such as the increasing use of acronyms and abbreviations indicating that the scientific community is beginning to accept a technology or idea as established (Reardon, Reference Reardon2014). Through automation, new computational tools have the capacity to process a massive volume of papers and patents to anticipate which developments will have the biggest impact in the future (Murdick, Reference Murdick2015). These advances in text analytics have recently led to the development of a particularly powerful open-source AI tool, Meta (https://meta.org/), to help biomedical scientists and funders to connect emerging research areas and potential collaborators and inform investment. Due to the complexity of emerging issues (and complex environment for machines to learn in), progress towards detecting issues effectively through AI is slow. Computers may never outperform humans at natural language understanding, but steady improvements in the technology, coupled with the speed at which text can be processed by computers – in a range of languages – will undoubtedly add value to horizon scanning in the future.

3.5 Acknowledgements

BCW was supported by the Centre for the Study of Existential Risk at the University of Cambridge and funded by the Templeton World Charity Foundation (TWCF). WJS is funded by Arcadia. Geoff Grossel provided useful insights into online scanning methods and applications.

4.1 Introduction

Does scientific evidence really matter in conservation? In this chapter we will argue that generating, collating and using scientific evidence is key to effective conservation, illustrated by a case study from our own work: how to get bats to safely cross roads. We tell the story of bat ‘gantries’ or bridges, and show what can go wrong in the absence of robust studies that test the effectiveness of conservation interventions. We will also discuss the importance of collating or synthesising multiple strands of evidence to identify the factors that make a conservation measure effective or ineffective, using a case study on underpasses under roads. Finally, we explore a key challenge – getting scientific evidence accepted and used routinely in conservation policy and practice.

Evidence takes a multitude of forms and can be defined in many ways, but in this chapter we will mostly use ‘evidence’ to refer to scientific tests of treatments or interventions, which compare the ‘treatment’ to a ‘control’ in some way and measure the effect quantitatively. We define evidence in this way as it is a broad description that can still address causality for interventions – did treatment X cause reaction Y? For example, it is not enough to know that some bats flew along bat gantries – we need to know, at a minimum, how many, and how many still flew low across the road. But more on that later.

4.2 Why do we need evidence-based conservation?

Modern medicine has many examples illustrating why the discipline needs a robust evidence base. However, basing medical treatments on scientific evidence was not always the norm. The use of randomised controlled trials to test medical treatments was initially considered unnecessary and unethical, and it was hotly contested. A good example comes from an early champion of evidence-based medicine, Archie Cochrane, who demonstrated that randomised controlled trials were necessary and that expert judgement alone could be flawed. In 1971, he presented preliminary results from a trial comparing home care for heart patients with care in the new Coronary Care Units (note that the findings may not be the same now). He had been criticised for risking the lives of patients allocated to the ‘home care’ group. What follows is in his own words:

Results such as these – where the preferred treatment of the time did not work, or actually made things worse – are used to demonstrate why scientific studies of impacts are important when treating people. A growing body of literature suggests that impact studies are also necessary for treating the health of the biosphere, although the ‘gold standard’ of randomised controlled trials is not always possible in this discipline (Pynegar et al., Reference Pynegar, Gibbons and Asquith2018). As we test more and more measures to conserve species and habitats, we find that many do not work. For example, studies have shown that widely used methods to make water voles move prior to building works were ineffective, risking accidental killing of the protected mammals (Gelling et al., Reference Gelling, Harrington and Dean2018); that reintroduction programmes of species from macaws (Volpe et al., Reference Volpe, Di Giacomo and Berkunsky2017) to tamarins (Beck et al., Reference Beck, Dietz and Castro1991) have resulted in high or total mortality for the released animals; and that artificial bat roosts built to replace those destroyed during building works often failed to attract any bats and, even when occupied, hosted about half the number of bats that the destroyed roost had (Stone et al., Reference Stone, Jones and Harris2013). These results underline the need to test conservation solutions and not to simply assume that good intentions will lead to good outcomes.

Our case studies focus on the environmental impacts of roads. Road construction has been shown to harm animals through habitat degradation, loss and fragmentation, direct mortality and barrier effects (Laurance et al., Reference Laurance, Goosem and Laurance2009; Benítez-Lόpez et al., Reference Benítez-Lόpez, Alkemade and Verweij2010; Rytwinski & Fahrig, Reference Rytwinski and Fahrig2012). Figure 4.1 summarises these cumulative impacts, which are likely to act at different rates and through a long extinction debt. Unfortunately, studies on a wide range of habitats and taxa from grasslands to vertebrates show that many road mitigation options simply do not work. A growing list of papers points to not only poor design of mitigation and monitoring, but a wider context of poor target setting, weak implementation, inadequate reporting and poor or absent enforcement (e.g. Rundcrantz, Reference Rundcrantz2006; Tischew et al., Reference Tischew, Baasch and Conrad2010; Beebee, Reference Beebee2013; Drayson & Thompson, Reference Drayson and Thompson2013; Villarroya & Puig, Reference Villarroya and Puig2013). We will not address all of these factors in this chapter, but they are important to consider when asking why ineffective measures have persisted for so long.

Figure 4.1 The multiple causes of bat population reduction by road construction and the delayed response (extinction debt).

Adapted from Forman et al. (Reference Forman, Sperling and Bissonette2003).

4.3 Case study: bats and roads

Why do we need mitigation for bats crossing roads? In the EU, all bat species have been protected under the EUROBATS agreement since 1994, in recognition of the declining populations of many species. As a consequence, whenever populations may be adversely affected by human activity, impact assessment and mitigation are a legal obligation. Over the last 10 years evidence for significant effects of roads on bats has grown and the need for effective mitigation has become increasingly evident (e.g. Altringham, Reference Altringham2008; Russell et al., Reference Russell, Butchkoski and Saidak2009; Lesiński et al., Reference Lesiński, Sikora and Olszewski2010; Berthinussen & Altringham Reference Berthinussen and Altringham2012b, Reference Berthinussen and Altringham2015). There are clear specifications within EUROBATS to mitigate against the impacts of roads on bats.

4.3.1 The need to test mitigation: bat gantries

The EUROBATS commitment to mitigate against the impact of roads on bats is very positive, but are the mitigation strategies being used actually working? Early studies assessed the use by bats of underpasses and overpasses primarily built for other purposes, such as to carry minor roads, footpaths or streams. If bats were seen near to these structures it was generally assumed that they were effective mitigation tools (Highways Agency, 2001, 2006; reports reviewed by O’Connor et al., Reference O’ Connor, Green and Wilson2011). Underpasses, culverts, footbridges and bridges for vehicles, all of various sizes, were widely adopted as mitigation solutions (Figure 4.2). Many were not subsequently surveyed for use by bats, or qualitative surveys were written up in often confidential reports. Many studies reported ‘use’ – small numbers of bats observed in underpasses or flying over bridges of various kinds, without reference to the number still crossing the road unsafely, or not crossing at all (see Highways Agency, 2006; O’Connor et al., Reference O’ Connor, Green and Wilson2011), and many lacked convincing definitions of use. This meant that future projects could not learn from the success or failure of previously built mitigation structures.

Figure 4.2 Two underpasses found to vary in effectiveness in guiding bats safely under roads. (a) An effective underpass on the A590, Cumbria, UK; (b) an ineffective underpass on the A66, Cumbria, UK. Boxplots show the number of bats crossing per survey using the underpass and crossing over the road above at safe and unsafe heights (above and below 5 m, traffic height). The variable success of underpasses underlines the need to understand the details of conservation interventions; in this example, the location of the underpasses impacted on how effective they were.

From Berthinussen and Altringham (Reference Berthinussen and Altringham2012b).

In addition to multi-use structures, some ‘bespoke’ structures were built and ‘bat gantries’ or ‘wire bridges’ (Figure 4.3) were widely adopted. Bat gantries were assumed to act as navigational aids to echolocating bats, encouraging them to continue using existing ‘commuting routes’ from roosts to feeding areas (which often follow linear features such as hedgerows) after road construction, but lifting them above the traffic. Ideally, crossing points should be built on known bat commuting routes determined by pre-construction surveys, as bats tend to be faithful to particular routes. However, many were built away from known bat commuting routes for engineering reasons, to fit in with landscape topography, to combine bat routes with minor roads or footpaths, or simply to reduce cost. It was assumed bats would find the new crossing points (Highways Agency, 2001), and in some cases new hedge planting was designed to guide them to these structures. In many guidance documents, environmental statements and mitigation plans it was implicit, or even explicit, that the bats would respond as predicted (Highways Agency, 2001; Limpens et al., Reference Limpens, Twisk and Veenbaas2005).

Figure 4.3 Two bat gantry designs: (a) wire mesh design on the A11, Norfolk, UK; (b) wire and ball design on the A590, Cumbria, UK. Boxplots show the results of surveys carried out to test the effectiveness of the gantries in guiding bats safely over the road. Data were recorded for the total number of bats crossing per survey, the numbers crossing at unsafe heights (below 5 m, traffic height) and the numbers using the gantry according to two definitions of ‘use’ (flying within either 2 m or 5 m of the wires above traffic height). The bat gantry story neatly demonstrates the need to test conservation interventions before rolling them out on a wide scale.

From Berthinussen and Altringham (Reference Berthinussen and Altringham2012b, Reference Berthinussen and Altringham2015).

In 2008, JDA was asked to provide evidence to a public inquiry for the effectiveness of these strategies (Altringham, Reference Altringham2008). No quantitative evidence was found to suggest that any of the strategies implemented were effective in protecting bats, particularly at the population level. However, neither was there evidence to suggest that they were ineffective. This prompted us to conduct our own research to determine the effects of roads on bats and the effectiveness of mitigation (Berthinussen & Altringham, Reference Berthinussen and Altringham2012a, Reference Berthinussen and Altringham2012b, Reference Berthinussen and Altringham2015).

In our research we emphasised the difference between qualitative assessments of the ‘use’ of a structure by a small number of bats and measures of effective protection at the population level. We also stressed that the number of bats present pre-construction is rarely assessed, meaning that post-construction bat numbers may already be a fraction of what was there before. We proposed two broad measures of effectiveness: (1) measurements of local bat activity and of the movement of bats along severed commuting routes before and after road construction, to assess landscape-scale impact and the permeability of new roads; and (2) measurements of the effectiveness of the crossing structures – the proportions of bats that use them to cross safely. Our research was limited by logistics to the second measure – do mitigation structures guide bats safely across roads?

The headline result was that ‘wire and ball’ bat gantries did not alter the behaviour of bats crossing roads – they were wholly ineffective (Berthinussen & Altringham, Reference Berthinussen and Altringham2012a; Figure 4.3b). This was a disturbing finding, as over the previous decade about 15 gantries had been built in the UK and continental Europe. Although our study showed that one design of bat gantry was ineffective, it was suggested that other designs would have greater success in guiding bats to fly at safe heights above roads. Our next study found that ‘wire mesh’ gantries (Figure 4.3a, of a different design to the ‘wire and ball’ structures) were equally ineffective (Berthinussen & Altringham, Reference Berthinussen and Altringham2015).

In summary, a mitigation measure widely used for over a decade was essentially untested and subsequently shown to be ineffective. This underscores the need for rigorous testing of the measures that we implement in the name of conservation. We also found (albeit based on a small sample size) that building all types of crossing away from known commuting routes, even with new planting to guide bats to them, was unsuccessful (Berthinussen & Altringham, Reference Berthinussen and Altringham2012a). This is important, as it shows that the location of mitigation measures is as important as the measure itself – effective measures need to be implemented with a good understanding of the local context. Furthermore, we found evidence that some underpasses were used by a high proportion of bats, and that the one green bridge tested in the UK – a large structure planted with trees, shrubs and ground cover – was used by over 90% of bats crossing the road in that area, suggesting that effective ways to allow bats to safely cross roads do exist.

4.4 Synthesising evidence

The bat gantry case study provides some insight into why we need to rigorously test the effectiveness of measures aiming to protect the natural world. However, this is just the first step towards implementing a truly evidence-based approach to conservation. The next step is to systematically bring together all the evidence, from many studies, on particular conservation measures. This approach is also borrowed from evidence-based medicine, where it has proven to be a lifesaver.

One of the most important developments in evidence-based medicine was the Cochrane Collaboration, an organisation set up to conduct systematic reviews of the scientific evidence on topics such as how well different treatments worked. In medicine – as in conservation – natural variation in populations means that it often takes large numbers of replicates for beneficial or detrimental effects to become apparent. Modern doctors, making potentially life-changing decisions, want to have the information on every study on a particular treatment to hand, not just the results from a single trial that may not be representative. The goal should be the same in conservation: to bring together all the evidence for an intervention to assess whether it works, whether it does harm, or whether it only works in certain situations or with certain variations of the intervention.

There are many examples of the importance of collating evidence in medicine. For example, a systematic review on cot death or sudden infant death syndrome (SIDS), using the studies already available in the 1970s, could have saved the lives of an estimated 60,000 babies. Due to a lack of evidence synthesis and an overreliance on expert opinion, medical practitioners advised parents to put children to sleep on their fronts until the 1990s, when studies and reviews led to the realisation that this sleeping position increased the risk of SIDS (Gilbert et al., Reference Gilbert, Salanti and Harden2005).

In conservation, collating or synthesising the data is as critical as it is in medicine (Sutherland et al., Reference Sutherland, Pullin and Dolman2004). While the most rigorous method, systematic review, is very important (see Chapter 7, the Collaboration for Environmental Evidence and Mistra EviEM), more cost-effective methods of collating the evidence may also be desirable in this underfunded discipline, where the evidence itself can be scarce and variable in quality (Sutherland & Wordley, Reference Sutherland and Wordley2018). ‘Synopses’ published by Conservation Evidence (www.conservationevidence.com) follow one such method, known as subject-wide evidence synthesis (Sutherland & Wordley, Reference Sutherland and Wordley2018). Researchers draw up lists of all the interventions that could benefit a given taxa or habitat, classified according to potential threats based on IUCN criteria (Threats Classification Scheme Version 3.2); the scientific studies for the effectiveness of each intervention are then collated and summarised. For example, we produced the Bat Synopsis (Berthinussen et al., Reference Berthinussen, Richardson and Altringham2013, updated 2019), which provides key messages and summaries of the relevant studies, to help conservationists see which interventions for bat conservation are likely to be the most effective, and under which circumstances. The summary of this synopsis in What Works in Conservation (Berthinussen et al., Reference Berthinussen, Richardson, Smith, Sutherland, Dicks and Petrovan2018) takes this a step further, by using expert scoring to categorise the interventions based on levels of effectiveness, certainty in the evidence available and potential harms.

The first Bat Synopsis (Berthinussen et al., Reference Berthinussen, Richardson and Altringham2013) listed 78 interventions that could be implemented to conserve bats, covering areas as diverse as logging, roost provision and wind turbine operation regimes. No evidence for effectiveness was found for 48 of the 78 interventions, many of which are used routinely in the UK and elsewhere. This does not mean they are ineffective, but simply that they had not been tested quantitatively when we checked the literature. For a further 12 interventions the evidence was too limited for assessment. This demonstrates the scarcity of experimental evidence for many possible management actions, severely limiting the ability of conservationists, ecological consultants, developers and government agencies to undertake evidence-based conservation or mitigation.

Of the 18 remaining interventions, 14 had some proven value as conservation tools for bats. These included using selective logging instead of conventional logging, turning off wind turbines at low wind speeds and minimising light pollution. An update to this synopsis was published in 2019 (Sutherland et al., Reference Sutherland, Dicks, Ockendon, Petrovan and Smith2019), expanding the list of interventions to 190 and adding new studies that were published in the intervening years. There are many interventions which have had valuable evidence added in this update, but we have not yet seen a shift to a majority of interventions being tested via multiple high-quality experiments.

4.5 Getting the evidence used

We are trying to bring this work, demonstrating the importance of generating and using evidence on the effectiveness of interventions, to as large an audience as possible, to ensure that those responsible for commissioning, designing, approving and testing mitigation structures are aware of it. The bat gantry studies have been reported in national newspapers, radio and television. This was achieved through press releases, by approaching media contacts directly and by being approachable when contacted. The work has also been reported in several books and papers (Altringham, Reference Altringham2011; Abbott et al., Reference Abbott, Berthinussen, Boonman, van der Ree, Grilo and Smith2015; Altringham & Kerth, Reference Altringham, Kerth, Voigt and Kingston2016; Sutherland & Wordley, Reference Sutherland and Wordley2017). JDA and AB have run seven workshops for practitioners on road mitigation measures for bats and talked at over 10 conferences in the UK and abroad. CFRW has mentioned this study in around 50 talks to conservationists and government agencies and used it as an example in an opinion piece on evidence use in conservation (Sutherland & Wordley, Reference Sutherland and Wordley2017). The Bat Synopsis and What Works in Conservation, which contains a summary of the Bat Synopsis, have also been widely promoted.

This awareness resulted in tens of thousands of views of the paper and relevant parts of the Bat Synopsis, and this exposure has translated into further successes. The impact of early work (Berthinussen & Altringham, Reference Berthinussen and Altringham2012a, Reference Berthinussen and Altringham2012b) led to a Defra-funded project to develop better mitigation monitoring protocols (Berthinussen & Altringham, Reference Berthinussen and Altringham2015) and a statutory conservation agency guidance note summarising the protocols. The approximately £1 M spent on bat gantries in the UK as of 2017 (Sutherland & Wordley, Reference Sutherland and Wordley2017) was brought up in the House of Lords by Lord John Krebs in January 2018, who used it to demonstrate why the UK government’s 25-year environment plan needed to explicitly commit to being evidence-based. Some road-building projects have taken heed of the evidence. The A40 Penblewin to Slebech Park Improvement in Wales opted to mitigate impacts on bats using underpasses of varying sizes, many built on known commuting routes, and funded more rigorous monitoring (Davies, Reference Davies2019).

However, not everyone is listening. Despite widespread reporting of the ineffectiveness of bat gantries in 2012, six gantries of a ‘wire-mesh’ design were built in Norfolk, England in 2014 at a reported cost of £350,000. These were probably planned before the 2012 paper was published, but plans were not modified in light of this study. In 2015 these gantries were also shown not to work (Berthinussen & Altringham, Reference Berthinussen and Altringham2015). Nevertheless, seven more gantries are under construction (as of 2018) at a cost of over £1 M on the North Norfolk Distributor Road (MacDonald, Reference MacDonald2014). In another example, the environmental statement for the proposed and controversial extension to the M4 across the Gwent Levels in Wales (Welsh Government, 2016) draws extensively on our 2015 Defra report (Berthinussen & Altringham, Reference Berthinussen and Altringham2015). However, it proposes numerous culverts for bats which, by the authors’ own admission, are almost all too small to be used by the target bat species. In addition, most will not be on known commuting routes.

Furthermore, there are still inadequate mechanisms in place to assess the effectiveness of mitigation measures. A feature of many environmental statements and mitigation plans is the absence of a monitoring plan capable of assessing mitigation success or failure. There is frequently no monitoring plan at all. This appears to be due to a reluctance or inability of government agencies to enforce effective monitoring, a reluctance on the part of many developers to pay for monitoring and a lack of understanding about how to design and conduct monitoring that is fit for purpose. As a result, developers and taxpayers spend money on unproven mitigation with no prospect of improved understanding.

4.6 How can evidence use in mitigation projects be improved?

First, there should be a key requirement that mitigation structures are tested for effectiveness, not just use, and a quantitative bar set for effectiveness (Berthinussen & Altringham, Reference Berthinussen and Altringham2015). Ecologists employed to assess mitigation effectiveness must be prepared to shun options proven to be ineffective. Professional bodies must fully support ecological consultants in implementing those measures shown to be effective, and sanction members who use methods known to be ineffective. Improvements may be much more evident if the enforcing statutory agencies are willing and able to deny planning permission to development projects that have poor mitigation strategies. There should be real commitment from governments to pledge to conserve species and habitats using evidence-based measures and discarding measures proven to be ineffective. This may require additional resources to assess existing and proposed legislation against evidence syntheses.

To identify effective and ineffective solutions there is a clear need for dedicated funding for rigorous tests of interventions. Monitoring interventions often requires long-term commitment which, in turn, requires adequate long-term funding. This could come from developers and government agencies, but a greater recognition by academic funding bodies of the value of applied questions would also have a huge impact. PhD projects could be encouraged to have applied components, testing interventions. Research council funding for academics to address applied questions of conservation importance and communicate them to practitioners would have a huge impact.

Greater power for statutory agencies to enforce existing laws, check up on implementation and demand replacements for ineffective solutions would dramatically improve mitigation effectiveness. A framework with greater incentives for developers to show that their mitigation has been effective would be beneficial. These could include a requirement to make the results of mitigation monitoring for effectiveness public, penalties for failures to do so and awards for new, proven effective solutions.

While many of these goals may not be realised in the near future, we can all promote approaches to conservation that are evidence-based and effective. If enough of us do it, it might just change the world.

5.1 Introduction

Most of the chapters in this book focus on how best to bring science into policy, often at the national scale and mostly with a developed-world perspective. Ensuring that national policy frameworks are conducive to conservation is vital, but it is also important to improve the effectiveness of science in supporting conservation interventions on the ground. Small-scale interventions aiming to change the behaviour of local resource users in developing countries make up a large proportion of global conservation effort and funding (Brockington & Scholfield, Reference Brockington and Scholfield2010). These types of intervention are challenging to do well, and often do not produce the desired results (Larrosa et al., Reference Larrosa, Carrasco and Milner-Gulland2016). Typically, there is little scientific input into either the design or evaluation of these projects, and evidence of effectiveness is limited (Roe et al., Reference Roe, Booker and Day2015). Small organisations in developing countries may not have the capacity or confidence to implement scientifically informed design and monitoring, and supporting them to collate and learn from evidence may not be a major priority for researchers or donors. Increased sharing of insights and techniques to support more robust and effective interventions could transform grassroots conservation (e.g. Woodhouse et al., Reference Woodhouse, de Lange and Milner-Gulland2016). In this spirit, we use case studies from four locations around the world to illustrate some of the challenging steps involved in understanding conservation issues and designing suitable interventions. These steps are often skipped or not made explicit, but are critical to success; they ensure that interventions have a strong foundation in evidence, making it more likely that their desired impacts are achieved.

First, we explore how to collect robust information on the prevalence of illegal resource use, as a first step towards understanding the extent of the problem, using a case study of bird hunting in a Cambodian grassland. Next, we consider how to bring together different sources of information to understand both resource use and local perspectives on conservation, using a case study from Tanzania. These two case studies about evidence gathering lead on to the next stage: intervention design. We start with an example, also from Tanzania, of developing a Theory of Change for a conservation intervention, in which the process by which actions lead to a desired result is identified, assumptions are made clear and the progress of the intervention towards its desired impact can be monitored. Finally, we explore the challenges of implementing one particularly prevalent intervention type – alternative livelihoods projects – using an example of a shark fishery in Indonesia. Together, these case studies provide a vivid illustration of the ways in which conservation researchers and practitioners are combining efforts to ensure that interventions are based on robust evidence and therefore more likely to succeed.

5.2 Asking questions about sensitive topics

Moderating human behaviour is critical to conservation success (Gore, Reference Gore2011; Milner-Gulland, Reference Milner-Gulland2012). However, if we are effectively to change human behaviour, we must first ensure we understand the nature of the behaviour we want to change. Central to this is determining both the prevalence of behaviours that are detrimental to biodiversity, and the characteristics of the people engaging in these behaviours. This is essential to ensure managers efficiently allocate resources to tackle threats, and that behavioural change interventions target the right audiences with the right incentives (St John et al., Reference St John, Edwards-Jones and Gibbons2010, Reference St John, Mai and Pei2015). However, obtaining such information can be extremely challenging, especially if the behaviour in question is illegal (Gavin et al., Reference Gavin, Solomon and Blank2010).

A common approach to ascertaining the true extent of illegal behaviours is asking direct questions (e.g. Gandiwa, Reference Gandiwa2011; Kiffner et al., Reference Kiffner, Peters and Stroming2015). Other studies mask the sensitivity of questions about illegal behaviours by mixing them with less-sensitive questions about other livelihood activities (e.g. Martin et al., Reference Martin, Caro and Mulder2012; Mgawe et al., Reference Mgawe, Mulder and Caro2012; Kiffner et al., Reference Kiffner, Peters and Stroming2015). Although direct questioning may help to cast some light on the nature of natural resource exploitation, it runs the risk of bias from untruthful responses (Nuno & St John, Reference Nuno and St John2015). Respondents may be scared to answer questions honestly for fear of incriminating themselves, or the possible repercussions they might face from revealing their behaviour. They may avoid answering questions altogether, terminate interviews early or underreport activities. If respondents do answer sensitive questions, social desirability bias may lead them to moderate their responses so their actions appear more socially acceptable. This is especially true of data captured in group settings, where pressure from peers may prevent others speaking freely and truthfully about their activities. It is also important to consider the ethical implications of directly asking respondents about their illegal activity; research has an ethical responsibility to ‘do no harm’, yet asking such questions can cause respondents to directly implicate themselves in illegal activities, potentially leading to severe consequences.

Indirect questioning has started to become more widely used in conservation science in response to some of these challenges. The method comes from psychology, and has been used when asking questions about sensitive issues such as drug use and racial prejudice (Imai, Reference Imai2011). The technique enables interviewees to respond in such a way that the interviewer cannot directly determine whether they have participated in the activity. Instead, data provide estimates of prevalence at the population level, affording both the respondent and the researcher greater levels of protection.

One form of indirect questioning increasingly applied in conservation is the Unmatched Count Technique or Item List Technique (see Gavin et al., Reference Gavin, Solomon and Blank2010; Nuno & St John, Reference Nuno and St John2015). The technique works by devising a short ‘control’ list of three to five innocuous items that are non-sensitive but relevant to the research topic, and a treatment list which also contains the sensitive item of interest (Figure 5.1). The sampled population is randomly shown either the control or treatment list. Respondents are asked to report only the total number of items that apply to them. Because only a number is reported, the researcher has no way of knowing which specific items apply to a given respondent. The difference in the mean number of items reported by the two groups provides an estimate of the proportion of respondents engaging in the sensitive behaviour (Thomas et al., Reference Thomas, Gavin and Milfont2015).

Figure 5.1 Using the Unmatched Count technique to ask about illegal bushmeat hunting in the Ugalla Wildlife Reserve, Tanzania. Picture by Paulo Wilfred.

5.3 Triangulating different sources of evidence to build a rounded picture

Social research methods such as focus groups, interviews and household surveys are increasingly being employed to investigate illegal behaviours and profile resource users (Young et al., Reference Young, Rose and Mumby2018). The current decade has seen an increase in the use of these mixed methods approaches to gain a more holistic understanding of resource use (e.g. Kahler & Gore, Reference Kahler and Gore2012; Harrison et al., Reference Harrison, Baker and Twinamatsiko2015). A combination of perspectives, using both qualitative and quantitative methods, is commonly preferred.

5.3.1 Case study: Ugalla Game Reserve

Ugalla Game Reserve (hereafter Ugalla; 5000 km²) in western Tanzania is predominantly miombo woodland. Its conservation value is high, serving as habitat for a wide range of species (UGR, 2006). It is part of the Malagarasi–Muyovosi Ramsar Site, and facilitates connectivity between protected areas in western Tanzania (Kalumanga, Reference Kalumanga2015; Riggio & Caro, Reference Riggio and Caro2017). The main legal activity in the reserve is trophy hunting, mostly by overseas tourists. A number of different approaches are used to conserve Ugalla, including irregular anti-poaching patrols and seasonal permission for fishing and beekeeping activities (July–December). These also aim to attract local support for conservation and build a sense of ownership of the reserve among local people. However, recent studies suggest that this conservation approach is ineffective (Wilfred & MacColl, Reference Wilfred and MacColl2015; Wilfred et al., Reference Wilfred, Milner-Gulland and Travers2017). Unauthorised use of natural resources (including poaching, illegal logging and fishing) is common and local communities hold negative attitudes towards the reserve and its management. In an attempt to shed light on the prevalence of illegal behaviours and inform the management of Ugalla, multiple research methods were used to gather relevant information. Between 2013 and 2016, household surveys and focus groups were conducted in villages around Ugalla, along with a survey of signs of illegal activity undertaken across the Protected Area.

For the household surveys, 533 households were randomly sampled in 2016 in the vicinity of Ugalla. The Unmatched Count Technique was used to estimate the prevalence of illegal behaviours (logging, illegal hunting and honey-gathering). The survey also included questions on households’ perceptions of the main threats to Ugalla and its wildlife, and what communities would do differently to improve Ugalla’s management effectiveness. Six single-sex focus groups from six randomly selected villages within 20 km of the Ugalla boundary, each with 4–6 participants, were conducted to verify findings from the household survey. Free-listed threats to Ugalla were ranked in decreasing order of their importance, and each threat was then divided by the total number of threats to calculate the salience score (Papworth et al., Reference Papworth, Milner-Gulland and Slocombe2013). The overall score for each threat was obtained by calculating the average salience score across the focus groups. The greater the salience, the more important the threat.

For signs of illegal activity in the reserve, 10 patrol tracks were randomly selected in 2014. Six transect starting points were placed at 3000-m intervals along each road. At each point, two 1500-m transects were walked on opposite sides and perpendicular to the road. Signs of illegal activity (e.g. tree stumps, sawpits, meat smoking racks, snares, trees felled for honey extraction, fish smoking racks, poacher camps) were noted 50 m either side of the transect line (Figure 5.2).

Figure 5.2 Paulo Wilfred and his research assistant recording an illegal meat smoking rack in Ugalla Wildlife Reserve.

The Unmatched Count Technique results suggested that poaching and illegal logging were performed by 28% (SE ± 6) and 20% (SE ± 5), respectively, of surveyed households; 18% (SE ± 6) of respondents gathered honey. The top four threats to Ugalla, as identified by respondents, were poaching (40% of respondents), logging (39%), fishing (11%) and honey gathering (8%). Of the top four threats to Ugalla free-listed and ranked by focus groups, logging had the highest salience (S = 0.5), followed by poaching (0.45). Within the reserve, 867 illegal activity signs were encountered. Signs related to logging had the highest frequency, followed by honey gathering, poaching and fishing (Figure 5.3). These results indicate that levels of illegal activity in Ugalla are high. The different methods consistently suggest that logging and poaching are the commonest illegal activities.

Figure 5.3 Signs of illegal activity encountered inside Ugalla Game Reserve in 2014. Total signs = 867.

Of the activities that survey respondents and focus group participants said they would undertake if they were the Ugalla manager, the most common recommendations were to: improve the well-being of people around Ugalla (17% of respondents, S = 0.11); ensure that local people have adequate land for their livelihood activities (16%, S = 0.35); promote local participation in conservation (16%, S = 0.13); improve law enforcement (15%, S = 0.14); raise conservation awareness (15%, S = 0.14); and improve local people’s relations with reserve managers (12%, S = 0.54).

5.4 Developing a Theory of Change for an intervention

It is vitally important to be clear about why we think that our intervention is the right thing to do, and what barriers there might be to success, before we start. This understanding needs to be set out in a logical way, so that it is understandable and appealing to project staff and donors, and so that it can later be tested. There are a number of approaches which can be used, falling under a general heading of causal chain models (Qiu et al., Reference Qiu, Game and Tallis2018). One such approach is Theory of Change (Center for Theory of Change, 2018), which shows how a project can reach its desired impact and goals through different pathways of change. It provides indicators that can be tested, thereby supporting evaluation of a project’s success or failure. This is useful both for internal and external users, to understand what works, and to guide the allocation of project resources.

5.4.1 Case study: Vijana na Mazingira

In 2016, Hans Cosmas Ngoteya designed a retrospective Theory of Change for Vijana na Mazingira (VIMA), the local conservation project which he runs in the Katavi–Rukwa ecosystem of western Tanzania. The project targets youths aged 12–35, with the goal of reducing pressure on natural resources from poaching, deforestation and encroachment. The Theory of Change was designed to support an evaluation of the effects of the project on attitudes, awareness and conservation behaviours by youths aged 18–35 participating in VIMA’s conservation education and alternative livelihood projects (Figure 5.4).

Figure 5.4 Hans Cosmas Ngoteya (second from right) setting up a beehive with local youths, as an alternative livelihood project.

In order to achieve a project’s desired impact, it is necessary first to understand the motivations for engaging in the behaviour that the project is aiming to modify. There are a number of frameworks available from social psychology that represent the factors that interact to influence behaviours. One of the most widely used in conservation is the Theory of Planned Behaviour (St John et al., Reference St John, Keane and Milner-Gulland2013). Hans used the Theory of Planned Behaviour to identify the different factors underlying the motivations of the VIMA project’s recipients (Figure 5.5). Based on Hans’ local knowledge and understanding of the project, a Theory of Planned Behaviour framework was developed for four desired project impacts, each of which represented a desired behavioural change. The Theory of Planned Behaviour was then used to identify how VIMA’s activities might tackle the different motivations underlying each behaviour.

Figure 5.5 A Theory of Planned Behaviour diagram illustrating the factors underlying the poaching behaviours of individuals targeted by the VIMA project.

A clear understanding of the motivations behind the behaviour, engendered by the Theory of Planned Behaviour exercise, can enable conservationists to map out the pathways of change the project should focus on, thereby generating a Theory of Change. The Theory of Planned Behaviour gives a representation of what underlies an individual’s behaviour, and this can be used to develop a Theory of Change for the planned intervention. In this case, the Theory of Planned Behaviour exercise highlighted that, typically, youths in Katavi–Rukwa viewed poaching as a way to feed their families and generate an income through bushmeat or ivory sales. Therefore, an intervention that developed alternative livelihood programmes could be an effective approach. This could include training youths in new income-generating activities (input), thereby providing alternative income sources (output), which will reduce their dependence on natural resources (outcome) and ultimately reduce their poaching behaviour (impact; Figure 5.6). At each step of this pathway lie assumptions; for example, that any alternative income source will replace, rather than supplement, income from hunting (Table 5.1).

Figure 5.6 Theory of Change for VIMA project showing interventions at the bottom and different pathways to reach the desired impacts. Numbers 1–10 are assumptions along the pathways of change (listed in Table 5.1).

Table 5.1 Assumptions underlying the Theory of Change

1.	Participants understand the education they are given
2.	If someone is educated about environmental issues it will improve their attitude towards conservation
3.	Knowledge about conservation issues leads to a decrease in acceptance of environmentally harmful behaviours
4.	There is dissemination of information from VIMA participants to the remainder of the community
5.	If someone’s attitude towards conservation improves, they will reduce their unsustainable resource-use behaviour
6.	If communities are against unsustainable resource use, illegal resource exploitation will decrease
7.	VIMA’s alternative livelihood programmes can be put into practice and generate income
8.	There is opportunity for the rest of the community to become involved in the alternative livelihood projects
9.	Alternative livelihoods will be used to reduce unsustainable use of natural resources
10.	Decreasing dependency on natural resources will reduce poaching, encroachment and deforestation

Baseline surveys, focused on the elements of the Theory of Planned Behaviour (attitudes, knowledge, social norms), provide a set of indicators against which change engendered by the intervention can be measured. Progress through the Theory of Change can also be monitored, using a set of more process-based indicators. For example, an input indicator might be the percentage of VIMA’s target audience engaged in the alternative income activities, an output indicator might be the income generated from the alternative livelihood, the outcome might be measured as improvements in household livelihood security and the impact might be measured using an indirect questioning technique such as the Unmatched Count Technique to quantify change in poaching prevalence.

5.5 Exploring alternatives to illegal behaviour

One of the key lessons learnt in the VIMA project was the importance of having a clear understanding of the motivations behind behaviour. Unfortunately, not all conservation projects that involve communities take this approach when designing an intervention. For example, alternative livelihood projects have long been used as strategy for reducing local threats to species, habitats or resources of conservation concern. Alternative livelihood projects are designed to reduce the prevalence of behaviours that are considered environmentally damaging and unsustainable (Wright et al., Reference Wright, Hill and Roe2016). However, a systematic review of alternative livelihood projects conducted by Roe et al. (Reference Roe, Booker and Day2015) found insufficient evidence to understand when, where or why alternative livelihood projects work. Even though there is uncertainty regarding the effectiveness of alternative livelihood projects, they continue to be a key strategy in both terrestrial and marine conservation. However, the assumptions on which they are based are often unrealistic; for example, that the alternative livelihood projects will substitute for the undesirable behaviour, that the resource users are a homogeneous group, and that targeting interventions at individuals will scale up to population-level change in pressure on resources (Wright et al., Reference Wright, Hill and Roe2016).

In marine conservation, a common response to perceived over-fishing is to provide alternative employment for existing fishers. This requires that the assumption of substitutability holds, so that fishers will willingly and happily settle into a new way of making a living (Pollnac et al., Reference Pollnac, Pomeroy and Harkes2001; Pollnac & Poggie, Reference Pollnac and Poggie2008). Pollnac et al. (Reference Pollnac, Pomeroy and Harkes2001) added that this is based on the assumption that fishing is a hard and undesirable occupation and hence an employment of last resort, that fishers are among the poorest of the poor and that the poor care little about the type of job they have as long as they make enough to live.

5.5.1 Case study: shark fishers in Tanjung Luar

Fishing pressure is generally considered to be the main cause of the decline of shark populations globally (Stevens et al., Reference Stevens, Bonfil and Dulvy2000; Robbins et al., Reference Robbins, Hisano and Connolly2006; Dharmadi et al., Reference Dharmadi and Satria2015). Indonesia is the world’s largest shark producer, with annual average production of 106,000 tons in 2000–2011, contributing 13% of global shark production (Dent & Clarke, Reference Dent and Clarke2015). Although the exact number is unknown, it is assumed that many Indonesian fishers are heavily dependent on shark fisheries as a source of income and food. However, shark production in Indonesia has been declining in recent years (Sub Directorate of Capture Fisheries Data and Statistics, 2016), which could be leading to a decline in income and livelihood security for fishers.

From 2014 to the present, the Wildlife Conservation Society (WCS) Indonesia Programme has carried out a study of shark fishers in Tanjung Luar, a shark-fishing community in East Lombok, in order to understand whether providing alternative livelihoods could help to reduce fishing pressure on sharks. Tanjung Luar is one of the main shark landing sites in Indonesia. It is home to a targeted shark fishery comprising approximately 50 boats employing surface and bottom longlines and one of the biggest fish markets on Lombok Island, with more than 5000 fishers using it to sell their catch. Fishing is the main livelihood of Tanjung Luar’s population and there are at least 150 households heavily dependent on the shark industry, either as fishers, meat processors or traders. Shark fishers in Tanjung Luar use 4–25 gross tonnage boats, with three or four crew members, and the average fishing trip is 14 days.

Due to growing international concern regarding their conservation status, several shark and ray species have been listed on CITES Appendix II. As a CITES member, Indonesia is required to implement management measures, such as quotas, size limits and export bans to ensure that international trade in these species is not detrimental to wild populations. These measures could have negative impacts on the income and livelihood security of Tanjung Luar’s fishers, who are already vulnerable to market fluctuations, particularly in export markets (Jaiteh et al., Reference Jaiteh, Loneragan and Warren2017). WCS Indonesia Programme’s study aimed to: (1) collect data on biological and operational characteristics of the fishery (Figure 5.7), (2) understand shark fishers’ current socioeconomic status and aspirations, (3) explore alternative livelihood options and (4) create dialogue between fishers and the management authorities.

Figure 5.7 WCS Indonesia team members measuring guitarfish at Tanjung Luar port.

Photo provided by WCS-Indonesia.

Livelihoods options explored with the fishers included diversifying the target catch to more resilient species (e.g. squid, tuna and reef fish) and tourism, yet WCS Indonesia Programme’s surveys showed that shark fishing offered higher revenues than other fisheries. An independent fisheries assessment by Masyarakat dan Perikanan Indonesia also showed similar results (MDPI, 2017). Tanjung Luar was known for its squid fishery in the 1980s, but the number of squid fishermen has increased rapidly, increasing competition and making the addition of new fishers unsustainable (MDPI, 2017). Some fishers in Tanjung Luar who catch tuna or skipjack mentioned that their catch is also declining, and The Indian Ocean Tuna Commission classifies yellowfin tuna as overfished (IOTC, 2017). Some shark fishers have already started to fish for groupers and snappers on the side, but the value of this catch is far less than their earnings from sharks. Tourism is promising, but the industry is still under-developed. To date, identifying feasible alternatives that provide economic incentives to shift away from shark fishing has proven challenging, as there are no legal or sustainable marine alternatives that offer similar profits.

Our research showed that shark fishers wish to remain shark fishers. Fishing is the only skill they know, and most of them said that they would continue to fish as usual even if their catch declined by 50%. Our landings survey showed that some commonly caught sharks are over-exploited. When findings were shared with fishers, although not all agreed with the results, shark fishers acknowledged that it is now harder to catch sharks and the sharks that are caught are smaller, a view also shared by shark fishers in eastern Indonesia (Jaiteh et al., Reference Jaiteh, Loneragan and Warren2017). The Tanjung Luar fishers’ response is not surprising, as similar reactions were also reported by Pollnac and Poggie (Reference Pollnac, Pomeroy and Harkes2006), with fishermen refusing to leave their existing fishery even though their incomes were declining; it is potentially their best option in the short run if they are still making a profit.

5.6 Discussion: interlacing research and practice

The four case studies presented here take us from research to practice; in so doing, they illustrate how integrated the two are. By starting with a strong theoretical framework (such as the Theory of Planned Behaviour) underpinning an intervention’s Theory of Change, unwarranted assumptions can be avoided, such as those which plague alternative livelihoods projects. Engaging with resource users before embarking on interventions can reveal dead ends, as illustrated in Tanjung Luar, where plans for an alternative livelihood project needed to be replaced by a more indirect process of advocacy and engagement with different parties, while building capacity for a livelihoods shift. A clear understanding of what the actual problem is, based on evidence rather than supposition, is vital; the example from Cambodia suggested that hunting was actually not a major threat to floricans, enabling conservation practitioners to focus on other threats.

Although a range of techniques is available for collecting information to underpin management, these should not be applied lightly. As the Ugalla example showed, the ideal of using mixed methods to gain a nuanced understanding takes time and resources, as well as expertise. Approaches such as the Unmatched Count Technique can look superficially appealing and easy to administer, but there are technical challenges in developing appropriate item lists, administering the questions in a way that makes respondents comfortable, and in data analysis. Even then, as the Cambodian example shows, the results may not be as informative as might be hoped. Time invested in foundational studies is well spent, but not all small NGOs can afford extensive research. Even then, however, it is possible to develop a robust Theory of Change, as a tool for exposing assumptions and supporting ongoing monitoring and evaluation, as the VIMA example showed.

Our case studies have specific lessons, but they also tell universal stories. The role of research in facilitating positive interactions between managers and local people is an interesting observation that was seen in both Ugalla and Tanjung Luar, while both the Cambodian and Ugalla case studies highlighted the importance of good local research assistants. All four case studies emphasised how research and practice need to intertwine more often and more routinely. This will enable conservationists (whether from governments or NGOs) to think through their interventions in advance, use appropriate methods to understand existing behaviour and local perspectives on ways forward, and thereby design locally appropriate, participatory interventions that support adaptive management.

6.1 Introduction

Rapid and interlinked changes in the biosphere, including degradation of the biodiversity and ecosystems that underpin human well-being, are reported with increasing regularity. As such, there is an urgent need for conservation initiatives that are capable of countering the speed and veracity of change, while meeting the needs of human societies on a crowded planet. While significant advancements in scientific knowledge in the fields of sustainability and conservation continue to be achieved, the forecasted rate of rapid ecological and social change requires the production of innovative mechanisms for management and policy.

One way of contributing to new solutions in a timely manner is to more effectively mobilise multiple knowledges, values and governance systems that can complement Western approaches to science. Together these can extend the collective knowledge base and contribute to collaboratively designing ways forward for looking after people and the biosphere. Compared with Western-based approaches, indigenous and local knowledge systems represent alternative ways of learning from and with the environment, through close and continuous observation framed by distinct worldviews with particular strengths and limitations (like all knowledge systems). Knowledge is embodied by the actors and in their practices, tools, and technologies, as well as in the institutions that organise the production, transfer and use of knowledge (Cornell et al., Reference Cornell, Berkhout and Tuinstra2013). There has recently been more attention focused on the urgent need for science and policy to recognise and mobilise the knowledge of indigenous people and local communities who steward substantial biodiversity across the globe (Brondizio & Le Tourneau, Reference Brondizio and Le Tourneau2016; Mistry & Berardi, Reference Mistry and Berardi2016). Collaborative ways for mobilising knowledge and learning across diverse knowledge systems can contribute complementary knowledge, innovations and new solutions. Involvement of multiple actors and knowledges can strengthen usefulness and legitimacy in decision-making and implementation (Sterling et al., Reference Sterling, Betley and Sigouin2017a; Gavin et al., Reference Gavin, McCarter and Berkes2018).

In this chapter, we draw attention to the potential for mobilising local and indigenous knowledge systems, institutions and actors in ways that allow meaningful use of their knowledge about landscapes and their functions as evidence for conservation. By doing this, we propose that innovative and collaborative mechanisms can be designed and implemented that will create opportunities for long-term sustainable governance and conservation of biodiversity.

We introduce the Multiple Evidence Base (MEB) approach to guide the design and implementation of conservation partnerships that enable engagements with indigenous and local knowledge as evidence as an entry point to promote sustainable governance of interrelated ecosystems and human well-being (Tengö et al., Reference Tengö, Brondizio and Elmqvist2014, Reference Tengö, Hill and Malmer2017). The approach was developed to guide inclusive processes for collaborations across knowledge systems, based on equity and usefulness for all actors involved. It emphasises that indigenous, local and scientific knowledge systems are complementary, equally valid and useful for informing sustainable governance of biodiversity and ecosystems. The MEB focuses on the theoretical and practical potential for collaborative knowledge-weaving processes to mobilise indigenous and local actors, institutions and practices to achieve long-term conservation and sustainability targets. We argue that collaborative approaches to conservation must be equitable and fair to be effective in the long term (Brondizio & Le Tourneau, Reference Brondizio and Le Tourneau2016; Sterling et al., Reference Sterling, Betley and Sigouin2017a; Gavin et al., Reference Gavin, McCarter and Berkes2018).

The utility and value of the MEB approach will be discussed in light of its aim to support more informed and efficient local, national and international policy processes and governance decisions for the integrated benefits of conservation, sustainable use and human well-being. We describe the current and potential role that a MEB approach may have in enhancing the efficacy of conservation science and policy by clarifying and strengthening synergies with indigenous knowledges and practices. To achieve this, we first review the peer-reviewed and grey literature to reflect on the extent of uptake of the MEB and how it has been applied in both science and policy-practice processes. Second, to illustrate the approach and reflect on successes and practical challenges, we take a deeper look at three case studies of piloting a MEB approach. The cases demonstrate the potential for the MEB approach to be used as both a framing tool for collaborative partnerships and a practical guide to weaving multiple knowledge systems. Lastly, we discuss ways forward to nurture conservation and mobilise partnerships that build on knowledge collaborations. We find that a MEB approach has potential to support the inclusion of a wider range of evidence in conservation practice, strengthen active participation of local actors and improve conservation partnerships through the recognition and revitalisation of local knowledge systems and governance.

6.2 The need for new approaches to collaborative conservation

There is a long history of attempts to reconcile conservation objectives with local livelihoods in integrated development and conservation processes, which have often been framed as ‘win–win’ opportunities with social–ecological benefits (Adams et al., Reference Adams, Aveling and Brockington2004). In the conservation literature, the importance of involving local people is well established, with mounting evidence that processes that meaningfully engage local people are more likely to succeed in protecting biodiversity (Waylen et al., Reference Waylen, Fischer and McGowan2010; Sterling et al., Reference Sterling, Betley and Sigouin2017a) and that failure to do so can lead to lack of trust and commitment, project failure, and in the worst case, lingering conflicts (Oldekop et al., Reference Oldekop, Holmes and Harris2015). While many indigenous peoples and local communities continue to be evicted from their ancestral lands and experience colonisation in the name of conservation, there is now a move towards recognising their connections to land and endogenous obligations to care for it as synergetic with biodiversity conservation outcomes (Knox, Reference Knox2017). This provides a foundation for enabling local people and conservation organisations to be strategic allies. Furthermore, there is increasing evidence that involving local actors in monitoring enhances management responses at local spatial scales, and increases the speed of decision-making to tackle environmental challenges at operational levels of resource management (Danielsen et al., Reference Danielsen, Burgess and Jensen2010; Sterling et al., Reference Sterling, Betley and Sigouin2017a).

Despite these generally acknowledged realities about the usefulness of engaging with indigenous peoples and local communities, they are often included as stakeholders in conservation, without recognition of their knowledge and expertise (Danielsen et al., Reference Danielsen, Burgess and Jensen2010). In the literature much attention is given to the uniqueness and utility of indigenous and local knowledge systems, which is often holistic, providing an understanding of integrated social–ecological systems, biocultural values and belief systems (Sheil et al., Reference Sheil, Boissière and Beaudoin2015; Sterling et al., Reference Sterling, Betley and Sigouin2017a). However, in practice, there often exists scepticism about the contemporary existence and/or effectiveness of indigenous and local knowledge as useful evidence in conservation. Similarly, holders of indigenous and local knowledge can be sceptical of the claims generated through western scientific approaches due both to the unfamiliarity of the epistemic practices employed and recent or ongoing experiences of colonisation and disempowerment (Nadasdy, Reference Nadasdy1999; Johnson et al., Reference Johnson, Alessa and Behe2015; Kealiikanakaoleohaililani & Giardina, Reference Kealiikanakaoleohaililani and Giardina2016; Mistry & Berardi, Reference Mistry and Berardi2016).

6.3 The multiple evidence base approach: connecting knowledge systems for the benefit of conservation and human well-being

The need to engage with diverse sources of knowledge for conservation has been recognised in high-level science–policy processes, such as the Convention on Biological Diversity, and the Intergovernmental Science Policy Platform on Biodiversity and Ecosystem Services (IPBES). From the outset, IPBES had the ambition to recognise and respect the contribution of indigenous and local knowledge to the conservation and sustainable use of biodiversity and ecosystems (Díaz et al., Reference Díaz, Demissew and Carabias2015). This was used as a window of opportunity to start an open dialogue to explore current divides between indigenous, local and scientific knowledge systems, and to elicit methods for collaborations based on equity, reciprocity and usefulness for all involved (see Tengö et al., Reference Tengö, Brondizio and Elmqvist2014). A science–policy–practice dialogue process brought together knowledge-holders and experts from diverse knowledge systems, convened by SwedBio at Stockholm Resilience Centre in collaboration with key partners representing indigenous peoples and local communities, such as the International Indigenous Forum on Biodiversity and the African Biodiversity Network. The active engagement from these networks, representing a diversity of knowledge systems and linking practices on the ground with global policy and science, created legitimacy and recognition of outcomes from the dialogues. The starting point was the pivotal dialogue meeting prior to the establishment of IPBES in the indigenous territory of Guna Yala, Panama, where essential principles for exchange across knowledge systems were identified: trust, respect, reciprocity, equity, transparency and free prior and informed consent (Tengö & Malmer, Reference Tengö and Malmer2012). Since then, the MEB approach has developed in parallel to the IPBES, while carefully paying attention to other interests and needs of the partners.

The MEB can be understood as a deep approach to collaborative knowledge-sharing that explicitly acknowledges that challenges are fundamentally due to different perspectives and practices concerning human–nature relationships, approaches to knowledge validation, knowledge governance and who qualifies as an ‘expert’. Also, it recognises that scientists have tended to dominate the design and implementation of collaborations across knowledge systems both historically and contemporarily (Nadasdy, Reference Nadasdy1999; Mistry & Berardi, Reference Mistry and Berardi2016). Another key component of an MEB approach is its emphasis on the need for mobilisation and validation of knowledge within knowledge systems themselves. That is, if scientific methods that often are specific and partial are applied to local knowledge that is practical, multidimensional and holistic, there is a risk of omission, misinterpretation and rejection of critical and useable knowledge.

The MEB approach views different knowledge systems as complementary and emphasises that joint analysis assists in working both with convergence and divergence (e.g. Molnár et al., Reference Molnár, Kis and Vadász2016a; Hohenthal et al., Reference Hohenthal, Räsänen and Minoia2018). For example, Molnár et al. (Reference Molnár, Kis and Vadász2016a) highlight that when discussing approaches to conservation in the Hungarian steppe, local herders focus on primarily utilitarian purposes, such as how they can manage the behaviour of their grazing animals in order to promote the health and diversity of grass assemblages for production. In comparison, conservationists working in the same landscapes focus almost solely on the protection of the plants themselves, with little regard to the impact on grazing animals. If this difference is ignored, or framed as a problem, it has the potential to create tension when attempting to collaboratively design and implement conservation initiatives in the region. Conversely, these different perspectives can be worked together to provide an enriched picture of exactly what is necessary for maintaining and enhancing biodiversity and social–ecological system function in the steppe.

In order to build evidence – whether new knowledge or existing – that is legitimate and useful for all actors in such collaborations for conservation, there is a need to engage with local knowledge systems and knowledge-holders from the outset, co-defining a common problem and facilitating equitable engagement through all activities, including mobilising and assessing knowledge. This process is outlined in the three phases of the MEB approach (Figure 6.1a). Collaboratively analysing and interpreting the complementary evidence from diverse sources is a way to triangulate information, strengthen legitimacy and relevance of existing knowledge and build a base for further learning.

Figure 6.1 The Multiple Evidence Base approach in action. (a) The three phases of a MEB approach: joint problem formulation, generating an enriched picture with contribution from multiple sources of evidence and joint analysis and evaluation of knowledge (Tengö et al., Reference Tengö, Brondizio and Elmqvist2014). (b) Actors, institutions and processes are at the core of the five tasks required for successful collaboration across diverse knowledge systems. The different colours of the lines and dots in parts (a) and (b) represent different knowledge systems, or streams of knowledge within knowledge systems (Tengö et al., Reference Tengö, Hill and Malmer2017).

As guidance for how to implement an MEB approach, five tasks were identified as critical (Figure 6.1b; Tengö et al., Reference Tengö, Hill and Malmer2017). First, to mobilise knowledge – to ensure that the knowledge is articulated, validated internally and free to be shared with others. Second, to translate knowledge, reciprocally, so that all actors can comprehend each others’ knowledge and where it is derived from. Third, to negotiate, to jointly address convergence and divergence between knowledge systems, and the extent to which the latter can be resolved, for example by understanding differences in underlying assumptions and values (Gagnon & Berteaux, Reference Gagnon and Berteaux2009; Molnár et al., Reference Molnár, Kis and Vadász2016a). Fourth, to synthesise. Here we emphasise synthesis based on a joint process that does not require that all knowledge is validated by one knowledge system (e.g. empirical validation by science). Lastly, to apply – and this is where we iterate the need to recognise the different needs and interests by different actors. Knowledge collaborations need to be designed in a way that is is perceived as useful and leads to constructive outcomes for all involved. The bridging of knowledge systems therefore requires the creation of settings for exchange of multiple forms of knowledge and learning across key aspects of the system (Figure 6.1b). We view the outcome as weaving – knowledge collaborations that respect the integrity of each knowledge system while working them together for practical collaboration (Johnson et al., Reference Johnson, Howitt and Cajete2016; Tengö et al., Reference Tengö, Hill and Malmer2017). In the next section, we use literature and our own experience to evaluate and discuss implementation of the MEB approach, with a specific focus on describing the outcomes in terms of evidence applied in conservation partnerships.

6.4 Reviewing the impact of the MEB in conservation and sustainability

The literature on knowledge collaborations for conservation and sustainability is wide-ranging. To focus on collaborations across knowledge systems (indigenous, local and scientific knowledge systems) and to generate further insights into the application, challenges and usefulness of a MEB approach, we reviewed articles that cite Tengö et al. (Reference Tengö, Brondizio and Elmqvist2014) or that mention ‘Multiple Evidence Base’ in the academic literature, represented by Scopus (123 articles), and the grey literature (219 results), represented by Google Scholar (as of 2018–02-01).

The results of this review demonstrate that the MEB approach has contributed to a general move towards broader participation of knowledge-holders in multi-level ecosystem assessments (Díaz et al., Reference Díaz, Demissew and Carabias2015; Nesshöver et al., Reference Nesshöver, Vandewalle and Wittmer2016), as well as citizen science, the importance of the plurality of knowledge systems in conservation (Prado & Murrieta, Reference Prado and Murrieta2015) and knowledge application in public policy and resource management (Bruckmeier, Reference Bruckmeier2016). This is part of a ‘shift that has occurred in the science–policy–society interface with a move towards greater inclusivity, and efforts to transcend traditional reductionist approaches’ (Jabbour & Flachsland, Reference Jabbour and Flachsland2017, p. 196).

The MEB approach is finding traction in diverse discussions including citizen science (Buytaert et al., Reference Buytaert, Zulkafli and Grainger2014) and community-based monitoring (Johnson et al., Reference Johnson, Alessa and Behe2015; Lyver et al., Reference Lyver, Timoti and Jones2017), collaborative management and decision-making (Mathevet et al., Reference Mathevet, Thompson and Folke2016), community-based conservation (Nkambule et al., Reference Nkambule, Buthelezi and Munien2016; Sterling et al., Reference Sterling, Betley and Sigouin2017a), measuring resilience (Quinlan et al., Reference Quinlan, Berbes-Blazquez and Haider2016; Sterling et al., Reference Sterling, Ticktin and Kipa Kepa Morgan2017b), approaches to modelling global change processes (Verburg et al., Reference Verburg, Dearing and Dyke2016), indigenous autonomy and cultural revitalisation (Gonzales, Reference Gonzales2015), value pluralism in ecological economics (Martín-López & Montes, Reference Martín-López and Montes2015; Kenter, Reference Kenter2016; Pascual et al., Reference Pascual, Balvanera and Diaz2017), biocultural values and diversity (Gavin et al., Reference Gavin, McCarter and Mead2015; Sterling et al., Reference Sterling, Ticktin and Kipa Kepa Morgan2017b) and political ecology, law and environmental justice (Gambon & Rist, Reference Gambon and Rist2018; Hohenthal et al., Reference Hohenthal, Räsänen and Minoia2018).

The majority of articles reviewed (51 percent) engage with the MEB approach in a relatively superficial manner to illustrate that combining multiple knowledge systems is a sustainability challenge. The literature is awash with programmatic articles with calls to include, combine and integrate knowledges to find solutions to sustainability problems (e.g. Balvanera et al., Reference Balvanera, Calderón-Contreras and Castro2017; Vasseur et al., Reference Vasseur, Horning and Thornbush2017). However, still very little attention is paid to exactly how this will be done. Additionally, 20 percent of articles reviewed represent collaborative processes in practice but do not apply a MEB approach. Many articles view actors as stakeholders and talk about ‘open participation and open consultation’ (e.g. Livoreil et al., Reference Livoreil, Geijzendorffer and Pullin2016) rather than addressing their role as knowledge-holders and experts and the need for equitable platforms for engagement, mobilisation and translation of indigenous and local knowledge.

The MEB approach has also received significant attention in the grey literature and science–policy–practice community. For example, it is called for as a way of ensuring equitable participation for indigenous, local and scientific knowledge in monitoring of the Convention on Biological Diversity. For example the Convention’s Aichi target 18 on traditional knowledge, innovation and practices, along with the Community-Based Monitoring and Information Systems, is a bottom-up approach developed by indigenous peoples and local communities to contribute their experiences and observations through monitoring (CBD, 2014; Farhan Ferrari et al., Reference Farhan Ferrari, de Jong and Belohrad2015). Further, a MEB approach has been encouraged in traditional knowledge inventories, as well as in the development of safeguards for biodiversity financial mechanisms and Reducing Emissions from Deforestation and Degradation (REDD+) under the United Nations Framework Convention on Climate Change.

To illustrate the implementation of the MEB approach in the literature, we have selected a small set of pertinent case studies. Table 6.1 presents an analysis using key features of the MEB approach – joint problem formulation, validation within knowledge system and the five tasks illustrated in Figure 6.1b.

The cases illustrate that, in different contexts, specific phases of the MEB approach presented by Tengö et al. (Reference Tengö, Brondizio and Elmqvist2014, Reference Tengö, Hill and Malmer2017) are more or less useful, and are operationalised in different ways. The process of co-defining the problem and questions together with all knowledge-holders appears to be a challenge not taken up in all cases, often with scientists or project proponents defining a problem, and then approaching indigenous and local knowledge-holders and local communities through consultation sessions to join and support the collaboration (e.g. Strangway et al., Reference Strangway, Dunn and Erless2016; Lyver et al., Reference Lyver, Timoti and Jones2017; Smith et al., Reference Smith, Basu and Chatterjee2017). However, other papers do emphasise the critical role of joint problem formulation for the success of conservation interventions (Brondizio et al., Reference Brondizio, Foufoula-Georgiou and Szabo2016; Galvin et al., Reference Galvin, Reid and Fernández-Giménez2016).

Maintaining the integrity of diverse knowledge systems throughout collaborative knowledge processes also appears to be a particular challenge in science-driven processes. Actively thinking about what validation of knowledge within knowledge systems means (rather than using science to validate local knowledge) and how it may be embedded in practice is absent from most papers. There are notable exceptions that explicitly reflect upon this challenge (e.g. Austin et al., Reference Austin, Vigilante and Cowell2017) and suggest new approaches, such as peer-to-peer validation by farmers (Smith et al., Reference Smith, Basu and Chatterjee2017; Table 6.1). Other papers do not address this explicitly, but still engage with how local knowledge systems evaluate knowledge (e.g. through interactions with internally acknowledged experts and their local institutions) (Molnár et al., 2016; Nguyen et al., Reference Nguyen, Luom and Parnell2017; see Table 6.1). Additionally, joint discussion and analysis of data across knowledge systems has sometimes been incorporated through formal consultation structures or committees (e.g. Strangway et al., Reference Strangway, Dunn and Erless2016; Austin et al., Reference Austin, Vigilante and Cowell2017; Reed & Abernethy, Reference Reed and Abernethy2018). The articles also illustrate the progress in development of methods to facilitate the phases and activities defined in Tengö et al. (Reference Tengö, Brondizio and Elmqvist2014, Reference Tengö, Hill and Malmer2017) to combine and relate multiple data through e.g. participatory scenario planning, focus groups (Danielsen et al., Reference Danielsen, Jensen and Burgess2014), fuzzy cognitive maps and community monitoring with digital devices (Brammer et al., Reference Brammer, Brunet and Burton2016). The use of art (Rathwell & Armitage, Reference Rathwell and Armitage2016; Polfus et al., Reference Polfus, Simmons and Neyelle2017), participatory maps (Robinson et al., Reference Robinson, Maclean and Hill2016) or film (Molnár et al., 2016) to mobilise, translate and present knowledge on an equitable platform has facilitated joint analysis and negotiation. Articles also illustrate practical ways of maintaining equity, such as creating research agreements or protocols concerning intellectual property; free, prior and informed consent; the roles and responsibilities of each member of the project team (Robinson et al., Reference Robinson, Maclean and Hill2016); and recognising indigenous and local knowledge-holders as authors on scientific articles (Molnár et al., Reference Molnár, Kis and Vadász2016a; Smith et al., Reference Smith, Basu and Chatterjee2017; Table 6.1).

The citations suggest that the mobilisation and translation activities suggested by Tengö et al. (Reference Tengö, Hill and Malmer2017) have had particular resonance in the conservation and sustainability literature. There has been consistent progress towards the explicit mobilisation and translation of indigenous knowledge and worldviews (Gonzales, Reference Gonzales2015; Vogt et al., Reference Vogt, Pinedo-Vasquez and Brondízio2016; Horstkotte et al., Reference Horstkotte, Utsi and Larsson-Blind2017; Timoti et al., Reference Timoti, Lyver and Matamua2017). In this way, the mobilisation of multiple knowledge systems contributes to a movement towards environmental justice and pluralism in decision-making (Hohenthal et al., Reference Hohenthal, Räsänen and Minoia2018), as well as recognising indigenous peoples’ autonomous actions towards dealing with climate change (Gonzales, Reference Gonzales2015).

In the next section, we use three in-depth case studies to further explore the value of a MEB approach to contribute to conservation partnership based on diverse sources of knowledge.

6.5 Exemplifying MEB cases and reflecting on lessons learned

Here we present three case studies that have explicitly implemented a MEB approach (Table 6.2). The first two set out processes to address local conservation and development issues. The third is an international dialogue meeting where the aim was to create a platform to discuss a fundamental crux in conservation globally – how to realise synergies between human rights and biodiversity conservation, and support local people and conservation organisations in becoming strategic allies.

6.6 Sharing lessons from the three cases

In the first case from Tharaka River, the importance of mobilising indigenous and local knowledge as a solid base for translation and negotiation phases was very clear. This then helped people speak about their knowledge, and also catalysed the revitalisation of eroded institutions and rules that previously served to protect the river, including the recognition and protection of the sacred sites. As the problem formulation was owned by the community, this enabled articulation of the importance of the sacred sites for understanding previous river governance, and motivated local people to restore the river. Later, they contacted the Natural Museums of Kenya, to provide support in gazetting their biodiversity-rich sacred sites for formal national recognition. This illustrates the important role that values and beliefs in diverse knowledge systems can play for conservation, how they may be identified, and how knowledge and governance capacity is embedded in the belief systems.

In the case from the Kimberley, the use of the MEB and the role of indigenous and local knowledge in collaborative management, created space and enthusiasm for experimenting with new ways of combining knowledge systems for management and governance of Saltwater Country. The KISSP demonstrated that working with multiple knowledge systems and disciplines in the context of unequal power relations requires design, support and monitoring of mechanisms that can maintain constant dialogue (e.g. the KISSP working group). Thinking of the collaboration as ‘intercultural’ was useful for understanding what capacity development was required for all actors. No single party had capacity deficits, but the collective needed to build joint capacity for weaving knowledge systems in ethical and equitable ways.

In the Global Dialogue on Human Rights and Conservation, reaching synergistic solutions between conservation and human rights once again was about overcoming power imbalances. The dialogue was an opportunity for key actors with different knowledges, experiences, worldviews and power to meet in a neutral context. Diversity of experiences (positive and negative) across scales and a careful mix of actors helped to overcome these imbalances during the dialogue. Mobilisation of indigenous and local knowledge and strengthening confidence among participating community representatives, but also knowledge about human rights and other legal aspects before the dialogue, was critical for deliberations. The recognition of indigenous rights and the value of their knowledge and practices for conservation expressed by researchers contributed to trust followed by constructive proposals. Positive experiences from successful collaborations in conservation of indigenous lands contributed to exploring ways forward in cases where conflicts persist. The learning across different sectors and scales, such as ecologists learning about human rights aspects, was appreciated in the evaluations. It also became clear that the deepest conflicts may not relate to conflicting evidence from different knowledge systems regarding ecology, but to controversial policy, such as the eviction of people from conservation areas.

In all three cases, the main challenge of the collaborative process was to overcome power imbalances and build trust and confidence. The focus on recognising, mobilising and discussing evidence from diverse knowledge systems was an entry point that contributed to the development of strong collaborative partnerships. Designing a process that was considered useful for all involved was critical to securing successful and sustainable outcomes, new and useful ways to combine and apply knowledge from diverse knowledge systems, and sometimes the generation of new knowledge. In all cases, the aim of creating synergies across knowledge systems for providing evidence on sustainable governance could be realised when all holders of knowledge gained from collaborations. A MEB approach, on whatever level it is conducted, emphasises the importance of collaborative processes that value multiple knowledges and practices needed to sustain the social–ecological landscape to the double benefit of sustainable livelihoods and conservation over the long term. Further, the collaborative relationships of trust developed provide new opportunities to align multiple modes of governance of ecosystems, to ensure decisions and policy are based on all available knowledge.

6.7 Discussion

In this chapter, we review the use of one recent and important approach to combining the knowledge of indigenous peoples, local communities and scientists for sustainability and conservation partnerships. We have focused on the MEB and its potential for building more inclusive understanding of multiple sources of evidence, how it is generated and how it is transmitted among diverse conservation actors. We argue that such an approach is important for better understanding of interlinked social–ecological systems, strengthening conservation partnerships and identifying new evidence-based pathways towards sustainability. Our review and the three case studies show examples of different ways to move forward that recognise the complementarity and integrity of knowledge systems in addressing specific problems (Molnár et al., 2016; Smith et al., Reference Smith, Basu and Chatterjee2017), create conditions (and methodologies) for full and open dialogue on how to move ahead, overcome power inequalities and navigate cultural differences (Robinson et al., Reference Robinson, Maclean and Hill2016; Reed & Abernethy, Reference Reed and Abernethy2018). We demonstrate reciprocal synergies between indigenous and local knowledge and conservation science and rich cases of how cross-fertilisation leads to stronger partnerships and better outcomes. The three case studies also show that the MEB requires partnerships that are underpinned by recognition, respect and understanding of diverse knowledge systems, and that the process for producing and applying common knowledge to problems cannot be viewed separately from the outcomes of partnerships. That is to say, much like the concept of adaptive management in conservation, the diversity and dynamism of knowledge systems dictate that the process of collaboration be taken as seriously as the achievement of conservation outcomes themselves (Gavin et al., Reference Gavin, McCarter and Berkes2018).

More work is needed to further elaborate how to implement a MEB approach in different processes and contexts. The IPBES process has struggled with the tension between open collaboration and the demands for structure set by the scientific knowledge governance. There is yet some way to go to better acknowledge and solve epistemic challenges, such as diverse modes of validation across knowledge systems (Löfmarck & Lidskog, Reference Löfmarck and Lidskog2017; Obermeister, Reference Obermeister2017). There is also a need to continue developing tools and approaches for bridging knowledge systems that are connected to local, cultural, social and ecological conditions. Our review illustrates that indigenous peoples, local communities and scientists have begun to tackle this challenge (Molnár et al., 2016; Robinson et al., Reference Robinson, Maclean and Hill2016; Smith et al., Reference Smith, Basu and Chatterjee2017), but further dialogue is required, both horizontally across local scales and vertically through local to global institutions.

We have shown that a MEB approach has been particularly effective in dialogues where there are power imbalances among actors and historical bias concerning the validity or usability of knowledge systems other than western approaches to science (see also Klenk & Meehan, Reference Klenk and Meehan2015). Building trust and respect is especially pertinent in the context of ongoing and historical injustices and abuse of indigenous rights, and requires the recognition of indigenous peoples as rights-holders and defenders of biodiversity, who maintain management and governance systems of vast ecosystems (Brondizio & Le Tourneau, Reference Brondizio and Le Tourneau2016; Mistry & Berardi, Reference Mistry and Berardi2016).

Tengo et al. (Reference Tengö, Hill and Malmer2017) suggest five tasks that can guide processes that build trust and agency (see Figure 6.1b), while at the same time building a stronger evidence base for action. We find in our review that the mobilisation task is often neglected, or that documentation of indigenous and local knowledge is not fully recognised. More research is needed, but mobilisation of knowledge and empowerment of knowledge-holders may be critical steps for successful knowledge collaborations that also contribute to strengthening collaborative governance capacity. We also find that explicit joint problem formulation and analysis across knowledge systems is absent from many processes and is clearly a challenge in regional and global assessments with rigid scientific formats (Livoreil et al., Reference Livoreil, Geijzendorffer and Pullin2016; Nesshöver et al., Reference Nesshöver, Vandewalle and Wittmer2016; Oubenal et al., Reference Oubenal, Hrabanski and Pesche2017). Our case examples clearly show the importance of creating the right conditions for joint problem formulation.

It should be acknowledged that the implementation of a MEB approach is demanding, in terms of time and other resources, and requires strong commitment from all parties. However, we reiterate that there is mounting evidence of the potential positive outcomes in terms of novel indicators, more efficient responses to and implementation of findings, as well as for synergies between conservation and human well-being, including human rights (Danielsen et al., Reference Danielsen, Burgess and Jensen2010; Johnson et al., Reference Johnson, Alessa and Behe2015; Sterling et al., Reference Sterling, Ticktin and Kipa Kepa Morgan2017b, Reference Sterling, Filardi and Toomey2017c). As found by the participants in our third case example on reconciling conservation and human rights, conservation initiatives can play a positive role by engaging with communities and increasing their recognition as actors and partners who hold important and useful knowledge.

Our experiences derive mainly from dialogues and collaborations with indigenous and local knowledge-holders who have deep connections, obligations to care for and a duty to fight for their rights to actively govern their ancestral territories. We are aware that in many other contexts, local knowledge-holders may be less empowered and traditional governance systems and cultural connections may be displaced and eroded. However, we believe that insights about dialogue and partnership between indigenous peoples, local communities and scientists can also be applied in western, urban and developing settings, where local knowledge and experience may be less evident but remains critical for nurturing effective stewardship of biodiversity and ecosystems. Ultimately, the MEB approach contributes to a much-needed conceptual mind shift to mobilise all knowledge that is useful for maintaining the life-supporting ecosystems in our world.

7.1 Introduction

The volume of evidence from scientific research and wider observation is greater than ever. Approximately 2.5 million articles are published annually (Plume & van Weijen, Reference Plume and van Weijen2014) and this rate is increasing at around 3–3.5% per year (Ware & Mabe, Reference Ware and Mabe2015). Conservation is no exception to this trend and the result is a rapidly expanding body of potentially useful information for decision-makers (Li & Zhao, Reference Li and Zhao2015). While the expansion of research represents an important increase in knowledge generation, much of this information is scattered in fragments over increasingly diverse sources. This, along with the sheer volume, makes it harder for decision-makers to find, access and digest all of the relevant information on a particular topic, resolve seemingly contradictory results or simply identify a lack of evidence. Evidence synthesis is the process of searching for, and summarising, a body of research on a specific topic in order to inform decisions. The extent of relevant research may range from nothing, or one or two primary studies, to many hundreds. Despite the obvious potential value of synthesising findings from multiple studies (where two studies may be all that is needed to add value through synthesis), methods of rigorous evidence synthesis have been largely neglected until recently. We argue that it is time to place evidence synthesis as a central pillar of evidence-informed decision-making in conservation and environmental management.

As an enterprise, evidence synthesis is very broad and includes many and diverse methodologies, some more rigorous than others. For example, syntheses labelled as ‘literature reviews’ often lack standardised methodology, fail to report their methods and therefore lack transparency or the potential for repeatability (O’Leary et al., Reference O’Leary, Kvist and Bayliss2016). Additionally, these literature reviews do not deal with the risk of bias in either the primary research (e.g. poor-quality experimental design and conclusions that may not be supported by a given study) or the synthesis process (e.g. selective use of information). Meta-analysis approaches have become popular where significant amounts of quantitative data are available, but they are often biased in the way they select and include studies in their analysis (Koricheva & Gurevitch, Reference Koricheva and Gurevitch2014). In response to these problems, more rigorous methodologies, such as systematic reviews, have been developed. These were first used in the health sector through the work of the Cochrane Collaboration (Higgins & Green, Reference Higgins and Green2011), and have subsequently been applied to conservation and environmental management by the Collaboration for Environmental Evidence (Pullin & Knight, Reference Pullin and Knight2009; Collaboration for Environmental Evidence, 2018).

In this chapter we make a case for rigorous evidence synthesis: we explain why these methods are appropriate, how they can benefit wider society and how evidence can be synthesised, shared and used as a public good. Although evidence synthesis can inform a broad range of decision-making contexts, we focus here on two major aspects of conservation where evidence might be useful. First, in measuring the direct and indirect impacts of human activity on the natural world, and second, the effectiveness of conservation efforts to mitigate those impacts.

7.2 The central role of evidence synthesis in informing decisions in conservation policy and practice

Many factors can contribute to making a decision. In contexts where social and political stakes are high, as is common for conservation policy, scientific evidence will likely only inform decisions, rather than act as the primary driving force behind them. Although evidence is sometimes crucial, it may equally be ignored or overruled by other factors, such as political context, infrastructure and capacity. Ideally, evidence synthesis should play a central role in providing reliable evidence and enabling the wider society to understand or challenge decisions that might affect them. Making decisions without considering all available evidence might perpetuate biases, increase the likelihood of taking a wrong or costly action, or lead to missed opportunities to achieve faster or more cost-efficient outcomes. In a democratic society, comprehensive and rigorous evidence synthesis and open communication makes ‘sidelining’ (i.e. deliberately ignoring evidence) and/or biased (i.e. selective) use of evidence by authorities more difficult without challenge and transparent justification.

Unfortunately, evidence synthesis is itself often ‘bypassed’ completely or manipulated to get the answer required (i.e. policy-based evidence) (Dicks et al., Reference Dicks, Walsh and Sutherland2014). There may be significant resistance to the use of transparent evidence synthesis in the face of vested interests, and this may partly explain why organised and independent evidence synthesis receives so little attention or funding. Rigorous scientific evidence could also be seen as a threat to those with entrenched beliefs. Beyond outright opposition, complacency or inaccessibility of evidence might inhibit adoption of synthesis findings even when good intentions towards informed decision-making exist.

Fortunately, most decision-makers in conservation want practical advice that is grounded in the best available evidence (Cook et al., Reference Cook, Mascia and Schwartz2013). Leveraging syntheses and integrating their findings into decision-making processes requires an understanding of how and when evidence is necessary, and what level of confidence is needed to inform a decision. Such considerations will determine the choice of synthesis method(s), which should reflect practical needs to guide management decisions or future research. Syntheses can be used either to generate a new theory, conceptual framework or hypothesis (e.g. applying existing theory to a different context) or to test an existing hypothesis (e.g. evaluating the effectiveness of an intervention). In the context of effectiveness of interventions, evidence syntheses are relevant to decisions at several critical stage points in the life cycle of a programme or initiative: (1) initial scoping of a new topic early on in strategic planning (e.g. informing a new strategy on land use for a philanthropic foundation (Snilstviet et al., Reference Snilstviet, Stevenson and Villar2016)); (2) identification or validation of specific intervention designs (e.g. understanding how gender composition affects outcomes of resource management groups (Leisher et al., Reference Leisher, Temsah and Booker2016)); (3) benchmarking of institutional outcomes against other programmes (e.g. investments in community forest management by the Global Environment Facility (Bowler et al., Reference Bowler, Buyung-Ali and Healey2010)); (4) evaluation of overall effectiveness of an intervention across multiple contexts or applications (e.g. effects of property regimes in different biomes (Ojanen et al., Reference Ojanen, Zhou and Miller2017)). Understanding the purpose of the syntheses for informing the different stages of decision-making will ensure selection of a suitable method, appropriate engagement of stakeholders and relevant communication of findings.

Some evidence synthesis methods, such as systematic review, have been described as following the ‘information deficit model’ (Owens, Reference Owens2000); that is to say, they follow the assumption that the simple production and push delivery of evidence that fills a gap will be sufficient to achieve uptake. However, this perception misrepresents the full process behind the methodology. Systematic reviews can be socially inclusive, with extensive stakeholder contribution to formulating a question and approach, including setting the scope of the topic. This engagement attempts to ensure the findings of a review will fill a real and important synthesis gap (a knowledge need where sufficient primary research exists to allow synthesis) and respond to stakeholder demand. When engaging with stakeholders, a balance needs to be struck between involving them in the design of the review and independence from undue vested interest (Haddaway & Crowe, Reference Haddaway and Crowe2018). In the field of conservation, this balance is very much dependent on the nature of the question and the extent of vested interests (Kløcker Larsen & Nilsson, Reference Kløcker Larsen and Nilsson2017). Many aspects of evidence synthesis are collective, with stakeholders having shared motivation to benefit from the findings. In other cases, evidence synthesis is conducted in contested areas, with stakeholders that hold opposing views and may be hostile to the process and its findings. In the latter case, it is important to have a process that allows consultation when appropriate but also provides independence when necessary. For example, for some key steps, such as initial formulation of the question, engagement with stakeholders is usually essential (Land et al., Reference Land, Macura and Bernes2017), while other steps may need to be conducted free of such vested interests. To date, systematic reviews have engaged with a spectrum of stakeholders at different levels. Some reviews, for example those that are more academic or have specific commissioners (e.g. private goods reviews (Oliver & Dickson, Reference Oliver and Dickson2016)), have only passively engaged stakeholders by informing or consulting them (typically only at the beginning of the review process), while others have employed more in-depth engagement, extending to co-design of review methods and scope (Land et al., Reference Land, Macura and Bernes2017).

Alongside the purpose of syntheses, the level of confidence required to make a decision determines their method and scope. In some instances, where evidence of effectiveness is key, uncertainty in the evidence base hampers decision-making. In such instances one might ask ‘How much evidence is enough?’ or ‘How much uncertainty is acceptable?’ (Salafsky & Redford, Reference Salafsky and Redford2013). The need for evidence synthesis in the conservation sector may also vary depending on aspects of spatial scale, complexity and controversy. For example, decisions regarding inexpensive and low-risk local-scale interventions (e.g. applied to improve biodiversity or habitat conditions in nature reserves) may benefit most from locally generated, rigorous evidence, or more commonly from primary research studies conducted in similar contexts. This evidence could be provided by a single, self-generated study (as in adaptive management), be internally generated by the relevant organisation, or come from collating evidence from similar case studies. In contrast, decisions regarding expensive, often large-scale, high-risk programmes (e.g. to eradicate poaching and illegal trade in wildlife), where stakeholders are likely to be global and might hold conflicting views, may benefit from an independent global-scale, multi-context evidence synthesis. This might require a rigorous analysis of what works, where and when and for whom, involving analysis of heterogeneity in outcome and identification of effect modifiers. Often within conservation, a broader set of evidence types (e.g. controlled trials, case studies, quantitative and qualitative research) is needed to fully capture the complexity of conservation contexts.

7.3 Key aspects of rigorous evidence synthesis and why they are needed

To be reliable, evidence syntheses should consider all available evidence and attempt to provide the most accurate and precise estimation of the truth. A suite of methodologies has been developed that maximises transparency and repeatability while minimising subjectivity, susceptibility to bias or influence of vested interest. The most widespread of these, systematic reviews and systematic maps, are well-documented secondary research methods that follow detailed guidance (e.g. Collaboration for Environmental Evidence, 2018) and use step-wise processes set out in an a-priori protocol to comprehensively identify and collate all available evidence (Table 7.1).

Systematic reviews in conservation and environmental management have most commonly aimed to answer specific cause-and-effect type questions, for example relating to the effect of a management intervention or exposure on a subject of concern. (e.g. ‘What is the impact of a specific factor x on a subject z?’). In contrast, systematic maps collate and catalogue available evidence on a relatively broad subject, describing the nature of the evidence base and highlighting evidence clusters and gaps, along with methodological patterns in primary research (Collaboration for Environmental Evidence, 2018). Systematic maps can be used as an initial step of an evidence synthesis pathway to identify subtopics suitable for a systematic review and subtopics where there is insufficient evidence to make synthesis of primary data worthwhile. In such latter cases, which are common in conservation, the map may identify individual primary studies that provide useful evidence (for an example of a systematic review question generated from a map, see www.eviem.se/en/projects/SR15-Prescribed-forest-burning/).

Systematic reviews were originally developed in response to an absence of easily accessible and rigorous synthesis of available evidence. However, recent assessments have shown that non-systematic reviews that aim to inform environmental policy and practice are still prevalent, but have low methodological reliability, suffering from lack of transparency and methodological rigour, and are consequently highly susceptible to bias (Woodcock et al., Reference Woodcock, Pullin and Kaiser2014, Reference Woodcock, O’Leary and Kaiser2017; O’Leary et al., Reference O’Leary, Kvist and Bayliss2016). Moreover, the term ‘systematic review’ is often used by authors (and not challenged by editors or peer reviewers) when the reviews are in no way systematic. The production of substandard and ‘fake’ systematic reviews is increasing in all fields, from public health to environmental management and education (Haddaway et al., Reference Haddaway, Land and Macura2016; Ioannidis, Reference Ioannidis2016; Haddaway, Reference Haddaway2017; Pussegoda et al., Reference Pussegoda, Turner and Garritty2017); they are ‘fake’ in the sense that they lack necessary comprehensiveness, transparency and reliability (Haddaway, Reference Haddaway2017). This further confuses the issue for potential readers, with only a handful of environmental journals requiring authors to follow accepted standards of conduct and reporting (see Collaboration for Environmental Evidence, 2018). A potential evidence user can use keywords like ‘systematic review’ in their search and have it return documents that claim to be such, when in fact they are not. The misuse of the term ‘systematic review’ can undermine efforts towards effective decision-making and is a key reason for establishing independent standards.

Stakeholders, including scientists, rarely have the time or training to differentiate between a ‘true’ systematic review and one that misses critical components of the method (resulting in increased risk of bias and lack of transparency) especially when published in an outlet such as a peer-reviewed journal. To enhance the uptake of more rigorous and reliable synthesis methodologies and maximise the potential of evidence to inform decisions, independent coordinating bodies have been founded in different sectors of society to provide guidelines and standards for evidence synthesis. In the field of medicine this process began in the 1990s with the establishment of the Cochrane Collaboration, which aimed to conduct systematic reviews in order to provide healthcare professionals with the best available evidence on the effectiveness of clinical interventions (Higgins & Green, Reference Higgins and Green2011). The methods were transferred to the field of conservation and environmental management in the early 2000s (Pullin & Stewart, Reference Pullin and Stewart2006) and are now under the coordination of the Collaboration for Environmental Evidence. These independent coordinating bodies provide guidelines for and training in the conduct of systematic reviews and systematic maps, as well as registering, endorsing and publishing such evidence syntheses. Syntheses registered through the coordinating bodies are scrutinised by methodology experts, guaranteeing a level of reliability and rigour (Collaboration for Environmental Evidence, 2018).

In circumstances where vested interests might potentially influence the outcome of an evidence synthesis, these independent organisations provide a framework and platform to assist the review team to achieve and demonstrate independence of the synthesis process. The framework allows for full engagement of commissioners and other stakeholders in formulation of the review question and planning of the review protocol, followed by independent peer review and publication of the protocol prior to the conduct of the review. In cases where conflict or the risk of undue influence from particular stakeholders is high, the review process should be conducted by an independent review team and the report submitted for independent peer review. Following this process, the review findings may be endorsed by the independent organisation.

7.4 New developments that address barriers to evidence synthesis and communication

There are persistent barriers to the conduct of environmental evidence syntheses and communication of their findings. First, the high resource costs required have been a major disincentive to producing high-quality syntheses, despite their critical value for effective conservation. Second, efficient and effective means of communicating results and facilitating their use for real-life decision-making scenarios are haphazardly applied. These barriers limit the ability of evidence synthesis to dynamically and adaptively respond to conservation challenges. However, new developments in big data and deep learning approaches are offering exciting opportunities to harness evidence syntheses and promote them to broader audiences.

Conducting rigorous evidence syntheses, such as systematic reviews, can carry both significant monetary and human resource costs (Dicks et al., Reference Dicks, Walsh and Sutherland2014). These costs are particularly prohibitive for organisations with critical needs for evidence, but who have limited time and resources to engage in such synthesis efforts or even to glean needed information from lengthy synthesis reports (Elliott et al., Reference Elliott, Turner and Clavisi2014). Moreover, high costs make updating syntheses to create a dynamic evidence base with the most up-to-date knowledge effectively impossible using current technology (Garritty et al., Reference Garritty, Tsertsvadze and Tricco2010). Additionally, the window of opportunity for decision-making may be shorter than the time in which a credible synthesis can be completed. Thus, to be useful to conservation, evidence syntheses must be optimised to efficiently find, collate and communicate existing evidence (Boyack & Klavans, Reference Boyack and Klavans2014).

In a policy space where decision-making timelines are short and demands for rigorous, reliable evidence are high, methods assisted by advances in computing can support rapid evidence collation as well as increase cost efficiency (Shemilt et al., Reference Shemilt, Khan and Park2016). Computer-assisted approaches range from tools that manage data and streamline the synthesis process to tools powered by machine learning algorithms that allow rapid screening and extraction of evidence with reduced human intervention (Kohl et al., Reference Kohl, McIntosh and Unger2018). Promising computer-assisted approaches, including automatic term recognition, document clustering, automatic document classification and document summarisation (Frantzi et al., Reference Frantzi, Ananiadou and Mima2000; O’Mara-Eves et al., Reference O’Mara-Eves, Thomas and McNaught2015) have been trialled in medical and health topics (Ananiadou et al., Reference Ananiadou, Rea and Okazaki2009) and are beginning to be tested in ecological topics (Westgate et al., Reference Westgate, Barton and Pierson2015; Grubert & Siders, Reference Grubert and Siders2016; Roll et al., Reference Roll, Correia and Berger-Tal2018).

These developments are encouraging for increased efficiency of the synthesis processes and potentially enabling dynamic syntheses that continuously update with new evidence as it becomes available. However, there are certain caveats and limitations that must be considered prior to widespread employment of computer-assisted tools. First, unlike medicine and fields such as economics, the semantics of conservation are highly heterogeneous and non-standardised (Westgate & Lindenmayer, Reference Westgate and Lindenmayer2017), posing difficulties for both efficient and comprehensive searching, and reliable application of machine learning algorithms to sort and mine text for desired patterns. Second, thus far, the performance of these approaches remains largely untested empirically, particular for conservation and environmental topics. As the value of evidence synthesis methods is in their transparency and credibility, reliable data on the efficacy of different computer-assisted approaches are important for uptake and expansion. Third, many existing computer-assisted platforms are fee-based or require programming skills, limiting their utility to a broader field of users. To improve global ability to address pervasive environmental threats, we need to democratise access to the tools that can help decision-making worldwide, not solely in countries or among researchers with means.

7.5 Mainstreaming evidence synthesis for decision support

Efforts to engage in open science and collaborative practice between conservation and technology fields will require forming collaborative partnerships and fostering conversation between evidence producers, evidence users and data scientists, to build a cohesive and engaged community of practice to open channels of communication to all users (Joppa, Reference Joppa2015). This will allow the broader community to use existing efforts as a starting point and avoid reinventing the wheel and wasting already limited resources (Lowndes et al., Reference Lowndes, Best and Scarborough2017). Furthermore, collaborative partnerships and creative funding can foster the long-term sustainability of tools that can live on to serve users. Too often, tools and platforms are created in good faith but require maintenance and updating and lack the ongoing funding and personnel to do so. This is particularly important as tools are most useful when they can dynamically respond to user needs and emerging technologies. This is a critical stepping stone for breaking down barriers to understanding and using evidence synthesis methodologies, as without a dynamic toolbox, synthesis methods will reman aloof from the needs of a diversifying and widening audience.

Evidence synthesis conducted to Collaboration for Environmental Evidence standards generates systematic reviews and systematic maps that are theoretically accessible to all. Yet, simply because something is available does not mean that the potential user is aware of it, knows where to find it, or even how to make sense of it. This is particularly the case for those new to the concept of evidence synthesis. Indeed, many practitioners and policy-makers rely on past experience or consult colleagues, rather than make use of the full suite of evidence (Pullin et al., Reference Pullin, Knight and Stone2004; Young et al., Reference Young, Corriveau and Nguyen2016). These issues create a number of inherent challenges for those decision-makers seeking to be evidence-informed and also broader potential audiences, such as stakeholders and wider society.

One of the mantras of science communication is ‘know your audience’ (Wilson et al., Reference Wilson, Ramey and Donaldson2016; Cooke et al., Reference Cooke, Gallagher and Sopinka2017) and to have impact, the findings of an evidence synthesis need to be effectively tailored and communicated to different groups of people in different ways and through different media. Communication efforts should, for example, be sensitive to the fact that different groups vary in their ‘trust’ of the science they encounter from different sources (e.g. academic journals, colleagues, social media) (Wilson et al., Reference Wilson, Ramey and Donaldson2016; Cooke et al., Reference Cooke, Gallagher and Sopinka2017).

A study that surveyed the willingness of practitioners to use a synopsis of relevant literature on bird conservation found that participants were more likely to use the evidence to inform decisions if it was easily accessible and in a clearly summarised format (Walsh et al., Reference Walsh, Dicks and Sutherland2014). Similar summaries are needed to complement evidence syntheses. These summaries may then need to be further refined and transformed into policy briefs. Policy briefs are often written through the cultural lens of a given organisation and a given issue, meaning that these are unlikely to be useful if prepared in a generic format. Sundin et al. (Reference Sundin, Andersson and Watt2018) recently proposed the use of storytelling as a tool to effectively communicate the results of evidence syntheses. This method could give meaning to the evidence and can be communicated through videos (e.g. see https://youtu.be/4uPowxn2skg), presentations or public forums (e.g. newspapers, magazines). Nevertheless, uptake of these methods in science communication is generally slow and also could still rely on poorly conducted syntheses (McKinnon et al., Reference McKinnon, Cheng and Dicks2018).

There has also been a rise in various knowledge management platforms and data-visualisation tools to explore underlying data that support evidence synthesis (e.g. www.3ieimpact.org/en/evaluation/evidence-gap-maps/, or www.cedar.iph.cam.ac.uk/resources/evidence/). These platforms present data from synthesis projects using interactive features and intuitive visualisations. For example, the Evidence for Nature and People Data Portal (www.natureandpeopleevidence.org) allows users to filter data according to desired parameters – such as diving into a data set to examine a specific intervention or outcome or geographic region, and visualising resultant trends. Syntheses, and in particular systematic maps, can be multi-layered and complex, precipitating a need for an interface that is graphical and intuitive, allowing a broader audience to use it (Figure 7.1).

Figure 7.1 An example of an evidence ‘heat map’ linking conservation interventions with human well-being outcomes. The map allows the user to assess the evidence base for gaps and gluts as well as clicking on each box to further examine the relevant studies.

(after McKinnon et al., Reference McKinnon, Cheng and Dupre2016)

If reported responsibly, these platforms and visualisations can play an important role in how stakeholders access evidence. A challenge for these approaches is to communicate that evidence syntheses are only estimates of the truth, which depend on the reliability of the evidence with which they were made. There is potential for evidence to be misinterpreted if the relative weight or reliability of a given element is misconstrued when visualised. Regardless of the output, it is important that authors of evidence syntheses communicate any uncertainty in the evidence and the risks associated with relying on studies that have high risk of bias.

Although it is laudable to communicate the findings of a topical evidence synthesis, additional efforts are also needed to communicate to practitioners the value of systematic reviews or maps, how they differ from other evidence synthesis methods and how they can be integrated with existing science advice and decision-making processes within different regions or institutions. Writing academic papers and delivering presentations at scientific conferences is unlikely to reach the typical practitioner, so creative approaches to outreach are needed to access and inform them.

Without use of rigorous evidence synthesis, policies and practice claiming to be ‘evidence-informed’ can be meaningless. For conservation and the environmental sector in general, the value of evidence synthesis has yet to be fully realised and we have the feeling that its time is yet to come. However, the recent methodological developments, awareness-raising and capacity development, together with new technologies for faster and more efficient conduct, suggest this time is not far away. Conservation is an interdisciplinary field and cannot remain for long in a state of relative evidence synthesis deficit in comparison with other sectors with which it seeks to be relevant. Although still marginalised, the methodology and infrastructure to build conservation’s evidence base through rigorous synthesis now exist at a global level. A commitment to evidence-informed decision-making that recognises the central role of rigorous evidence synthesis is required by key actors in the sector if these potential benefits are to be achieved.

8.1 Introduction

We should not be surprised by the scale of the challenge when trying to link a body of scientific knowledge to the complex, shifting and seemingly unpredictable world of policy, or to the massively decentralised, globally distributed world of conservation practice (Young et al., Reference Young, Waylen and Sarkki2014). One side of the challenge is developing a consensual understanding of the science itself. By nature, scientific knowledge is continually progressing, with theories, empirical data and new interpretations emerging all the time. Even within a single discipline, it can be hard to convey what is known at a particular point in time, and this often involves presenting different scientific viewpoints. For instance, there is substantial variation around the world in public health advice regarding alcohol consumption, with ‘safe’ limits in the UK being 50% of those in the USA (Wood et al., Reference Wood, Kaptoge and Butterworth2018). In conservation, the challenge is even greater, as relevant research cuts across the natural, physical and social sciences.

The other side of the challenge is working out how, and when, to offer relevant scientific knowledge to decision-makers, in order to have the greatest impact on the decisions being made. This is the focus of our chapter. We argue that it is a question of correct alignment: of selecting the right knowledge to address the needs of decision-makers, ensuring that knowledge is accessible to them, and articulating it within their decision-making processes.

First, we consider how well current efforts to synthesise evidence in conservation align with the needs of decision-makers. Then we describe three mechanisms that might be used to enhance the alignment of available knowledge with decision-making, starting at small local scales and moving to the global scale: decision support tools, active knowledge exchange and large-scale scientific assessments. For each mechanism, we provide examples and draw out general guidelines regarding the circumstances in which it is likely to be most effective.

8.2 How well do current evidence synthesis activities align with policy and practice needs?

When scientific evidence is needed for decision-making, the process of obtaining and analysing the evidence is often demand-led. An organisation faced with a difficult management or policy decision will undertake or commission a review to answer a specific question. For example, the UK Government Department of Environment, Food and Rural Affairs (Defra) commissioned a review of evidence on the status of pollinators (Vanbergen et al., Reference Vanbergen, Heard and Breeze2014) before designing the National Pollinator Strategy for England (Defra, 2014). When this happens, the evidence synthesis is well-aligned with the policy and practice needs, summarising relevant material that can be found in the time available. However, it also puts immense time pressure on the evidence synthesis process, because decision-making can only happen once the evidence has been reviewed. This tends to lead to the selection of evidence synthesis methods such as rapid evidence assessments, traditional non-systematic literature reviews and expert consultations, which are not the most rigorous or unbiased approaches available (Dicks et al., Reference Dicks, Haddaway and Hernández-Morcillo2017).

The Collaboration for Environmental Evidence (www.environmentalevidence.org) and the Conservation Evidence project (www.conservationevidence.com) aim to address the needs of conservation practitioners and policy-makers with more rigorous methods of knowledge synthesis, namely systematic reviews, systematic maps (Collaboration for Environmental Evidence, 2013; see also Chapter 7) and subject-wide evidence syntheses (Sutherland et al., Reference Sutherland, Taylor and MacFarlane2019b; see also Chapter 4). They do so by actively involving stakeholders in the selection of topics to synthesise and the collation and subsequent evaluation of the evidence found (Dicks et al., Reference Dicks, Wright and Ashpole2016; Haddaway et al., Reference Haddaway, Kohl and Rebelo da Silva2017).

To evaluate the overall success of this alignment effort, we recently asked how well evidence collated by the Conservation Evidence project on the subject of sustainable food production matched the priority knowledge needs of decision-makers. Five independent exercises (Pretty et al., Reference Pretty, Sutherland and Ashby2010; Dicks et al., Reference Dicks, Abrahams and Atkinson2013a, Reference Dicks, Bardgett and Bell2013b; Ingram et al., Reference Ingram, Wright and Foster2013; Jones et al., Reference Jones, Mead and Kaiser2014), involving 240 people from across business, practice, policy-making and academia, had generated 286 priority questions faced by decision-makers. We sorted these into five categories, following the Driver–Pressure–State–Impact–Response (DPSIR) framework (Maxim et al., Reference Maxim, Spangenberg and O’Connor2009). This conceptual framework describes interactions between society and the environment in a way that is meaningful for policy. Social and economic developments (Driving Forces, D) exert Pressures (P) on the environment and, as a consequence, the State (S) of the environment changes. This leads to Impacts (I) on ecosystems, human health and society, which may elicit a societal Response (R) that feeds back on D, S or I. We added a category for questions about underlying science that did not fit the DPSIR categories (Figure 8.1).

Figure 8.1 Categorisation of 286 priority questions identified by stakeholders as relevant to sustainable food production (Pretty et al., Reference Pretty, Sutherland and Ashby2010; Dicks et al., Reference Dicks, Abrahams and Atkinson2013a, Reference Dicks, Bardgett and Bell2013b; Ingram et al., Reference Ingram, Wright and Foster2013; Jones et al., Reference Jones, Mead and Kaiser2014) according to the Driver–Pressure–State–Impact–Response framework. Examples of questions are provided for each category. The extracted segment represents questions already answered by evidence summaries provided by the Conservation Evidence project.

Of all the priority questions, 189 (66%) were about responses (R), which are the focus of the Conservation Evidence project. Evidence had already been summarised that could help answer 35 of these questions (12% overall; Smith et al., Reference Smith, Dicks and Sutherland2015; Sutherland et al., Reference Sutherland, Dicks and Ockendon2019a).

In a similar vein, Cook et al. (Reference Cook, Possingham and Fuller2013a) investigated the contribution of systematic reviews to conservation decision-making, finding that 35% of the 43 reviews considered practical on-the-ground management, while most addressed interventions relevant to policy. Cook et al. (Reference Cook, Possingham and Fuller2013a) argued that the benefits for conservation could be significantly enhanced by increasing the number of systematic reviews focused on questions of direct management relevance.

These two analyses show there is some alignment between high-quality evidence synthesis methods and the needs of conservation practitioners and policy-makers, but it could be improved. Below, we provide a series of examples of mechanisms to enhance this alignment at a range of scales.

8.3 Decision support systems

Decision support systems are tools designed to assist decision-makers, for example, by visually or numerically illustrating different possible outcomes to a question, or leading users through logical decision steps (Dicks et al., Reference Dicks, Walsh and Sutherland2014). Often software-based, they represent a link between relevant science and decision-making (Dicks et al., Reference Dicks, Walsh and Sutherland2014; Figure 8.2). Decision support systems are useful for incorporating evidence into decisions related to a specific question that has been widely and repeatedly addressed. It is also important that the evidence can be converted into simple numerical or visual formats.

Figure 8.2 A schematic showing how scientific information could support environmental decision-making (Dicks et al., Reference Dicks, Walsh and Sutherland2014). The triangle on the left shows an evidence hierarchy, in which summaries, such as those produced by the Conservation Evidence project, integrate evidence from across studies and systematic reviews, and form the basis for information flowing into decision support systems. In these circumstances, environmental decisions (shown by the ‘Decision’ diamond on the right) are based on the best-available evidence, combined with the expertise and local knowledge of the practitioner or policy-maker (described by the ‘Experience’ box). Dashed lines illustrate bypass routes currently taken to inform environmental decisions.

There are many decision support tools available covering various aspects of environmental science. For instance, Zasada et al. (Reference Zasada, Piorr and Novo2017) identified 60 research projects funded between 2002 and 2013 under the European Commission’s 6th and 7th Framework Programmes that had developed decision support tools for landscape and environmental management. Of these, only 61% still existed in 2014, and only half were updated after the projects that developed them ended, although this seems a pre-requisite for ongoing use. The uptake of decision support systems depends on a range of factors, including ease of use, performance, whether they are recommended by peers and the level of marketing (Rose et al., Reference Rose, Sutherland and Parker2016). Uptake can be enhanced by ensuring that users are closely involved in the conception and design of the tools (Rose et al., Reference Rose, Parker and Fodey2018).

While decision support systems are often designed by researchers as a way of incorporating scientific knowledge into practice, most are based on one particular model, study or approach to a scientific question and represent a ‘bypass’ of the evidence hierarchy (Figure 8.2 and see Dicks et al., Reference Dicks, Walsh and Sutherland2014). There are only a few examples where they represent the best-available scientific knowledge, based on rigorous synthesis of evidence.

One such decision support tool is the online biodiversity metric incorporated into the Cool Farm Tool (available at www.coolfarmtool.org), which provides scores for the likely benefits for biodiversity of a range of farm management actions. The actions that are included are selected according to a combination of expert judgement and assessments of summarised evidence conducted by the Conservation Evidence project. Each farm management action is assigned scores reflecting the benefit for overall biodiversity, and also for 11 species groups (e.g. woodland birds, beneficial invertebrates), weighted according to the evidence. Actions that are strongly supported by the evidence provided by the Conservation Evidence syntheses (Sutherland et al., Reference Sutherland, Dicks and Ockendon2019a) are scored more highly than those for which effectiveness is not known.

Another example is the set of greenhouse gas emission calculators used in agriculture to support mitigation by changing farm management. These tools incorporate models of greenhouse gas emissions and carbon storage according to vegetation type and farming practice (Richards et al., Reference Richards, Metzel and Chirinda2016). These calculators combine empirical models with emission factors collated by the Intergovernmental Panel on Climate Change (see ‘National and International Scientific Assessments’). Although the outputs from these tools are only as good as the data that they are based on, new information can be added to improve their performance as it becomes available. For example, Richards et al. (Reference Richards, Metzel and Chirinda2016) demonstrated that two widely used software tools tend to overestimate emissions from smallholder farms in tropical environments, but suggest that this is probably due to a systematic bias in literature, with most data coming from temperate regions, rather than bias in the models themselves. As empirical data are included from a wider range of environments, more accurate disaggregated emissions factors will become available for different parts of the world. If the decision support systems are maintained and updated, this new knowledge will directly influence decision-making at farm level.

8.4 Active knowledge exchange mechanisms

Active knowledge exchange mechanisms are the most diverse alignment mechanism of the three considered in this chapter. Our concept is similar to that of ‘boundary organisations’ identified by some other authors (Guston, Reference Guston2001; Cook et al., Reference Cook, Mascia and Schwartz2013b), in that they operate in both scientific and practical spheres, but retain distinct lines of accountability to both groups. They can take a variety of institutional forms, from a dedicated, self-funded or government-funded organisation to a network of people working together across organisations (see also Chapter 13).

The reputation of such a body depends on its ability to produce or broker knowledge that is salient, credible and legitimate (Cash et al., Reference Cash, Clark and Alcock2003; Sarkki et al., Reference Sarkki, Tinch and Niemelä2015) while maintaining transparency. Credibility refers to the scientific adequacy of the technical evidence and arguments. Salience is the relevance of the brokered knowledge to the needs of decision-makers. Legitimacy reflects the perceptions that the production of information has been respectful of stakeholders’ divergent values and beliefs, unbiased in its conduct and fair in its treatment of views and interests. Achieving all these values requires adequate attention to governance from the outset.

Here, we provide examples of knowledge exchange mechanisms operating at a subnational scale, related to a particular environmental issue or landscape (Wadden Sea case study); at a national or international scale but restricted to environmental science (EKLIPSE mechanism); and at a national or international scale ranging across all scientific knowledge (European Scientific Advice Mechanism, and UK Parliamentary Office of Science and Technology).

8.5 National and international scientific assessments

A longer-term approach to aligning evidence synthesis with conservation policy decisions involves governments or international bodies mandating large-scale, scientific assessments in broad areas of strong policy interest. Examples include the assessment reports conducted by the Intergovernmental Panel on Climate Change (IPCC; www.ipcc.ch), Intergovernmental Science Policy Platform on Biodiversity and Ecosystem Services (IPBES; www.ipbes.net) and Millennium Ecosystem Assessment (www.millenniumassessment.org; see Chapter 16 for further details of mechanism and function of the Millennium Ecosystem Assessment and the IPBES science–policy platform). These global assessments involve hundreds or even thousands of scientists around the world, including indigenous and local knowledge-holders in the case of IPBES (Sutherland et al., Reference Sutherland, Gardner and Haider2014; see also Chapter 16).

Generally, governments define the scope of the assessment and identify or nominate a set of experts to conduct it (IPCC, 2015). The nominated experts form working groups and develop report texts, which are subject to extensive, transparent review, first by other experts and then by governments. Following review, the report texts are converted into concise summary documents (usually called ‘Summary for Policy-makers’), the final text of which is agreed by governments. Each statement in the summary document must be traceable back to the full scientific report and, from there, to individual pieces of research or sources of knowledge. Through this process, science and policy influence one another in a two-way exchange of knowledge over very large temporal and spatial scales.

The IPCC, which has been active for almost three decades, has built a strong reputation for providing an overview of climate science across a range of disciplines, from geophysics to economics. There are now clear links from the scientific understanding of human-induced climate change and its impacts to policies controlling greenhouse gas emissions at national and international levels. Most recently, the Paris Climate Agreement of December 2015 is a global accord under which nations have made pledges and set emissions targets to keep global temperature rise below 2°C (Clemencon, Reference Clemencon2016; Tobin et al., Reference Tobin, Schmidt and Tosun2018). A large quantity of scientific research underlies these policy pledges, which would likely not have happened, or not have been so extensive, without the IPCC assessment process. Forty-five different global climate models are now being used together to link levels of greenhouse gas emissions to long-term global temperature rise under different emissions scenarios (Collins et al., Reference Collins, Knutti, Arblaster, Stocker, Qin and Plattner2013). There is also a plethora of analyses and modelling connecting economic activity to greenhouse gas emissions (e.g. Vandyck et al., Reference Vandyck, Keramidas and Saveyn2016) and threshold temperate rises with specific impacts on environments, economies and human well-being (IPCC, Reference Field, Barros and Dokken2014).

The Millennium Ecosystem Assessment (2005) was the first global evaluation of the status of ecosystems, and developed the ecosystem services framework for understanding how nature can benefit people. The ecosystem services concept originated in the academic world (Potschin & Haines-Young, Reference Potschin, Haines-Young, Potschin, Haines-Young, Fish and Turner2016), but the Millennium Ecosystem Assessment formalised the thinking, providing a conceptual framework and nomenclature for ecosystem services. Since its publication, a growing number of countries have conducted their own national ecosystem assessments (Schrӧter et al., Reference Schröter, Albert and Marques2016) and the policy ground is being set for their results to be used in national natural-capital accounting. Both Aichi Biodiversity Target 2 from the Convention on Biological Diversity’s Strategy Plan 2011–2020 (Convention on Biological Diversity, 2010) and Action 5 of the EU Biodiversity Strategy to 2020 (European Commission, 2011) call for biodiversity values to be incorporated into national accounting.

Large-scale assessments are most effective at aligning scientific evidence with decisions when there is a broad issue of strong political interest, such as climate change or biodiversity loss. The assessments are expensive (see Table 8.1), so there must be substantial political commitment and a source of funds over the relatively long term.

Given the obvious power of national and international scientific assessments to influence policy, it is now more important than ever to incorporate into them the transparent, unbiased repeatable methods that have been developed for evidence synthesis. Currently, the rigour and reliability of large-scale scientific assessments rely on extensive peer review, rather than systematic searching or careful elicitation methods that reduce bias. Evidence synthesis methods are usually not reported (with some exceptions, such as chapter 6 of the Intergovernmental Science Policy Platform on Biodiversity and Ecosystem Services pollination report; IPBES, 2016). However, such assessments are conducted over long timescales, with the IPCC, for example, producing a global assessment report every 5–10 years. With this amount of time and money available (see Table 8.1) there is a clear opportunity to develop rigorous processes of evidence synthesis within this framework. As a first step, we urge policy-makers and institutions involved in commissioning large-scale scientific assessments to require authors to report their underlying synthesis methods.

8.6 What does it all cost?

The cost of the alignment mechanisms outlined in this chapter varies considerably, both within and among the different activities (Table 8.1). These costs should be interpreted in the context of total spending on scientific research. For example, the budget of the European Commission’s flagship scientific research programme, Horizon 2020, is approximately £8 billion per year.

The organisations that fund research and aspire to be evidence-informed already invest heavily in improving interactions between science, policy and practice. Unfortunately, they frequently fund expensive decision support systems that are not maintained or used a few years later (Zasada et al., Reference Zasada, Piorr and Novo2017) and large-scale reviews or scientific assessments that do not follow clear protocols to reduce bias. The challenge in aligning evidence synthesis with decision-making is not to find the money, but to demand and enable improved rigour and continuity in activities that are already taking place.

No single mechanism will be best for aligning evidence with policy and practice in all contexts. Each has strengths and weaknesses, and can be applied in different circumstances and at different scales. International assessments have redirected policies and scientific endeavour on a very large scale, but would be unlikely to align specific scientific findings with conservation practice at smaller scales. At smaller scales, the potential of decision support systems to incorporate rigorously collated environmental evidence has hardly been tapped.

At every level, mechanisms to link synthesised evidence with policy and practice decisions need to be funded sufficiently to ensure salience, legitimacy, credibility and transparency. These linking mechanisms need access to methods of collating and communicating evidence that are well-developed, transparent and widely understood (Cook et al., Reference Cook, Nichols and Webb2017; Dicks et al., Reference Dicks, Haddaway and Hernández-Morcillo2017) and are just as important as the research itself, if not more so.