Skip to main content Accessibility help
×
Home

Contents:

Information:

  • Access
  • Open access

  • Conservation Research, Policy and Practice
  • Online publication date: April 2020
  • pp 27-142
  • Publisher: Cambridge University Press

Figures:

Actions:

      • Send chapter to Kindle

        To send this chapter to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle.

        Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        Available formats
        ×

        Send chapter to Dropbox

        To send content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about sending content to Dropbox.

        Available formats
        ×

        Send chapter to Google Drive

        To send content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about sending content to Google Drive.

        Available formats
        ×
Export citation

Part I Identifying priorities and collating the evidence

Chapter Three Scanning horizons in research, policy and practice

Bonnie C. Wintle
University of Melbourne
Mahlon C. Kennicutt II
Texas A&M University
and William J. Sutherland
University of Cambridge
3.1 Introduction

Conservationists have long had to deal with a number of prominent, recurring issues, such as habitat loss and fragmentation, pollution, invasive species and wildlife harvesting, to name a few. On top of these well-known challenges, others have emerged. Over the last half century, these have included the impact of halogenated pesticides and defoliants, acid rain from coal-fired electricity generation, ecological impacts of biofuel production and atmospheric releases of ozone-depleting chemicals. In more recent times, concerns have emerged around microplastics and exploitation of the Arctic, although some changes also bring opportunities for conservation, such as using mobile phones to collect data. New and emerging issues tend to make policy and practice more difficult. They add to an already challenging agenda, and often require a response when knowledge of the problem is limited.

Emerging from the relatively new field of ‘futures’ studies, horizon scanning is still developing as a method. By crowd sourcing information and drawing on communities of practice to sort, verify and analyse that information, horizon scanning offers an efficient way to look for early indications of poorly recognised threats and opportunities (Sutherland & Woodroof, Reference Sutherland and Woodroof2009; van Rij, Reference van Rij2010). It aims to minimise surprises by foreseeing these threats and opportunities, enabling policy-makers and researchers to respond quickly to developing problems. Horizon scanning is an approach primarily used to retrieve, sort and organise information from different sectors that is relevant to the question at hand, in a similar process to intelligence gathering. It can also include varying degrees of analysis, interpretation and prioritisation, but deciding which issues to act on, and how to act on them, typically takes place after the horizon scanning, and is assisted by other ‘futures’ tools, such as visioning, causal layered analysis, scenario planning and backcasting (e.g. Glenn & Gordon, Reference Glenn and Gordon2009; Inayatullah, Reference Inayatullah and Gutierrez Junquera2013; Cook et al., Reference Cook, Inayatullah and Burgman2014a). Recent frameworks have also been developed to link different futures tools, such as horizon scanning and scenario planning, together (Rowe et al., Reference Rowe, Wright and Derbyshire2017).

Horizon scanning outputs come in a wide range of forms. Some broadly describe a single trend that cuts across different parts of society, such as the rise of big data, or the future of a general area of interest, such as ‘Environmental Sustainability and Competitiveness’ (Policy Horizons Canada, 2011). These outputs are usually aligned with more general foresight programmes. Other exercises look at a set of more specific potential threats, such as invasive species that may arrive in the UK and threaten biodiversity (Roy et al., Reference Roy, Peyton and Aldridge2014), and compare them in an approach similar to risk assessment. For the last 10 years, conservation scientists have run annual horizon scans to identify emerging issues with the potential to impact global conservation (e.g. Sutherland et al., Reference Sutherland, Butchart and Connor2018). A similar approach has also been used to identify important scientific questions that, if answered, would help guide conservation practice and policy (e.g. Sutherland et al., Reference Sutherland, Adams and Aronson2009).

As with any policy advisory work, there is always a risk that useful information is gathered but not followed up, as decisions are often driven by other, usually non-scientific, factors. This risk may be higher with unsolicited (grassroots scans produced by a community of practitioners, researchers or academics) rather than solicited scans (called for by policy- and decision-makers). It can be unclear where the responsibility lies for integrating outputs into policy-making, and uptake depends on the organisational culture at the time (Delaney & Osborne, Reference Delaney and Osborne2013). Schultz (Reference Schultz2006) pointed to a conceptual contradiction between evidence-based policy and horizon scanning, where the latter searches for issues that may not be fully supported by a definitive body of evidence. A more optimistic perspective is that horizon scanning needs to be embedded in a broader strategic foresight framework, to increase the likelihood that findings are translated into practice (e.g. van Rij, Reference van Rij2010; Cook et al., Reference Cook, Inayatullah and Burgman2014a). As mentioned above, horizon scanning identifies emerging and novel threats and opportunities as a first step, but other foresight tools serve different purposes along the pathway to adopting appropriate policy. These other foresight tools are not explicitly covered in this chapter, but we provide an example, The Antarctic Science Scan and Roadmap Challenges Exercise, of a hybrid horizon scanning activity where an accompanying road map was also produced to outline actionable recommendations (Box 3.2).

In this chapter, we introduce both general and specific approaches to horizon scanning, outline some ways of achieving and measuring impact and explore how horizon scanning may evolve in the future.

3.2 Approaches to horizon scanning

‘Exploratory horizon scanning’ identifies novel issues by searching for the first ‘signals’ of change across a wide range of sources (such as an early scientific paper describing a potentially impactful new technology). ‘Issue-centred scanning’ monitors issues that have already been identified by searching for additional signals that confirm or deny that the issue is truly emerging (Amanatidou et al., Reference Amanatidou, Butter and Carabias2012). Signals can be organised into clusters (multiple pieces of information) that can either contribute to the evidence base around pre-identified issues, or form a long list of novel issues that are potentially emerging (Figure 3.1). The long list of issues can be further analysed and prioritised into a shortlist using methods detailed below. Some horizon scanning exercises take further steps to make the output more useful for the end user, for example, by assessing the policy relevance of the issues or the feasibility of addressing them, and by identifying those that warrant ongoing monitoring (Sutherland et al., Reference Sutherland, Allison and Aveling2012).

Figure 3.1 General framework for horizon scanning, reflecting the key steps in the procedure (ovals), inputs and products (rounded rectangles), key outputs (rectangles), actors and end users (triangles), and activities and methods (floating text).

Process adapted from Amanatidou et al. (Reference Amanatidou, Butter and Carabias2012).

There is a range of different ways to carry out horizon scanning; we introduce the main stages and provide some specific examples in the boxed texts and Table 3.1. Because our definition of horizon scanning concentrates largely on information retrieval, sorting and, to some extent, analysis and prioritisation, we focus here on methods that facilitate these activities.

Table 3.1 Approaches to horizon scanning (some activities and examples overlap)

ApproachExamples
Manual search of an invited expert group with Delphi-style prioritisationGlobal conservation (e.g. Sutherland et al., Reference Sutherland, Butchart and Connor2018), Antarctic science (e.g. Kennicutt et al., Reference Kennicutt, Chown and Cassano2015), bioengineering (Wintle et al., Reference Wintle, Boehm and Rhodes2017), Mediterranean conservation (Kark et al., Reference Kark, Sutherland and Shanas2016)
Manual search of a large crowd-sourced group (open call) with Delphi-style prioritisation (invited)Future of the Illegal Wildlife Trade (Esmail et al., Reference Esmail, Wintle and Sas-Rolfes2019)
Automated open-source search and manual analysis/prioritisation (usually by a community of experts)IBIS (Grossel et al., Reference Grossel, Lyon, Nunn, Robinson, Burgman, Nunn and Walshe2017), Global Disease Detection Program (Centers for Disease Control and Prevention, www.cdc.gov/globalhealth/healthprotection/gdd/index.html), HealthMap (www.healthmap.org/en/), ProMed (www.promedmail.org/)
Advanced text analytics to identify emerging issues and research areas (e.g. sentiment analysis, machine learning)FUSE Program (www.iarpa.gov/index.php/research-programs/fuse), Meta (https://meta.org/), X risk database (www.x-risk.net/)
Manual searches within an organisation (by employees, interns or volunteers), results tagged and catalogued in a databaseUS Forest Service (Hines et al., Reference Hines, Bengston and Dockry2018), UK Department for Environment, Food and Rural Affairs (Garnett et al., Reference Garnett, Lickorish and Rocks2016)
Comprehensive programme (including scanning, sentiment analysis, scenario planning; manual and automated)Singapore’s Centre for Strategic Futures (www.csf.gov.sg/), partnered with the Risk Assessment and Horizon Scanning Programme Office
Expert opinion (voting, survey)Global Risks Report 2019 (World Economic Forum, 2019)
Regular meeting of a cross-disciplinary horizon-scanning group to discuss emerging issues and build databaseAustralasian Joint Agencies Scanning Network (www.ajasn.com.au/), Human Animal Infections and Risk Surveillance group (www.gov.uk/government/collections/human-animal-infections-and-risk-surveillance-group-hairs#risk-assessments-and-process)
3.2.1 Scoping

Like any major project, horizon scans need to be scoped and clear guidelines developed to assist scanners. A comprehensive scoping exercise addresses the following questions.

  • What is the guiding question that defines what you want to know?

  • How broadly or narrowly defined is the field of interest?

  • What are the key drivers of change and activities in the field? It is common to organise thinking around a STEEP (Social, Technological, Economic, Environmental and Political factors) framework.

  • What is the spatial scope? For instance, are you seeking issues with global or more localised impact?

  • How far into the future should scanners be projecting?

  • Who should be involved?

  • Who are the potential end users?

Many of these considerations will be constrained by the resources available and the needs of the end user, but tools such as stakeholder analysis (Reed et al., Reference Reed, Graves and Dandy2009), domain mapping (Lesley et al., Reference Lesley, Floyd and Oermann2002) and issues trees (Government Office for Science, 2017) can be useful. Scoping exercises may also involve some pilot scanning to get a feel for how well-defined the task is. For example, preliminary scanning in a US Forest Service project that aimed to identify emerging issues that could affect forests and forestry in the future revealed that ‘natural resources and the environment’ was too broad a topic for their exercise. Instead, it was narrowed to ‘forests’ (Hines et al., Reference Hines, Bengston and Dockry2018).

Horizon scans that rely heavily on people rather than computers to do the scanning reflect the biases of those participants. A well-structured procedure for obtaining judgements from participants (e.g. Figure 3.2) will go a long way to mitigate psychological biases (Burgman, Reference Burgman2015b), but in order to capture a broad array of perspectives, involving a diverse group of people to identify and prioritise candidate issues is critical. A cognitively diverse group – comprising individuals who think differently – is thought to maximise collective wisdom and objectivity (Page, Reference Page2008). A good proxy for cognitive diversity is demographic diversity. Achieving demographic diversity can be challenging in practice. For example, there may be language barriers to overcome, and people with certain occupations (e.g. scholars) may be over-represented in horizon scans conducted by researchers. Inviting contributions from further afield, both geographically and from outside immediate peer circles, broadens the scope of issues considered. This might be achieved by putting out an open call for issues online and advertising it through relevant websites and email lists (e.g. Esmail et al., Reference Esmail, Wintle and Sas-Rolfes2019), or posting a call for ideas on social media.

Figure 3.2 The Delphi-style horizon-scanning approach often used in conservation (Sutherland et al., Reference Sutherland, Fleishman and Mascia2011).

Figure reproduced from Wintle et al. (Reference Wintle, Boehm and Rhodes2017), published under the Creative Commons Attribution 4.0 Licence.
3.2.2 Gathering inputs

Inputs to a scan can either be gathered manually (by people) or with the aid of automated software, which is then (usually) analysed by people. Manual scanning typically involves a group of people monitoring current research and relevant trends (e.g. technology trends, disease trends or population trends) via desktop searches, attending conferences and consulting other people in their networks. Information can be manually scanned in news articles, social media, publications, grey literature and other output of relevant organisations (such as models and projections). This is typically the first step in a ‘Delphi-style’ method that then goes on to analyse and prioritise candidate issues in a structured approach, usually involving one or more expert workshops (see Boxes 3.1 and 3.2 for examples and further descriptions of the procedure). Scanners could be provided with guidelines by a facilitator to direct their search, including suggestions of where to look. Manual methods have the advantage of accessing content that may not exist online (e.g. grey literature or unpublished research), or content that may be difficult to locate in the absence of known keywords to direct database and online searches. The downside of manual methods is that they are labour-intensive and may be exposed to the biases of the searcher, as they are less systematic.

Box 3.1. A Delphi-style method for horizon scanning in conservation

With its foundations in the Delphi Method (Linstone & Turoff, Reference Linstone and Turoff1975; Mukherjee et al., Reference Mukherjee, Huge and Sutherland2015), this structured approach (Figure 3.2) was first applied in horizon scanning for conservation by Sutherland et al. (Reference Sutherland, Bailey and Bainbridge2008). There are now several variants. The key features that make this approach ‘Delphi-style’ are iteration (issues are submitted, scored, discussed and scored again) and anonymity of submissions and scoring. Typically, about 25 conservation experts from around the world participate in the following procedure. Over the course of several months, participants independently scan material from a variety of sources (e.g. papers, reports, websites, conferences) looking for issues (threats or opportunities) that are relatively novel, but that we should start planning for. Over email, each participant anonymously submits short summaries of two to five issues they have selected as the best ‘horizon-scanning’ candidates, defined as reflecting a combination of novelty, plausibility and potential future impact on global conservation. The facilitator compiles the issue summaries and circulates them back to the group, who anonymously score each issue in terms of its suitability as a ‘horizon-scanning’ item (using the definition above). A shortlist of the top scoring issues, containing perhaps twice the total number sought, is recirculated back to participants. Each participant is assigned approximately five issues (not their own) to investigate further, gathering further evidence to support or oppose the issues’ suitability. This means each issue will be cross-examined by at least two to three people. These five issues are usually assigned to people who are not considered experts in that subject matter, in the hope that they will have fewer preconceptions about the issue and that the experts will add their knowledge anyway. The whole group then meets at a workshop and systematically discusses each of the shortlisted issues (e.g. to consider new perspectives, relevant research, and whether the issue is genuinely novel or just a repackaging of an old issue). The issues are kept anonymous to reduce biases and allow for an open discussion. After the discussion, participants individually score the issues a second time. The top-scoring 15 are redrafted by one of the other group members and published each year in Trends in Ecology & Evolution (e.g. Sutherland et al., Reference Sutherland, Butchart and Connor2018).

Box 3.2. Antarctic science scan and Roadmap Challenges project

The international Antarctic community came together to horizon scan the highest priority scientific questions that researchers should aspire to answer in the next two decades and beyond. The approach included online submission of questions from the science research community, followed by a subset of 75 representatives (by nomination and voting) attending a workshop. At the workshop, approximately 1000 submitted questions were winnowed down to the 80 most important through methodical debate, discussion, revision and elimination by voting. All information used, including the 1000 submitted questions, was made publicly available in a database at a horizon scan website (Kennicutt et al., Reference Kennicutt, Chown and Cassano2014). The horizon scan was followed by the Antarctic Roadmap Challenges project that was designed to delineate the critical requirements for delivering the highest priority research identified. The project addressed the challenges of enabling technologies, facilitating access to the region, providing logistics and infrastructure and capitalising on international cooperation. The process uniquely brought together scientists, research funders and those that provide the logistics for field research in the Antarctic. Online surveys of the community were conducted to identify the highest priority technological needs, and to assess the feasibility (time to development) and cost of these requirements. Sixty experts were assembled at a workshop to consider a series of topic-specific summary papers submitted by a range of Antarctic communities, survey results and summaries from the horizon scan, as well as existing documents addressing future Antarctic science directions, technologies and logistics requirements (Kennicutt et al., Reference Kennicutt, Chown and Cassano2015).

Computer-assisted scanning is increasingly used for automating the process of gathering a vast quantity of inputs, often crowd-sourced and usually from the internet (Palomino et al., Reference Palomino, Bardsley and Bown2012). Several such tools are now used in agriculture and health biosecurity to provide early detection of disease outbreaks (see Table 3.1 and Box 3.3 for examples) (Salathé et al., Reference Salathé, Bengtsson and Bodnar2012; Kluberg et al., Reference Kluberg, Mekaru and McIver2016; Grossel et al., Reference Grossel, Lyon, Nunn, Robinson, Burgman, Nunn and Walshe2017). Early online information, such as a tweet about a Tasmanian devil with a tumour on its face, or a YouTube video about a new device for targeting an invasive species, although unverified to begin with, may be critical for establishing the first in a series of signals that suggests a new or emerging threat (Grossel et al., Reference Grossel, Lyon, Nunn, Robinson, Burgman, Nunn and Walshe2017). Information on the internet can be retrieved in a number of ways. Keywords can be inserted into whole web search engines and/or particular websites can be targeted in more depth (e.g. Twitter can be searched using search terms, handles and hashtags). Research, news and current affairs can also be accessed via the RSS feeds of particular news and science sites, or by email and subscription to social media and blogs. Online data are often retrieved with the help of web scraping (accessing and storing particular web pages) and web crawling (accessing and storing links, and links of links from that page) (Hartley et al., Reference Hartley, Nelson and Arthur2013). With the recent increase in ‘fake news’, web searches require some form of quality control and vetting of sources: a process that can also be useful for exposing fake news. Large volumes of text scraped from the web, articles, patents, reports and other publications can be mined and filtered for potential relevance using automated software, such as machine learning algorithms.

Box 3.3. Online horizon scanning: intelligence-gathering for biosecurity

The International Biosecurity Intelligence System (IBIS) is a generic web-based application that focuses on animal, plant and marine health, and provides continuing surveillance of emerging pests and diseases, including environmental ones (Grossel et al., Reference Grossel, Lyon, Nunn, Robinson, Burgman, Nunn and Walshe2017). It also detects other environmental issues, such as harmful algal blooms. It is open source, in that it gathers articles from regular feeds of trusted sources (e.g. industry news, research) and publicly available online material, like news reports, blogs, published literature and Twitter feeds. Searches can be directed by broadly relevant keywords, such as ‘disease’ or ‘outbreak’ or ‘dead’, in addition to specific diseases of concern (e.g. ‘oyster herpes virus’). Articles can also be manually submitted by registered users to the application directly. A large expert community – the registered users, who are self-selected and approved by the administrator – then filter the articles, promoting those that they deem important and relevant to the home page, and demoting those that appear to be irrelevant or junk. Automated tools also assist with filtering (e.g. with machine learning and network cluster analysis), but as machine learning is still in its infancy, its use is limited to disease outbreaks from trusted sources. Items classified as junk by people are retained in a database to help the system’s artificial intelligence (AI) algorithms learn. The broader user community (anyone who signs up online) is alerted to items that have been flagged by the registered users as important, via a daily email new digest. IBIS is also ‘open-analysis’, meaning that analysis of the publicly available information is performed openly by registered users. They can create or contribute to an emerging/ongoing issues dashboard that features a window for adding content, a Delphi-based forecasting section, links to related reports, share functions, comments and a map showing the location of events of interest (e.g. an outbreak). Registered users can also conduct their own searches and use integrated analytical tools to construct intelligence reports. IBIS has been effective for guiding policies and active risk management decisions for the Australian Government since 2006. The system may produce up to five Intel briefs a week on major issues affecting biosecurity and trade, allowing the government to respond to threats much faster than before. For instance, the system picked up a report of oyster herpes virus from a UK farm, which had previously purchased used aquaculture equipment from a disease-stricken oyster farm in France. Intelligence from IBIS revealed that businesses that had been closed down by the disease had been liquidating their equipment and selling to other countries. In response to this, the Australian Government changed its biosecurity policy to decontaminate all used aquaculture equipment on arrival (Burgman, Reference Burgman2015a).

Automated scanning is fast, systematic and comprehensive in its scope, but often relies on people – sometimes experts – to screen, review, and perhaps investigate all reports before on-posting or incorporating them (Lyon, Reference Lyon2010). For tools that scan across a wide range of topics, and those that use ongoing surveillance, this can be onerous and time-consuming. There are three other notable challenges to relying on online content for horizon scanning. First, material needs to already be posted on the web, and there may be a delay before an event, such as an invasive species incursion, is reported online. The second is that useful content is not always publicly available, as it can lie behind pay walls, be stored on intranets (e.g. grey literature), or secured because it is commercially, politically or personally sensitive. The third challenge is that most methods for obtaining online content rely on using the right keywords, which requires some idea of what you are looking for.

3.2.3 Sorting, cataloguing and clustering

Tagging and cataloguing content derived from both manual and automated scans (e.g. by relevance, credibility, source type, sectoral origin) (e.g. Garnett et al., Reference Garnett, Lickorish and Rocks2016; Hines et al., Reference Hines, Bengston and Dockry2018) occurs concurrently with input gathering by scanners. Content can be further reorganised and vetted at a later stage. During this process, new search terms to direct further scanning can be generated, or existing search terms refined. Content can be organised according to a framework that also considers the level of response required and the strength of the evidence, which can help prioritise risks and other identified issues at a later stage (Garnett et al., Reference Garnett, Lickorish and Rocks2016). Clustering methods, such as network analysis (Könnölä et al., Reference Könnölä, Salo and Cagnin2012; Saritas & Miles, Reference Saritas and Miles2012), are useful for capturing cross-cutting issues that affect a number of topics of interest.

3.2.4 Analysing and prioritising

At this stage, a long list of issues will have been compiled, with some more suitable to the project aims than others. This can be an opportune time to reiterate objectives. Do you seek issues that most have not heard of? Do you intend to identify broad, developing topics or very specific developments (for example, the ‘increase in hydropower’ versus ‘fragmentation effects of hydropower in the Andean Amazon’)? Are you interested in issues likely to arise soon or events that have a smaller probability of playing out in the long-term future? Does the output need to be useful to policy-makers? Many exercises, especially those with follow-up plans, aim to prioritise a select number of ‘most suitable’ issues, and the precise manner in which such prioritisation decisions are made makes a real difference to the quality of the output (Sutherland & Burgman, Reference Sutherland and Burgman2015). Our experience with exercises that aim to identify novel issues is that participants gravitate towards well-known although important issues. Avoiding this requires strong chairing and a group that accepts the objective. To help overcome the problem, each participant can be asked whether they have heard of each issue, so that well-known topics can be excluded from the shortlist.

Within a manual Delphi-style approach (described in Boxes 3.1 and 3.2), issues are prioritised through an iterative scoring or voting process, usually facilitated online or in a workshop with a group of experts. The goal is to reduce a pool of potential horizon scanning items or ideas to a smaller subset. The number of items, or issues, covered in the final list can vary, but tends to reflect around 10–30% of the initial items put forward (e.g. Kennicutt et al., Reference Kennicutt, Chown and Cassano2014; Parker et al., Reference Parker, Acland and Armstrong2014; Kark et al., Reference Kark, Sutherland and Shanas2016; Wintle et al., Reference Wintle, Boehm and Rhodes2017; Sutherland et al., Reference Sutherland, Butchart and Connor2018). As a point of comparison, the horizon scans described in Box 3.1 describe 15 issues annually, while the Antarctic hybrid horizon scan identified 80, shorter, priority scientific questions (Box 3.2). The final number may be constrained by how many the end user can realistically give their attention to (for a busy policy-maker, this may only be 15–20 half-page summaries), but is also driven by the number of (in)appropriate issues submitted. The main purpose of prioritisation is to remove issues that do not satisfy the selection criteria (novelty, plausibility, potential impact) and select those that are the most urgent or time-sensitive. Prioritisation of issues will inevitably involve trade-offs, especially where different group members have different perspectives. Because individuals’ diverging opinions can be masked in aggregated scores, analysing interrater concordance (e.g. with Kendall’s W) affords insights into the level of agreement between contributors. In a diverse group, we would expect a wide variety of viewpoints to be voiced, but a core of shared opinions is often discernible (e.g. Wintle et al., Reference Wintle, Boehm and Rhodes2017).

Items identified in a computerised scan (e.g. articles returned from a keyword search) are also prioritised by groups of people with varying levels of content expertise. People may be employed to sort through material, such as in governmental horizon-scanning programmes like in Singapore, or they may volunteer to do so because they are interested in the output, such as a farmer or epidemiologist concerned with news of disease outbreaks. Initially, items are sorted according to their relevance to the scanning aims (often done in the initial tagging/sorting process). Irrelevant items are discarded or moved to low priority. A second form of prioritisation involves flagging issues or topics that are particularly noteworthy (Grossel et al., Reference Grossel, Lyon, Nunn, Robinson, Burgman, Nunn and Walshe2017). This can be because signals have grown stronger (more evidence is gathered to suggest an issue is becoming a threat or presenting an opportunity for action) (Cook et al., Reference Cook, Wintle and Aldrich2014b), or it might be because the potential consequences are so severe that the issue warrants immediate attention, even when evidence is limited or the probability is low (‘wild cards’).

3.2.5 Using the output

The previous step described prioritisation within the horizon scan to reduce a candidate set of issues. In that step, issues are ideally not judged according to importance, but rather according to less-subjective criteria, such as the likelihood of occurring or exceeding some threshold within a given timeframe. Prioritising which issues are the most important, and therefore should be acted on, is a different goal, and might be decided through follow-up, explicitly values-driven exercises involving representatives from government or relevant organisations (e.g. Sutherland et al., Reference Sutherland, Allison and Aveling2012).

Bringing together a cross-section of policy-makers in a follow-up exercise can be useful, not only to identify those issues that require further monitoring or evidence before being acted on, but also to encourage prioritisation of cross-organisational issues, knowledge sharing, and collaborative development of policy. Ideally, feasibility assessments of the options available would be included (as carried out in the extension of the recent Antarctic scan, Box 3.2).

3.2.6 Evaluating the process

Assessing the success of horizon scans in identifying emerging issues is challenging, and has rarely been attempted. However, a recent review by Sutherland et al. (Reference Sutherland, Fleishman and Clout2019) examined the first of the annual global conservation scans described in Box 3.1 (Sutherland et al., Reference Sutherland, Clout and Côté2010) to consider how the issues identified in 2009 had developed. This was assessed using several approaches: a mini-review was carried out for each topic; the trajectory of the number of articles in the scientific literature and news media that mentioned each topic in the years before and after their identification was examined; and a Delphi-style scoring process was used to assess each topic’s change in importance. This showed that five of the 15 topics, including microplastic pollution, synthetic meat and environmental applications of mobile-sensing technology, appeared to have shown increased salience and effects. The development of six topics was considered moderate, three had not emerged and the effects of one topic were considered low.

As part of the same exercise, 12 global conservation organisations were questioned in 2010 about their awareness of, and current and anticipated involvement in, each of the topics identified in 2009 (Sutherland et al., Reference Sutherland, Allison and Aveling2012). This survey was repeated in 2018 (Sutherland et al., Reference Sutherland, Fleishman and Clout2019). Awareness of all topics had increased, with the largest increases associated with microplastic pollution and synthetic meat; the change in organisational involvement was highest for microplastics and mobile-sensing technology. Perhaps the most surprising result was the number that had not heard of what are now mainstream issues: 77% for microplastics, 54% for synthetic meat and 31% for the use of mobile sensing technology. A decade ago the idea of collecting environmental data using phones was cutting-edge.

Thus, efforts have begun to examine the development of previously identified horizon-scan topics, but further research into the impact of horizon scans, and a consideration of issues that may have been ‘missed’ (not identified but subsequently emerged as important) is needed.

3.3 Making a difference with horizon scanning

Gauging the extent to which horizon-scanning outputs inform policy, future research directions and resource investments is not always straightforward and no-one has yet tested the effectiveness of this process. In instances where the primary decision-making organisation uses horizon scanning internally to assist with deliberations (e.g. scans to set priorities for a government agency), actions can be mapped directly against outcomes. In these cases, implementing the actions indicates impact. In other cases, scans can be driven by a community outside of government to set agreed future directions that can then be used to persuade external resource allocators. Even in cases where policy appears to reflect issues flagged in a horizon scan, it is difficult to trace direct influence, as inputs from multiple sources are often blended in final policy decisions without attribution. It also may take years for real-world impact to be realised. Nevertheless, there are ways in which uptake of horizon-scanning output can be encouraged.

As a starting point, horizon scanning outputs can be matched to the organisations they are most relevant to. For example, policy-makers and practitioners can come together in a follow-up workshop to assess the importance of previously identified horizon-scanning issues for their organisation (Sutherland et al., Reference Sutherland, Allison and Aveling2012, Reference Sutherland, Fleishman and Clout2019). Or, the end user (e.g. policy-makers and practitioners) can be engaged in the horizon scan from the outset, as in a recent scan of research priorities for protected areas (Dudley et al., Reference Dudley, Hockings and Stolton2018). Similarly, horizon-scanning networks involving representatives from a range of government agencies, such as the Australasian Joint Agencies Scanning Network, or the UK Human Animal Infections and Risk Surveillance group, provide an ongoing forum for sharing information on new and emerging issues that potentially impact different departments and organisations. Regular meetings and reports are used to deliver this information to policy-makers in a timely way (Delaney & Osborne, Reference Delaney and Osborne2013).

In-depth follow-up analyses of horizon-scanning issues may also help policy-makers decide which to target first. A formal risk analysis of likelihood and consequences might be most appropriate for horizon-scanning outputs that compare similarly well-defined issues, for example, comparing one invasive species with another (e.g. Roy et al., Reference Roy, Peyton and Aldridge2014). It may be more challenging if some of the issues in the candidate set are more coarse-grained than others (e.g. comparing ocean warming with a specific emerging fungal disease in some snakes). Nonetheless, risk-based prioritisation at least offers a framework for comparing and forecasting issues (Brookes et al., Reference Brookes, Hernandez-Jover and Black2014) and for formally considering the strength of evidence for each (Garnett et al., Reference Garnett, Lickorish and Rocks2016).

Simply making horizon-scanning outputs known and available to policy-makers can encourage uptake. For example, issues identified in the annual global conservation scans (Box 3.1; Sutherland et al., Reference Sutherland, Butchart and Connor2018) have previously helped inform the UK’s Natural Environment Research Council ‘Forward Look’ strategic planning, but when a decision-maker does not already have a use in mind, it may be unclear what to do with horizon-scanning information without more context and guidance. Detecting signals and potential issues is only the first step towards making a difference: further intelligence about drivers is then needed to make sense of that information. For example, incorporating available data and modelling on air traffic movements with disease surveillance data might have helped anticipate the emergence of West Nile Virus in the United States in 1999 (Garmendia et al., Reference Garmendia, Van Kruiningen and French2001; Brookes et al., Reference Brookes, Hernandez-Jover and Black2014). It is the combination of horizon scanning, intelligence analysis (which provides context for the scanning output) and forecasting the chances of events unfolding that is particularly helpful in translating scanning outputs for policy-making. This can be embedded in a workflow, parts of which can be automated, such as compiling the context, narrative and structure into a digestible report on an important emerging issue (e.g. Box 3.3). When forecasting and open-analysis communities are already in place, this workflow can be delivered efficiently (Grossel et al., Reference Grossel, Lyon, Nunn, Robinson, Burgman, Nunn and Walshe2017).

Horizon scanning that occurs within organisations is evolving into a more effective tool than it was in its infancy. To facilitate the spread of best practice and reduce duplication, the UK has seen greater integration of horizon-scanning activities between different government departments, mainly in response to the Day Review (Reference Day2013). The review recommended that horizon scans: (i) look beyond short-term agendas and parliamentary terms, (ii) focus on specific areas rather than broad topics in order to get more traction, (iii) are championed by those who use them in strategic decision-making, (iv) produce shorter outputs that are more likely to get the attention of senior decision-makers and (v) draw on inputs and existing analyses sourced from a ‘wide range of external institutions, academia, industry specialists and foreign governments’. The extent to which all these recommendations have been implemented is unclear, but they represent a clear set of guidelines to follow.

There are a range of other useful frameworks that can be used for translating scanning outputs including roadmapping the steps towards acting on different horizon-scanning issues, for example, by assessing the feasibility and estimating how long it would take to develop technologies needed to address particular research gaps (Box 3.2; Kennicutt et al., Reference Kennicutt, Chown and Cassano2015). The Antarctic science scan and roadmap has since been used to set National Antarctic Program goals, judge the effectiveness and relevance of past investments, and guide investment of other national programmes (National Academies of Sciences Engineering and Medicine, 2015; www.nsf.gov/funding/pgm_summ.jsp?pims_id=505320&org=OPP&from=home).

3.4 Future directions

We have discussed some of the pros and cons of different approaches to horizon scanning. If using a manual approach, structured methods are essential for mitigating the social and psychological biases that human horizon scanners are prone to, especially when forecasting complex and uncertain futures (Hanea et al., Reference Hanea, McBride and Burgman2017). Although historically it has been criticised for confusing opinion with systematic prediction (Sackman, Reference Sackman1975), an iterative Delphi-style approach offers the advantage of drawing on the collective wisdom of a group, while affording individuals the opportunity to give private, anonymous judgements and revise them in light of information and reasoning provided by others. Compared with other elicitation approaches, such as traditional meetings, the Delphi method has also been found to improve forecasts and group judgements (Rowe & Wright, Reference Rowe, Wright and Armstrong2001). Manual approaches could be further improved by making the search for issues more systematic. Semi-automated tools and AI will increasingly enable searches uninfluenced by the biases of the manual searcher. For example, the Dutch ‘Metafore’ horizon-scanning approach (De Spiegeleire et al., Reference De Spiegeleire, van Duijne, Chivot, Daim, Chiavetta, Porter and Saritas2016), developed in The Hague Centre for Strategic Studies, already uses some automated approaches to systematically collect, parse, visualise and analyse a large ‘futures’ database to complement their manual scanning.

Future horizon scanning and intelligence gathering may also see more open-analysis, ‘citizen science’ tools becoming adopted. While organisations are increasingly scanning open-source material (including news and social media), analyses typically remain internal (Grossel et al., Reference Grossel, Lyon, Nunn, Robinson, Burgman, Nunn and Walshe2017). This means the analyses are generally not available to external users in an unfiltered form or in a timely way, which is particularly important for risks such as disease spread. Governments may opt for confidentiality for both security and political reasons. For instance, negative public perceptions about a suspected emerging herpes virus in oysters might affect trade, which might delay the disclosure of this information by authorities, in turn delaying risk mitigation actions (Grossel et al., Reference Grossel, Lyon, Nunn, Robinson, Burgman, Nunn and Walshe2017). Intelligence tools (e.g. Box 3.3) that draw on a community of users to openly analyse news and information on potentially emerging issues offer more timely and transparent synthesis of information, which encourages more responsive decision-making. Examples of this can be seen in citizen science, for example where citizen volunteers have helped analyse satellite-based information in the wake of natural disasters to help emergency responders to rapidly assess the damage (Yore, Reference Yore2017). In conservation science, involving a broader community of people in a participatory process like open-analysis may also increase public support for science and the environment (Dickinson & Bonney, Reference Dickinson and Bonney2012). More open-source and open-analysis scanning tools in the future will also likely be complemented with better information visualisation and GIS (e.g. including maps that indicate where a relevant incident has taken place) (Dickinson et al., Reference Dickinson, Shirk and Bonter2012), not only for identifying novel issues and monitoring issues that are already emerging, but also for locating and efficiently communicating this information.

Advanced text analytics, including text mining, will also provide a more comprehensive and systematic approach to future horizon scans. Indeed, some horizon-scanning centres, such as Singapore’s Risk Assessment and Horizon Scanning programme, already use sentiment analysis – a way of computationally categorising subjective opinions expressed in text (e.g. positive, negative or neutral) – to uncover themes in content retrieved by their analysts. Even more sophisticated text analytics are becoming available, for example, to explore areas of disagreement, conflict or debate in the text of scientific literature to help track developments in science and technology (Babko-Malaya et al., Reference Babko-Malaya, Meyers and Pustejovsky2013). They can also be used to detect language expressing excitement about a new idea, and other indicators of emergence, such as the increasing use of acronyms and abbreviations indicating that the scientific community is beginning to accept a technology or idea as established (Reardon, Reference Reardon2014). Through automation, new computational tools have the capacity to process a massive volume of papers and patents to anticipate which developments will have the biggest impact in the future (Murdick, Reference Murdick2015). These advances in text analytics have recently led to the development of a particularly powerful open-source AI tool, Meta (https://meta.org/), to help biomedical scientists and funders to connect emerging research areas and potential collaborators and inform investment. Due to the complexity of emerging issues (and complex environment for machines to learn in), progress towards detecting issues effectively through AI is slow. Computers may never outperform humans at natural language understanding, but steady improvements in the technology, coupled with the speed at which text can be processed by computers – in a range of languages – will undoubtedly add value to horizon scanning in the future.

3.5 Acknowledgements

BCW was supported by the Centre for the Study of Existential Risk at the University of Cambridge and funded by the Templeton World Charity Foundation (TWCF). WJS is funded by Arcadia. Geoff Grossel provided useful insights into online scanning methods and applications.

References
Amanatidou, E., Butter, M., Carabias, V., et al. 2012. On concepts and methods in horizon scanning: lessons from initiating policy dialogues on emerging issues. Science & Public Policy, 39, 208221.
Babko-Malaya, O., Meyers, A., Pustejovsky, J., et al. 2013. Modeling debate within a scientific community. In 2013 International Conference on Social Intelligence and Technology (pp. 5763). New York, NY: IEEE.
Brookes, V. J., Hernandez-Jover, M., Black, P. F., et al. 2014. Preparedness for emerging infectious diseases: pathways from anticipation to action. Epidemiology and Infection, 143, 20432058.
Burgman, M. 2015a. Governance for effective policy-relevant scientific research: the shared governance model. Asia & the Pacific Policy Studies, 2, 441451.
Burgman, M. A. 2015b. Trusting Judgements: How to Get the Best out of Experts. Cambridge: Cambridge University Press.
Cook, C. N., Inayatullah, S., Burgman, M. A., et al. 2014a. Strategic foresight: how planning for the unpredictable can improve environmental decision-making. Trends in Ecology & Evolution, 29, 531541.
Cook, C. N., Wintle, B. C., Aldrich, S. C., et al. 2014b. Using strategic foresight to assess conservation opportunity. Conservation Biology, 28, 14741483.
Day, J. 2013. Review of Cross-government Horizon Scanning. London: Cabinet Office.
De Spiegeleire, S., van Duijne, F. & Chivot, E. 2016. Towards Foresight 3.0: the HCSS Metafore Approach – a multilingual approach for exploring global foresights. In Daim, T. U., Chiavetta, D., Porter, A. L. & Saritas, O., editors, Anticipating Future Innovation Pathways Through Large Data Analysis (pp. 99117). Cham: Springer International Publishing.
Delaney, K. & Osborne, L. 2013. Public sector horizon scanning-stocktake of the Australasian joint agencies scanning network. Journal of Futures Studies, 17, 5570.
Dickinson, J. L. & Bonney, R. 2012. Citizen Science: Public Collaboration in Environmental Research.Ithaca, NY: Cornell University Press.
Dickinson, J. L., Shirk, J., Bonter, D., et al. 2012. The current state of citizen science as a tool for ecological research and public engagement. Frontiers in Ecology and the Environment, 10, 291297.
Dudley, N., Hockings, M., Stolton, S., et al. 2018. Priorities for protected area research. Parks, 24, 3550.
Esmail, N., Wintle, B.C., Sas-Rolfes, M., et al. 2019. Emerging illegal wildlife trade issues in 2018: a global horizon scan. SocArXiv. April 25. https://doi.org/10.31235/osf.io/b5azx
Garmendia, A. E., Van Kruiningen, H. J. & French, R. A. 2001. The West Nile virus: its recent emergence in North America. Microbes and Infection, 3, 223230.
Garnett, K., Lickorish, F. A., Rocks, S. A., et al. 2016. Integrating horizon scanning and strategic risk prioritisation using a weight of evidence framework to inform policy decisions. Science of the Total Environment, 560–561, 8291.
Glenn, J. C. & Gordon, T. J., editors. 2009. Futures Research Methodology – Version 3.0. The Millennium Project.
Government Office for Science. 2017. The Futures Toolkit, Edition 1.0.
Grossel, G., Lyon, A. & Nunn, M. 2017. Open-source intelligence gathering and open-analysis intelligence for biosecurity. In Robinson, A. P., Burgman, M. A., Nunn, M. & Walshe, T., editors, Invasive Species: Risk Assessment and Management (pp. 8492). Cambridge: Cambridge University Press.
Hanea, A. M., McBride, M., Burgman, M.A., et al. 2017. Investigate Discuss Estimate Aggregate for structured expert judgement. International Journal of Forecasting, 33, 267279.
Hartley, D. M., Nelson, N. P., Arthur, R. R., et al. 2013. An overview of Internet biosurveillance. Clinical Microbiology and Infection, 19, 10061013.
Hines, A., Bengston, D. N., Dockry, M. J., et al. 2018. Setting up a horizon scanning system: a U.S. federal agency example. World Futures Review, 10, 136151.
Inayatullah, S. 2013. Futures studies: theories and methods. In Gutierrez Junquera, F., editor, There’s a Future: Visions for a Better World (pp. 3666). Madrid: Banco Bilbao Vizcaya Argentaria Open Mind.
Kark, S., Sutherland, W. J., Shanas, U., et al. 2016. Priority questions and horizon scanning for conservation: a comparative study. PLOS ONE, 11, e0145978.
Kennicutt, M. C., Chown, S. J., Cassano, J. J., et al. 2014. Polar research: six priorities for Antarctic science. Nature, 512, 2325.
Kennicutt, M. C., Chown, S. J., Cassano, J. J., et al. 2015. A roadmap for Antarctic and Southern Ocean science for the next two decades and beyond. Antarctic Science, 27, 318.
Kluberg, S., Mekaru, S., McIver, D., et al. 2016. Global capacity for emerging infectious disease detection: 1996–2014. Emerging Infectious Diseases, 22(10), e151956.
Könnölä, T., Salo, A., Cagnin, C., et al. 2012. Facing the future: scanning, synthesizing and sense-making in horizon scanning. Science and Public Policy, 39, 222231.
Lesley, M., Floyd, J. & Oermann, M. 2002. Use of MindMapper software for research domain mapping. Computers Informatics Nursing, 20, 229235.
Linstone, H. A. & Turoff, M. 1975. The Delphi Method: Techniques and Applications. Reading, MA: Addison-Wesley Pub. Co.
Lyon, A. 2010. Review of online systems for biosecurity intelligence-gathering and analysis. ACERA Project 1003.
Mukherjee, N., Huge, J., Sutherland, W. J., et al. 2015. The Delphi technique in ecology and biological conservation: applications and guidelines. Methods in Ecology and Evolution, 6. 10971109.
Murdick, D. 2015. Foresight and Understanding from Scientific Exposition (FUSE): predicting technical emergence from scientific and patent literature. IARPA, editor. US Office of the Director of National Intelligence, www.iarpa.gov/images/files/programs/fuse/04-FUSE.pdf.
National Academies of Sciences Engineering and Medicine. 2015. A Strategic Vision for NSF Investment in Antarctic and Southern Ocean Research. Washington, DC: Author.
Page, S. E. 2008. The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies. Princeton, NJ: Princeton University Press.
Palomino, M. A., Bardsley, S., Bown, K., et al. 2012. Web-based horizon scanning: concepts and practice. Foresight, 14, 355373.
Parker, M., Acland, A., Armstrong, H. J., et al. 2014. Identifying the science and technology dimensions of emerging public policy issues through horizon scanning. PLoS ONE, 9, e96480.
Policy Horizons Canada. 2011. Leading the Pack or Lagging Behind: A Foresight Study on Environmental Sustainability and Competitiveness. Government of Canada.
Reardon, S. 2014. Text-mining offers clues to success: US intelligence programme analyses language in patents and papers to identify next big technologies. Nature News, 509, 410.
Reed, M. S., Graves, A., Dandy, N., et al. 2009. Who’s in and why? A typology of stakeholder analysis methods for natural resource management. Journal of Environmental Management, 90, 19331949.
Rowe, E., Wright, G. & Derbyshire, J. 2017. Enhancing horizon scanning by utilizing pre-developed scenarios: analysis of current practice and specification of a process improvement to aid the identification of important ‘weak signals’. Technological Forecasting & Social Change, 125, 224235.
Rowe, G. & Wright, G. 2001. Expert opinions in forecasting: role of the Delphi technique. In Armstrong, J. S., editor, Principles of Forecasting: A Handbook for Researchers and Practitioners (pp. 125144). Norwell, MA: Kluwer Academic Publishers.
Roy, H. E., Peyton, J., Aldridge, D. C., et al. 2014. Horizon scanning for invasive alien species with the potential to threaten biodiversity in Great Britain. Global Change Biology, 20, 38593871.
Sackman, H. 1975. Delphi Critique: Expert Opinion, Forecasting, and Group Process. Lexington, MA: Lexington Books.
Salathé, M., Bengtsson, M., Bodnar, T. J., et al. 2012. Digital epidemiology. PLoS Computational Biology, 8, e1002616.
Saritas, O. & Miles, I. 2012. Scan‐4‐Light: a Searchlight function horizon scanning and trend monitoring project. Foresight, 14, 489510.
Schultz, W. L. 2006. The cultural contradictions of managing change: using horizon scanning in an evidence-based policy context. Foresight, 8, 312.
Sutherland, W. J., Adams, W. M., Aronson, R. B., et al. 2009. One hundred questions of importance to the conservation of global biological diversity. Conservation Biology, 23, 557567.
Sutherland, W. J., Allison, H., Aveling, R., et al. 2012. Enhancing the value of horizon scanning through collaborative review. Oryx, 46, 368374.
Sutherland, W. J., Bailey, M. J., Bainbridge, I. P., et al. 2008. Future novel threats and opportunities facing UK biodiversity identified by horizon scanning. Journal of Applied Ecology, 45, 821833.
Sutherland, W. J. & Burgman, M. 2015. Policy advice: use experts wisely. Nature, 526, 317318.
Sutherland, W. J., Butchart, S. H. M., Connor, B., et al. 2018. A 2018 horizon scan of emerging issues for global conservation and biological diversity. Trends in Ecology and Evolution, 33, 4758.
Sutherland, W. J., Clout, M., Côté, I. M., et al. 2010. A horizon scan of global conservation issues for 2010. Trends in Ecology and Evolution, 25, 17.
Sutherland, W. J., Fleishman, E., Clout, M., et al. 2019. Ten years on: a review of the first global conservation horizon scan. Trends in Ecology and Evolution, 34, 139153.
Sutherland, W. J., Fleishman, E., Mascia, M. B., et al. 2011. Methods for collaboratively identifying research priorities and emerging issues in science and policy. Methods in Ecology and Evolution, 2, 238247.
Sutherland, W. J. & Woodroof, H. J. 2009. The need for environmental horizon scanning. Trends in Ecology & Evolution, 24, 523527.
van Rij, V. 2010. Joint horizon scanning: identifying common strategic choices and questions for knowledge. Science and Public Policy, 37, 718.
Wintle, B. C., Boehm, C. R., Rhodes, C., et al. 2017. A transatlantic perspective on 20 emerging issues in biological engineering. Elife, 6, e30247.
World Economic Forum. 2019. The Global Risks Report 2019: 14th Edition. Geneva: WEF.
Yore, R. 2017. Here’s how citizen scientists assisted with the disaster response in the Caribbean. The Conversation, 18 October 2017.

Chapter Four Generating, collating and using evidence for conservation

John D. Altringham
University of Leeds
Anna Berthinussen
Conservation First
and Claire F. R. Wordley
University of Cambridge
4.1 Introduction

Does scientific evidence really matter in conservation? In this chapter we will argue that generating, collating and using scientific evidence is key to effective conservation, illustrated by a case study from our own work: how to get bats to safely cross roads. We tell the story of bat ‘gantries’ or bridges, and show what can go wrong in the absence of robust studies that test the effectiveness of conservation interventions. We will also discuss the importance of collating or synthesising multiple strands of evidence to identify the factors that make a conservation measure effective or ineffective, using a case study on underpasses under roads. Finally, we explore a key challenge – getting scientific evidence accepted and used routinely in conservation policy and practice.

Evidence takes a multitude of forms and can be defined in many ways, but in this chapter we will mostly use ‘evidence’ to refer to scientific tests of treatments or interventions, which compare the ‘treatment’ to a ‘control’ in some way and measure the effect quantitatively. We define evidence in this way as it is a broad description that can still address causality for interventions – did treatment X cause reaction Y? For example, it is not enough to know that some bats flew along bat gantries – we need to know, at a minimum, how many, and how many still flew low across the road. But more on that later.

4.2 Why do we need evidence-based conservation?

Modern medicine has many examples illustrating why the discipline needs a robust evidence base. However, basing medical treatments on scientific evidence was not always the norm. The use of randomised controlled trials to test medical treatments was initially considered unnecessary and unethical, and it was hotly contested. A good example comes from an early champion of evidence-based medicine, Archie Cochrane, who demonstrated that randomised controlled trials were necessary and that expert judgement alone could be flawed. In 1971, he presented preliminary results from a trial comparing home care for heart patients with care in the new Coronary Care Units (note that the findings may not be the same now). He had been criticised for risking the lives of patients allocated to the ‘home care’ group. What follows is in his own words:

The results at that stage showed a slight numerical advantage for those who had been treated at home. I rather wickedly compiled two reports: one reversing the number of deaths on the two sides of the trial. As we were going into the committee, in the anteroom, I showed some cardiologists the results. They were vociferous in their abuse: ‘Archie’, they said ‘we always thought you were unethical. You must stop this trial at once’. I let them have their say for some time, then apologized and gave them the true results, challenging them to say as vehemently, that coronary care units should be stopped immediately. There was dead silence and I felt rather sick because they were, after all, my medical colleagues.

Results such as these – where the preferred treatment of the time did not work, or actually made things worse – are used to demonstrate why scientific studies of impacts are important when treating people. A growing body of literature suggests that impact studies are also necessary for treating the health of the biosphere, although the ‘gold standard’ of randomised controlled trials is not always possible in this discipline (Pynegar et al., Reference Pynegar, Gibbons and Asquith2018). As we test more and more measures to conserve species and habitats, we find that many do not work. For example, studies have shown that widely used methods to make water voles move prior to building works were ineffective, risking accidental killing of the protected mammals (Gelling et al., Reference Gelling, Harrington and Dean2018); that reintroduction programmes of species from macaws (Volpe et al., Reference Volpe, Di Giacomo and Berkunsky2017) to tamarins (Beck et al., Reference Beck, Dietz and Castro1991) have resulted in high or total mortality for the released animals; and that artificial bat roosts built to replace those destroyed during building works often failed to attract any bats and, even when occupied, hosted about half the number of bats that the destroyed roost had (Stone et al., Reference Stone, Jones and Harris2013). These results underline the need to test conservation solutions and not to simply assume that good intentions will lead to good outcomes.

Our case studies focus on the environmental impacts of roads. Road construction has been shown to harm animals through habitat degradation, loss and fragmentation, direct mortality and barrier effects (Laurance et al., Reference Laurance, Goosem and Laurance2009; Benítez-Lόpez et al., Reference Benítez-Lόpez, Alkemade and Verweij2010; Rytwinski & Fahrig, Reference Rytwinski and Fahrig2012). Figure 4.1 summarises these cumulative impacts, which are likely to act at different rates and through a long extinction debt. Unfortunately, studies on a wide range of habitats and taxa from grasslands to vertebrates show that many road mitigation options simply do not work. A growing list of papers points to not only poor design of mitigation and monitoring, but a wider context of poor target setting, weak implementation, inadequate reporting and poor or absent enforcement (e.g. Rundcrantz, Reference Rundcrantz2006; Tischew et al., Reference Tischew, Baasch and Conrad2010; Beebee, Reference Beebee2013; Drayson & Thompson, Reference Drayson and Thompson2013; Villarroya & Puig, Reference Villarroya and Puig2013). We will not address all of these factors in this chapter, but they are important to consider when asking why ineffective measures have persisted for so long.

Figure 4.1 The multiple causes of bat population reduction by road construction and the delayed response (extinction debt).

Adapted from Forman et al. (Reference Forman, Sperling and Bissonette2003).
4.3 Case study: bats and roads

Why do we need mitigation for bats crossing roads? In the EU, all bat species have been protected under the EUROBATS agreement since 1994, in recognition of the declining populations of many species. As a consequence, whenever populations may be adversely affected by human activity, impact assessment and mitigation are a legal obligation. Over the last 10 years evidence for significant effects of roads on bats has grown and the need for effective mitigation has become increasingly evident (e.g. Altringham, Reference Altringham2008; Russell et al., Reference Russell, Butchkoski and Saidak2009; Lesiński et al., Reference Lesiński, Sikora and Olszewski2010; Berthinussen & Altringham Reference Berthinussen and Altringham2012b, Reference Berthinussen and Altringham2015). There are clear specifications within EUROBATS to mitigate against the impacts of roads on bats.

4.3.1 The need to test mitigation: bat gantries

The EUROBATS commitment to mitigate against the impact of roads on bats is very positive, but are the mitigation strategies being used actually working? Early studies assessed the use by bats of underpasses and overpasses primarily built for other purposes, such as to carry minor roads, footpaths or streams. If bats were seen near to these structures it was generally assumed that they were effective mitigation tools (Highways Agency, 2001, 2006; reports reviewed by O’Connor et al., Reference O’ Connor, Green and Wilson2011). Underpasses, culverts, footbridges and bridges for vehicles, all of various sizes, were widely adopted as mitigation solutions (Figure 4.2). Many were not subsequently surveyed for use by bats, or qualitative surveys were written up in often confidential reports. Many studies reported ‘use’ – small numbers of bats observed in underpasses or flying over bridges of various kinds, without reference to the number still crossing the road unsafely, or not crossing at all (see Highways Agency, 2006; O’Connor et al., Reference O’ Connor, Green and Wilson2011), and many lacked convincing definitions of use. This meant that future projects could not learn from the success or failure of previously built mitigation structures.

Figure 4.2 Two underpasses found to vary in effectiveness in guiding bats safely under roads. (a) An effective underpass on the A590, Cumbria, UK; (b) an ineffective underpass on the A66, Cumbria, UK. Boxplots show the number of bats crossing per survey using the underpass and crossing over the road above at safe and unsafe heights (above and below 5 m, traffic height). The variable success of underpasses underlines the need to understand the details of conservation interventions; in this example, the location of the underpasses impacted on how effective they were.

From Berthinussen and Altringham (Reference Berthinussen and Altringham2012b).

In addition to multi-use structures, some ‘bespoke’ structures were built and ‘bat gantries’ or ‘wire bridges’ (Figure 4.3) were widely adopted. Bat gantries were assumed to act as navigational aids to echolocating bats, encouraging them to continue using existing ‘commuting routes’ from roosts to feeding areas (which often follow linear features such as hedgerows) after road construction, but lifting them above the traffic. Ideally, crossing points should be built on known bat commuting routes determined by pre-construction surveys, as bats tend to be faithful to particular routes. However, many were built away from known bat commuting routes for engineering reasons, to fit in with landscape topography, to combine bat routes with minor roads or footpaths, or simply to reduce cost. It was assumed bats would find the new crossing points (Highways Agency, 2001), and in some cases new hedge planting was designed to guide them to these structures. In many guidance documents, environmental statements and mitigation plans it was implicit, or even explicit, that the bats would respond as predicted (Highways Agency, 2001; Limpens et al., Reference Limpens, Twisk and Veenbaas2005).

Figure 4.3 Two bat gantry designs: (a) wire mesh design on the A11, Norfolk, UK; (b) wire and ball design on the A590, Cumbria, UK. Boxplots show the results of surveys carried out to test the effectiveness of the gantries in guiding bats safely over the road. Data were recorded for the total number of bats crossing per survey, the numbers crossing at unsafe heights (below 5 m, traffic height) and the numbers using the gantry according to two definitions of ‘use’ (flying within either 2 m or 5 m of the wires above traffic height). The bat gantry story neatly demonstrates the need to test conservation interventions before rolling them out on a wide scale.

In 2008, JDA was asked to provide evidence to a public inquiry for the effectiveness of these strategies (Altringham, Reference Altringham2008). No quantitative evidence was found to suggest that any of the strategies implemented were effective in protecting bats, particularly at the population level. However, neither was there evidence to suggest that they were ineffective. This prompted us to conduct our own research to determine the effects of roads on bats and the effectiveness of mitigation (Berthinussen & Altringham, Reference Berthinussen and Altringham2012a, Reference Berthinussen and Altringham2012b, Reference Berthinussen and Altringham2015).

In our research we emphasised the difference between qualitative assessments of the ‘use’ of a structure by a small number of bats and measures of effective protection at the population level. We also stressed that the number of bats present pre-construction is rarely assessed, meaning that post-construction bat numbers may already be a fraction of what was there before. We proposed two broad measures of effectiveness: (1) measurements of local bat activity and of the movement of bats along severed commuting routes before and after road construction, to assess landscape-scale impact and the permeability of new roads; and (2) measurements of the effectiveness of the crossing structures – the proportions of bats that use them to cross safely. Our research was limited by logistics to the second measure – do mitigation structures guide bats safely across roads?

The headline result was that ‘wire and ball’ bat gantries did not alter the behaviour of bats crossing roads – they were wholly ineffective (Berthinussen & Altringham, Reference Berthinussen and Altringham2012a; Figure 4.3b). This was a disturbing finding, as over the previous decade about 15 gantries had been built in the UK and continental Europe. Although our study showed that one design of bat gantry was ineffective, it was suggested that other designs would have greater success in guiding bats to fly at safe heights above roads. Our next study found that ‘wire mesh’ gantries (Figure 4.3a, of a different design to the ‘wire and ball’ structures) were equally ineffective (Berthinussen & Altringham, Reference Berthinussen and Altringham2015).

In summary, a mitigation measure widely used for over a decade was essentially untested and subsequently shown to be ineffective. This underscores the need for rigorous testing of the measures that we implement in the name of conservation. We also found (albeit based on a small sample size) that building all types of crossing away from known commuting routes, even with new planting to guide bats to them, was unsuccessful (Berthinussen & Altringham, Reference Berthinussen and Altringham2012a). This is important, as it shows that the location of mitigation measures is as important as the measure itself – effective measures need to be implemented with a good understanding of the local context. Furthermore, we found evidence that some underpasses were used by a high proportion of bats, and that the one green bridge tested in the UK – a large structure planted with trees, shrubs and ground cover – was used by over 90% of bats crossing the road in that area, suggesting that effective ways to allow bats to safely cross roads do exist.

4.4 Synthesising evidence

The bat gantry case study provides some insight into why we need to rigorously test the effectiveness of measures aiming to protect the natural world. However, this is just the first step towards implementing a truly evidence-based approach to conservation. The next step is to systematically bring together all the evidence, from many studies, on particular conservation measures. This approach is also borrowed from evidence-based medicine, where it has proven to be a lifesaver.

One of the most important developments in evidence-based medicine was the Cochrane Collaboration, an organisation set up to conduct systematic reviews of the scientific evidence on topics such as how well different treatments worked. In medicine – as in conservation – natural variation in populations means that it often takes large numbers of replicates for beneficial or detrimental effects to become apparent. Modern doctors, making potentially life-changing decisions, want to have the information on every study on a particular treatment to hand, not just the results from a single trial that may not be representative. The goal should be the same in conservation: to bring together all the evidence for an intervention to assess whether it works, whether it does harm, or whether it only works in certain situations or with certain variations of the intervention.

There are many examples of the importance of collating evidence in medicine. For example, a systematic review on cot death or sudden infant death syndrome (SIDS), using the studies already available in the 1970s, could have saved the lives of an estimated 60,000 babies. Due to a lack of evidence synthesis and an overreliance on expert opinion, medical practitioners advised parents to put children to sleep on their fronts until the 1990s, when studies and reviews led to the realisation that this sleeping position increased the risk of SIDS (Gilbert et al., Reference Gilbert, Salanti and Harden2005).

In conservation, collating or synthesising the data is as critical as it is in medicine (Sutherland et al., Reference Sutherland, Pullin and Dolman2004). While the most rigorous method, systematic review, is very important (see Chapter 7, the Collaboration for Environmental Evidence and Mistra EviEM), more cost-effective methods of collating the evidence may also be desirable in this underfunded discipline, where the evidence itself can be scarce and variable in quality (Sutherland & Wordley, Reference Sutherland and Wordley2018). ‘Synopses’ published by Conservation Evidence (www.conservationevidence.com) follow one such method, known as subject-wide evidence synthesis (Sutherland & Wordley, Reference Sutherland and Wordley2018). Researchers draw up lists of all the interventions that could benefit a given taxa or habitat, classified according to potential threats based on IUCN criteria (Threats Classification Scheme Version 3.2); the scientific studies for the effectiveness of each intervention are then collated and summarised. For example, we produced the Bat Synopsis (Berthinussen et al., Reference Berthinussen, Richardson and Altringham2013, updated 2019), which provides key messages and summaries of the relevant studies, to help conservationists see which interventions for bat conservation are likely to be the most effective, and under which circumstances. The summary of this synopsis in What Works in Conservation (Berthinussen et al., Reference Berthinussen, Richardson, Smith, Sutherland, Dicks and Petrovan2018) takes this a step further, by using expert scoring to categorise the interventions based on levels of effectiveness, certainty in the evidence available and potential harms.

The first Bat Synopsis (Berthinussen et al., Reference Berthinussen, Richardson and Altringham2013) listed 78 interventions that could be implemented to conserve bats, covering areas as diverse as logging, roost provision and wind turbine operation regimes. No evidence for effectiveness was found for 48 of the 78 interventions, many of which are used routinely in the UK and elsewhere. This does not mean they are ineffective, but simply that they had not been tested quantitatively when we checked the literature. For a further 12 interventions the evidence was too limited for assessment. This demonstrates the scarcity of experimental evidence for many possible management actions, severely limiting the ability of conservationists, ecological consultants, developers and government agencies to undertake evidence-based conservation or mitigation.

Of the 18 remaining interventions, 14 had some proven value as conservation tools for bats. These included using selective logging instead of conventional logging, turning off wind turbines at low wind speeds and minimising light pollution. An update to this synopsis was published in 2019 (Sutherland et al., Reference Sutherland, Dicks, Ockendon, Petrovan and Smith2019), expanding the list of interventions to 190 and adding new studies that were published in the intervening years. There are many interventions which have had valuable evidence added in this update, but we have not yet seen a shift to a majority of interventions being tested via multiple high-quality experiments.

4.4.1 Example of evidence synthesis: road underpasses

For many of the interventions addressed in the Bat Synopsis, our greatest contribution was to demonstrate that no evidence existed for the efficacy of these measures – hopefully spurring more research and a more critical eye towards choosing conservation measures. But for a handful, we could begin to tease out what made an intervention effective in some circumstances but not others – one of the many benefits of summarising multiple studies. One such intervention is the use of underpasses to get bats to cross roads safely.

In the 2013 Bat Synopsis we found four studies, from Germany, Ireland and the UK, which between them showed that at least nine bat species used underpasses (none purpose-built for bats), with up to 96% of the bats crossing through underpasses rather than the road above (although this varied greatly) (Berthinussen et al., Reference Berthinussen, Richardson and Altringham2013). By summarising the key details of each study, we can see that some species use underpasses frequently while others do not appear to use underpasses at all, and that only a few species appear to use small underpasses, such as drainage pipes of diameter less than 1.5 m. There are indications that effectiveness increases with diameter and when underpasses are placed on known bat commuting routes – conclusions supported by ongoing studies (Davies, Reference Davies2019). The 2019 update of the Bat Synopsis added two further studies, which tested much larger underpasses and still found the largest structures to be the most effective, but also explored the differing responses of various functional guilds of bats. These details are critical. Further testing and refining of underpasses, followed by evidence synthesis, should help to ensure that future underpasses are as effective as they can be.

4.5 Getting the evidence used

We are trying to bring this work, demonstrating the importance of generating and using evidence on the effectiveness of interventions, to as large an audience as possible, to ensure that those responsible for commissioning, designing, approving and testing mitigation structures are aware of it. The bat gantry studies have been reported in national newspapers, radio and television. This was achieved through press releases, by approaching media contacts directly and by being approachable when contacted. The work has also been reported in several books and papers (Altringham, Reference Altringham2011; Abbott et al., Reference Abbott, Berthinussen, Boonman, van der Ree, Grilo and Smith2015; Altringham & Kerth, Reference Altringham, Kerth, Voigt and Kingston2016; Sutherland & Wordley, Reference Sutherland and Wordley2017). JDA and AB have run seven workshops for practitioners on road mitigation measures for bats and talked at over 10 conferences in the UK and abroad. CFRW has mentioned this study in around 50 talks to conservationists and government agencies and used it as an example in an opinion piece on evidence use in conservation (Sutherland & Wordley, Reference Sutherland and Wordley2017). The Bat Synopsis and What Works in Conservation, which contains a summary of the Bat Synopsis, have also been widely promoted.

This awareness resulted in tens of thousands of views of the paper and relevant parts of the Bat Synopsis, and this exposure has translated into further successes. The impact of early work (Berthinussen & Altringham, Reference Berthinussen and Altringham2012a, Reference Berthinussen and Altringham2012b) led to a Defra-funded project to develop better mitigation monitoring protocols (Berthinussen & Altringham, Reference Berthinussen and Altringham2015) and a statutory conservation agency guidance note summarising the protocols. The approximately £1 M spent on bat gantries in the UK as of 2017 (Sutherland & Wordley, Reference Sutherland and Wordley2017) was brought up in the House of Lords by Lord John Krebs in January 2018, who used it to demonstrate why the UK government’s 25-year environment plan needed to explicitly commit to being evidence-based. Some road-building projects have taken heed of the evidence. The A40 Penblewin to Slebech Park Improvement in Wales opted to mitigate impacts on bats using underpasses of varying sizes, many built on known commuting routes, and funded more rigorous monitoring (Davies, Reference Davies2019).

However, not everyone is listening. Despite widespread reporting of the ineffectiveness of bat gantries in 2012, six gantries of a ‘wire-mesh’ design were built in Norfolk, England in 2014 at a reported cost of £350,000. These were probably planned before the 2012 paper was published, but plans were not modified in light of this study. In 2015 these gantries were also shown not to work (Berthinussen & Altringham, Reference Berthinussen and Altringham2015). Nevertheless, seven more gantries are under construction (as of 2018) at a cost of over £1 M on the North Norfolk Distributor Road (MacDonald, Reference MacDonald2014). In another example, the environmental statement for the proposed and controversial extension to the M4 across the Gwent Levels in Wales (Welsh Government, 2016) draws extensively on our 2015 Defra report (Berthinussen & Altringham, Reference Berthinussen and Altringham2015). However, it proposes numerous culverts for bats which, by the authors’ own admission, are almost all too small to be used by the target bat species. In addition, most will not be on known commuting routes.

Furthermore, there are still inadequate mechanisms in place to assess the effectiveness of mitigation measures. A feature of many environmental statements and mitigation plans is the absence of a monitoring plan capable of assessing mitigation success or failure. There is frequently no monitoring plan at all. This appears to be due to a reluctance or inability of government agencies to enforce effective monitoring, a reluctance on the part of many developers to pay for monitoring and a lack of understanding about how to design and conduct monitoring that is fit for purpose. As a result, developers and taxpayers spend money on unproven mitigation with no prospect of improved understanding.

4.5.1 Why is evidence ignored?

Why are proven methods rejected, often in favour of methods that have been demonstrated not to work? Why is there an apparent reluctance to seek out, use or accept evidence, or to collect it, among some decision-makers, including some government agencies and ecological consultants? Sutherland and Wordley (Reference Sutherland and Wordley2017) explored a few general psychological and structural reasons for this phenomenon, and more detail on this topic is given in Chapter 9 of this book. Here we share some of our own experiences of the failure to use evidence in road mitigation.

The real or perceived higher financial cost of effective mitigation solutions is one concern. Mitigation consumes a very small part of the total cost of a road-building project, but mitigation and monitoring are obvious targets when budgets are tight or overrun. Effective mitigation may or may not be more expensive than ineffective options, but ineffective mitigation is simply a waste of resources.

A desire to simplify the planning and implementation of mitigation is another reason why some parties are reluctant to challenge or change accepted approaches. Road building is complex, making off-the-shelf, approved mitigation solutions an attractive option. Being able to implement development projects as quickly and cheaply as possible can make mitigation a tick-box exercise – complying with regulation at minimal cost may be more important than implementing effective mitigation. Mitigation solutions such as bat gantries can be designed and built relatively cheaply and, if experts say they will work, then they fulfil all legal requirements and may be assumed to require little or no ‘expensive’ monitoring. To question their effectiveness can put in jeopardy budgets, work schedules, building specifications, even the project itself. A reluctance to listen to objections is understandable, but not excusable. Consultant ecologists can be placed in a difficult position. Their scientific training, personal concern about nature and professional standards all demand unbiased assessment. However, their livelihoods depend upon contracts from developers who are frequently not obliged to commit to effective mitigation and monitoring.

Finding, evaluating and applying the evidence on mitigation strategies can also be a challenge. With the existence of freely available downloadable material (such as the Bat Synopsis) in a concise, jargon-free form, decision-makers should be more aware of what works and what does not. However, ecological consultants and statutory agency staff still need time to find, read and digest the information, and require some scientific training to evaluate the evidence. The difference between quantitative evidence and anecdote is not always understood and ‘professional judgement’ may be relied on even when it runs counter to the evidence. However, it does not have to be painful for developers, consultants or planners to improve on current practice. Adoption of good mitigation practices early in a project can avoid the problems of making corrections during the project, and investment in effective technologies may lower the costs of solutions such as large underpasses and green bridges.

4.6 How can evidence use in mitigation projects be improved?

First, there should be a key requirement that mitigation structures are tested for effectiveness, not just use, and a quantitative bar set for effectiveness (Berthinussen & Altringham, Reference Berthinussen and Altringham2015). Ecologists employed to assess mitigation effectiveness must be prepared to shun options proven to be ineffective. Professional bodies must fully support ecological consultants in implementing those measures shown to be effective, and sanction members who use methods known to be ineffective. Improvements may be much more evident if the enforcing statutory agencies are willing and able to deny planning permission to development projects that have poor mitigation strategies. There should be real commitment from governments to pledge to conserve species and habitats using evidence-based measures and discarding measures proven to be ineffective. This may require additional resources to assess existing and proposed legislation against evidence syntheses.

To identify effective and ineffective solutions there is a clear need for dedicated funding for rigorous tests of interventions. Monitoring interventions often requires long-term commitment which, in turn, requires adequate long-term funding. This could come from developers and government agencies, but a greater recognition by academic funding bodies of the value of applied questions would also have a huge impact. PhD projects could be encouraged to have applied components, testing interventions. Research council funding for academics to address applied questions of conservation importance and communicate them to practitioners would have a huge impact.

Greater power for statutory agencies to enforce existing laws, check up on implementation and demand replacements for ineffective solutions would dramatically improve mitigation effectiveness. A framework with greater incentives for developers to show that their mitigation has been effective would be beneficial. These could include a requirement to make the results of mitigation monitoring for effectiveness public, penalties for failures to do so and awards for new, proven effective solutions.

While many of these goals may not be realised in the near future, we can all promote approaches to conservation that are evidence-based and effective. If enough of us do it, it might just change the world.

References
Abbott, I., Berthinussen, A., Boonman, M., et al. 2015. Bats and roads. In van der Ree, R., Grilo, C. & Smith, D., editors, Handbook of Road Ecology (pp. 290299). Chichester: John Wiley & Sons. doi:10.1002/9781118568170.ch39
Altringham, J. D. 2008. Bat Ecology and Mitigation; Proof of Evidence; Public enquiry into the A350 Westbury bypass. Neston: White Horse Alliance.
Altringham, J. D. 2011. Bats: From Evolution to Conservation, 2nd edition. Oxford: Oxford University Press. doi:10.1093/acprof:osobl/9780199207114.001.0001
Altringham, J. & Kerth, G. 2016. Bats and roads. In Voigt, C. C. & Kingston,T., editors, Bats in the Anthropocene: Conservation of Bats in a Changing World (pp. 3562). Cham: Springer International Publishing. https://doi.org/10.1007/978–3-319–25220-9
Beck, B., Dietz, J., Castro, L., et al. 1991.Losses and reproduction of reintroduced golden lion tamarins Leontopithecus rosalia. Dodo, 27, 5061.
Beebee, T. J. C. 2013. Effects of road mortality and mitigation measures on amphibian populations. Conservation Biology, 27, 657668. doi:10.1111/cobi.12063
Benítez-Lόpez, A., Alkemade, R. & Verweij, P. A. 2010. The impacts of roads and other infrastructure on mammal and bird populations: a meta-analysis. Biological Conservation, 143, 13071316. https://doi.org/10.1016/j.biocon.2010.02.009https://doi.org/10.1016/j.biocon.2010.02.009
Berthinussen, A. & Altringham, J. 2012a. Do bat gantries and underpasses help bats cross roads safely? PLoS ONE, 8, e38775. https://doi.org/10.1371/journal.pone.0038775
Berthinussen, A. & Altringham, J. 2012b. The effect of a major road on bat activity and diversity. Journal of Applied Ecology, 49, 8289. doi:10.1111/j.1365-2664.2011.02068.x
Berthinussen, A. & Altringham, J. 2015. WC1060: Development of a cost-effective method for monitoring the effectiveness of mitigation for bats crossing linear transport infrastructure. Final report to Defra. http://sciencesearch.defra.gov.uk/Default.aspx?Module=More%26Location=None%26ProjectID=18518
Berthinussen, A., Richardson, O. & Altringham, J. 2013. Bat Conservation: Evidence for the Effects of Interventions. Synopses of Conservation Evidence series.Exeter: Pelagic Publishing.
Berthinussen, A., Richardson, O. C., Smith, R. K., et al. 2018. Bat conservation. In Sutherland, W. J., Dicks, L. V., Petrovan, S. O., et al., editors, What Works in Conservation (pp. 6793). Cambridge: Open Book Publishers. http://dx.doi.org/10.11647/OBP.0060
Davies, J. 2019. Monitoring the effectiveness of mitigation for horseshoe bats associated with a new road in Wales. Conservation Evidence, 16, 1723.
Drayson, K. & Thompson, S. 2013. Ecological mitigation measure in English environmental impact assessment. Journal of Environmental Management, 119, 103110. https://doi.org/10.1016/j.jenvman.2012.12.050
Forman, R. T. T., Sperling, D., Bissonette, J. A., et al. 2003. Road Ecology: Science and Solutions. Washington, DC: Island Press.
Gelling, M., Harrington, A. L., Dean, M., et al. 2018. The effect of using ‘displacement’ to encourage the movement of water voles Arvicola amphibius in lowland England. Conservation Evidence, 15, 2025.
Gilbert, R., Salanti, G., Harden, M., et al. 2005. Infant sleeping position and the sudden infant death syndrome: systematic review of observational studies and historical review of recommendations from 1940 to 2002. International Journal of Epidemiology, 34, 874887. doi:10.1093/ije/dyi088
Highways Agency. 2001. Nature Conservation Advice in Relation to Bats: Design Manual for Roads and Bridges. Volume 10, Environmental Design. Section 4, Nature Conservation. Part 3, HA80/99. Guildford: Highways Agency UK.
Highways Agency. 2006. Best Practice in Enhancement of Highway Design for Bats. Guildford: Highways Agency and Bat Conservation Trust.
Laurance, W. F., Goosem, M. & Laurance, S. G. 2009. Impacts of roads and linear clearings on tropical forests. Trends in Ecology & Evolution, 24, 659669. https://doi.org/10.1016/j.tree.2009.06.009
Lesiński, G., Sikora, A. & Olszewski, A. 2010. Bat casualties on a road crossing a mosaic landscape. European Journal of Wildlife Research, 57, 217223. https://doi.org/10.1007/s10344-010–0414-9
Limpens, H. J. G. A., Twisk, P. & Veenbaas, G. 2005. Bats and road construction. Dutch Ministry of Transport, Public Works and Water Management Directorate-General for Public Works and Water Management, Road and Hydraulic Engineering Institute, Delft, the Netherlands and the Association for the Study and Conservation of Mammals, Arnhem, the Netherlands
MacDonald, M. 2014. The Norfolk County Council (Norwich Northern Distributor Road (A1067 to A47(T))) Order 6.2 Environmental Statement: Volume II: Chapter 2. The Scheme. PINS Reference Number: TR010015 Document Reference: 6.2 Regulation Number: 5(2)(a).
Maynard, A. & Chalmers, I., editors. 1997. Non-random Reflections: On Health Services Research: On the 25th Anniversary of Archie Cochrane’s Effectiveness and Efficiency. London: BMJ Books. https://doi.org/10.1136/bmj.316.7143.1543
O’ Connor, G., Green, R. & Wilson, S. 2011. A Review of Bat Mitigation in Relation to Highway Severance. Guildford: Highways Agency.
Pynegar, E. L., Gibbons, J. M., Asquith, N. M., et al. 2018. What role should Randomised Control Trials play in providing the evidence base underpinning conservation?. PeerJ PrePrints.
Rundcrantz, K. 2006. Environmental compensation in Swedish road planning. European Environment, 16, 350367. doi:10.1002/eet.429
Russell, A. L., Butchkoski, C. M., Saidak, L., et al. 2009. Road-killed bats, highway design, and the commuting ecology of bats. Endangered Species Research, 8, 4960. doi:10.3354/esr00121
Rytwinski, T. & Fahrig, L. 2012. Do species life history traits explain population responses to roads? A meta-analysis. Biological Conservation, 147, 8798. http://dx.doi.org/10.1016/j.biocon.2011.11.023
Stone, E. L., Jones, G. & Harris, S. 2013. Mitigating the effect of development on bats in England with derogation licensing. Conservation Biology, 27, 13241334. doi:10.1111/cobi.12154
Sutherland, W.J., Dicks, L.V., Ockendon, N., Petrovan, S.O., and Smith, R.K. What Works in Conservation 2019. Cambridge, UK: Open Book Publishers, 2019
Sutherland, W. J., Pullin, A. S., Dolman, P. M., et al. 2004. The need for evidence-based conservation. Trends in Ecology & Evolution, 19, 305308.
Sutherland, W. J. & Wordley, C. F. R. 2017. Evidence complacency hampers conservation. Nature Ecology & Evolution, 1, 12151216, doi:10.1038/s41559-017-0244-1
Sutherland, W. J. & Wordley, C. F. R. 2018. A fresh approach to evidence synthesis. Nature, 558, 364366. doi:10.1038/d41586-018-05472-8
Tischew, S., Baasch, A., Conrad, M. K., et al. 2010. Evaluating restoration success of frequently implemented compensation measures: results and demands for control procedures. Restoration Ecology, 18, 467480. doi:10.1111/j.1526-100X.2008.00462.x
Villarroya, A. & Puig, J. 2013. A proposal to improve ecological compensation practice in road and railway projects in Spain. Environmental Impact Assessment Review, 42, 8794. http://dx.doi.org/10.1016/j.eiar.2012.11.002
Volpe, N. L., Di Giacomo, A. S. & Berkunsky, I. 2017. First experimental release of the red-and-green macaw Ara chloropterus in Corrientes, Argentina. Conservation Evidence, 14, 2020.
Welsh Government. 2016. M4 Corridor around Newport. Environmental Statement Volume 1. Chapter 10: Ecology and Nature Conservation. http://gov.wales/topics/transport/roads/schemes/m4/corridor-around-newport/environmental-info/ecology-nature/?lang=en

Chapter Five Understanding local resource users’ behaviour, perspectives and priorities to underpin conservation practice

E.J. Milner-Gulland , Harriet Ibbett
University of Oxford
Paulo Wilfred
The Open University of Tanzania
Hans Cosmas Ngoteya
Landscape and Conservation Mentors Organisation
and Peni Lestari
Wildlife Conservation Society Indonesia Program
5.1 Introduction

Most of the chapters in this book focus on how best to bring science into policy, often at the national scale and mostly with a developed-world perspective. Ensuring that national policy frameworks are conducive to conservation is vital, but it is also important to improve the effectiveness of science in supporting conservation interventions on the ground. Small-scale interventions aiming to change the behaviour of local resource users in developing countries make up a large proportion of global conservation effort and funding (Brockington & Scholfield, Reference Brockington and Scholfield2010). These types of intervention are challenging to do well, and often do not produce the desired results (Larrosa et al., Reference Larrosa, Carrasco and Milner-Gulland2016). Typically, there is little scientific input into either the design or evaluation of these projects, and evidence of effectiveness is limited (Roe et al., Reference Roe, Booker and Day2015). Small organisations in developing countries may not have the capacity or confidence to implement scientifically informed design and monitoring, and supporting them to collate and learn from evidence may not be a major priority for researchers or donors. Increased sharing of insights and techniques to support more robust and effective interventions could transform grassroots conservation (e.g. Woodhouse et al., Reference Woodhouse, de Lange and Milner-Gulland2016). In this spirit, we use case studies from four locations around the world to illustrate some of the challenging steps involved in understanding conservation issues and designing suitable interventions. These steps are often skipped or not made explicit, but are critical to success; they ensure that interventions have a strong foundation in evidence, making it more likely that their desired impacts are achieved.

First, we explore how to collect robust information on the prevalence of illegal resource use, as a first step towards understanding the extent of the problem, using a case study of bird hunting in a Cambodian grassland. Next, we consider how to bring together different sources of information to understand both resource use and local perspectives on conservation, using a case study from Tanzania. These two case studies about evidence gathering lead on to the next stage: intervention design. We start with an example, also from Tanzania, of developing a Theory of Change for a conservation intervention, in which the process by which actions lead to a desired result is identified, assumptions are made clear and the progress of the intervention towards its desired impact can be monitored. Finally, we explore the challenges of implementing one particularly prevalent intervention type – alternative livelihoods projects – using an example of a shark fishery in Indonesia. Together, these case studies provide a vivid illustration of the ways in which conservation researchers and practitioners are combining efforts to ensure that interventions are based on robust evidence and therefore more likely to succeed.

5.2 Asking questions about sensitive topics

Moderating human behaviour is critical to conservation success (Gore, Reference Gore2011; Milner-Gulland, Reference Milner-Gulland2012). However, if we are effectively to change human behaviour, we must first ensure we understand the nature of the behaviour we want to change. Central to this is determining both the prevalence of behaviours that are detrimental to biodiversity, and the characteristics of the people engaging in these behaviours. This is essential to ensure managers efficiently allocate resources to tackle threats, and that behavioural change interventions target the right audiences with the right incentives (St John et al., Reference St John, Edwards-Jones and Gibbons2010, Reference St John, Mai and Pei2015). However, obtaining such information can be extremely challenging, especially if the behaviour in question is illegal (Gavin et al., Reference Gavin, Solomon and Blank2010).

A common approach to ascertaining the true extent of illegal behaviours is asking direct questions (e.g. Gandiwa, Reference Gandiwa2011; Kiffner et al., Reference Kiffner, Peters and Stroming2015). Other studies mask the sensitivity of questions about illegal behaviours by mixing them with less-sensitive questions about other livelihood activities (e.g. Martin et al., Reference Martin, Caro and Mulder2012; Mgawe et al., Reference Mgawe, Mulder and Caro2012; Kiffner et al., Reference Kiffner, Peters and Stroming2015). Although direct questioning may help to cast some light on the nature of natural resource exploitation, it runs the risk of bias from untruthful responses (Nuno & St John, Reference Nuno and St John2015). Respondents may be scared to answer questions honestly for fear of incriminating themselves, or the possible repercussions they might face from revealing their behaviour. They may avoid answering questions altogether, terminate interviews early or underreport activities. If respondents do answer sensitive questions, social desirability bias may lead them to moderate their responses so their actions appear more socially acceptable. This is especially true of data captured in group settings, where pressure from peers may prevent others speaking freely and truthfully about their activities. It is also important to consider the ethical implications of directly asking respondents about their illegal activity; research has an ethical responsibility to ‘do no harm’, yet asking such questions can cause respondents to directly implicate themselves in illegal activities, potentially leading to severe consequences.

Indirect questioning has started to become more widely used in conservation science in response to some of these challenges. The method comes from psychology, and has been used when asking questions about sensitive issues such as drug use and racial prejudice (Imai, Reference Imai2011). The technique enables interviewees to respond in such a way that the interviewer cannot directly determine whether they have participated in the activity. Instead, data provide estimates of prevalence at the population level, affording both the respondent and the researcher greater levels of protection.

One form of indirect questioning increasingly applied in conservation is the Unmatched Count Technique or Item List Technique (see Gavin et al., Reference Gavin, Solomon and Blank2010; Nuno & St John, Reference Nuno and St John2015). The technique works by devising a short ‘control’ list of three to five innocuous items that are non-sensitive but relevant to the research topic, and a treatment list which also contains the sensitive item of interest (Figure 5.1). The sampled population is randomly shown either the control or treatment list. Respondents are asked to report only the total number of items that apply to them. Because only a number is reported, the researcher has no way of knowing which specific items apply to a given respondent. The difference in the mean number of items reported by the two groups provides an estimate of the proportion of respondents engaging in the sensitive behaviour (Thomas et al., Reference Thomas, Gavin and Milfont2015).

Figure 5.1 Using the Unmatched Count technique to ask about illegal bushmeat hunting in the Ugalla Wildlife Reserve, Tanzania. Picture by Paulo Wilfred.

5.2.1 Case study: Bengal florican

Ibbett et al. (Reference Ibbett, Lay and Phlai2019) used the Unmatched Count Technique to investigate prevalence of illegal behaviours and to identify the characteristics of resource users in central Cambodia. In the dry season, the seasonally inundated grasslands surrounding the Tonle Sap lake are home to the world’s largest remaining population of Bengal florican (Houbaropsis bengalensis), a critically endangered bustard species (Birdlife International, 2015). Recently, agricultural abandonment, scrub advancement and the emergence of dry-season rice – a form of intensive, irrigated rice cultivation – have dramatically reduced grassland cover. The Tonle Sap florican population is estimated to have declined by 44–66% since dry-season rice was first cultivated on the floodplain in 2004 (Packman et al., Reference Packman, Showler and Collar2014). However, conservation managers lack adequate understanding of the drivers of dry-season rice expansion. There is also evidence that hunting, a historic driver of decline, may persist in local communities (Packman, Reference Packman2011).

Ibbett et al. (2019) used a mixed-methods approach to investigate these issues. Because hunting is potentially a sensitive activity (hunting wildlife is illegal in protected areas), the Unmatched Count Technique was selected to identify the prevalence of bird hunting and florican egg collection. The Unmatched Count Technique was combined with direct questioning and delivered through a household questionnaire, which captured information on household demographics, livelihood activities and awareness of bird species. Due to the florican’s scarcity, Unmatched Count Technique questions concerned the hunting of larger grassland birds in general, with questions phrased as ‘How many of the following animals/types of egg have people in your household caught in the last 12 months?’ A warm-up question about different fruits consumed in the household was asked in order to introduce respondents to the technique.

A sample of 616 households across 21 villages was secured. The warm-up question identified a significant difference between control and treatment groups, suggesting the technique was working as expected. However, no significant difference was identified between control and treatment groups for egg collecting or large bird hunting, suggesting the prevalence of these activities did not significantly differ from zero. When questioned directly, just 8.6% of households reported hunting birds in the previous 12 months, the majority of which were small, abundant game birds, such as buttonquail and ducks. Those that reported hunting birds were more likely to come from households which also collected other wildlife products, such as frogs and crickets.

5.2.2 Lessons learnt

While indirect questioning techniques avoid some of the pitfalls of traditional techniques, they are not without limitations. In this case, the Unmatched Count Technique failed to detect the presence of bird hunting, unlike direct questioning. This may be explained by the generally low prevalence of this activity and the probabilistic nature of the approach, which means that confidence intervals are large. Part of the issue is that the direct question was about bird hunting in general, and showed low levels of hunting of common species, while the Unmatched Count Technique question investigated targeted hunting of large bird species. Only one or two incidences of hunting large bird species were directly reported. Similar experiences of inability to estimate prevalence have been reported by others when using the Unmatched Count Technique to investigate illegal activities (e.g. Nuno et al., Reference Nuno, Blumenthal and Austin2018). Therefore, the Unmatched Count Technique is unlikely to be useful when estimating the prevalence of an extremely rare activity. Indirect questioning is also not a panacea for sensitivity; if an activity is highly sensitive, particularly if it violates social norms, respondents may still not answer truthfully when the item is in a list; this can even result in negative estimates for prevalence (e.g. Fairbrass et al., Reference Fairbrass, Nuno and Bunnefeld2016).

Compared to other indirect techniques, such as the Randomised Response Technique (see Nuno & St John, Reference Nuno and St John2015), the Unmatched Count Technique is often preferred because it can provide higher estimates of prevalence, is simple to understand and adaptable, and thus useful in developing countries where levels of illiteracy may be high (Gavin et al., Reference Gavin, Solomon and Blank2010; Nuno & St John, Reference Nuno and St John2015). Despite this, the concept can still be difficult for respondents to grasp. Respondents may be wary, especially if they have previously had negative encounters with researchers. Taking time to thoroughly talk through the technique, using a warm-up question and explaining each list item is essential to avoid these issues. Often, conservation researchers rely on the help of translators or local research assistants. Selecting the very best help available and providing extensive training to assistants is essential in order to prevent information from getting ‘lost in translation’. Local research assistants can also provide knowledge to ensure designs are appropriate. This is particularly helpful when working in illiterate communities, or when relying on pictorial prompts.

5.3 Triangulating different sources of evidence to build a rounded picture

Social research methods such as focus groups, interviews and household surveys are increasingly being employed to investigate illegal behaviours and profile resource users (Young et al., Reference Young, Rose and Mumby2018). The current decade has seen an increase in the use of these mixed methods approaches to gain a more holistic understanding of resource use (e.g. Kahler & Gore, Reference Kahler and Gore2012; Harrison et al., Reference Harrison, Baker and Twinamatsiko2015). A combination of perspectives, using both qualitative and quantitative methods, is commonly preferred.

5.3.1 Case study: Ugalla Game Reserve

Ugalla Game Reserve (hereafter Ugalla; 5000 km2) in western Tanzania is predominantly miombo woodland. Its conservation value is high, serving as habitat for a wide range of species (UGR, 2006). It is part of the Malagarasi–Muyovosi Ramsar Site, and facilitates connectivity between protected areas in western Tanzania (Kalumanga, Reference Kalumanga2015; Riggio & Caro, Reference Riggio and Caro2017). The main legal activity in the reserve is trophy hunting, mostly by overseas tourists. A number of different approaches are used to conserve Ugalla, including irregular anti-poaching patrols and seasonal permission for fishing and beekeeping activities (July–December). These also aim to attract local support for conservation and build a sense of ownership of the reserve among local people. However, recent studies suggest that this conservation approach is ineffective (Wilfred & MacColl, Reference Wilfred and MacColl2015; Wilfred et al., Reference Wilfred, Milner-Gulland and Travers2017). Unauthorised use of natural resources (including poaching, illegal logging and fishing) is common and local communities hold negative attitudes towards the reserve and its management. In an attempt to shed light on the prevalence of illegal behaviours and inform the management of Ugalla, multiple research methods were used to gather relevant information. Between 2013 and 2016, household surveys and focus groups were conducted in villages around Ugalla, along with a survey of signs of illegal activity undertaken across the Protected Area.

For the household surveys, 533 households were randomly sampled in 2016 in the vicinity of Ugalla. The Unmatched Count Technique was used to estimate the prevalence of illegal behaviours (logging, illegal hunting and honey-gathering). The survey also included questions on households’ perceptions of the main threats to Ugalla and its wildlife, and what communities would do differently to improve Ugalla’s management effectiveness. Six single-sex focus groups from six randomly selected villages within 20 km of the Ugalla boundary, each with 4–6 participants, were conducted to verify findings from the household survey. Free-listed threats to Ugalla were ranked in decreasing order of their importance, and each threat was then divided by the total number of threats to calculate the salience score (Papworth et al., Reference Papworth, Milner-Gulland and Slocombe2013). The overall score for each threat was obtained by calculating the average salience score across the focus groups. The greater the salience, the more important the threat.

For signs of illegal activity in the reserve, 10 patrol tracks were randomly selected in 2014. Six transect starting points were placed at 3000-m intervals along each road. At each point, two 1500-m transects were walked on opposite sides and perpendicular to the road. Signs of illegal activity (e.g. tree stumps, sawpits, meat smoking racks, snares, trees felled for honey extraction, fish smoking racks, poacher camps) were noted 50 m either side of the transect line (Figure 5.2).

Figure 5.2 Paulo Wilfred and his research assistant recording an illegal meat smoking rack in Ugalla Wildlife Reserve.

The Unmatched Count Technique results suggested that poaching and illegal logging were performed by 28% (SE ± 6) and 20% (SE ± 5), respectively, of surveyed households; 18% (SE ± 6) of respondents gathered honey. The top four threats to Ugalla, as identified by respondents, were poaching (40% of respondents), logging (39%), fishing (11%) and honey gathering (8%). Of the top four threats to Ugalla free-listed and ranked by focus groups, logging had the highest salience (S = 0.5), followed by poaching (0.45). Within the reserve, 867 illegal activity signs were encountered. Signs related to logging had the highest frequency, followed by honey gathering, poaching and fishing (Figure 5.3). These results indicate that levels of illegal activity in Ugalla are high. The different methods consistently suggest that logging and poaching are the commonest illegal activities.

Figure 5.3 Signs of illegal activity encountered inside Ugalla Game Reserve in 2014. Total signs = 867.

Of the activities that survey respondents and focus group participants said they would undertake if they were the Ugalla manager, the most common recommendations were to: improve the well-being of people around Ugalla (17% of respondents, S = 0.11); ensure that local people have adequate land for their livelihood activities (16%, S = 0.35); promote local participation in conservation (16%, S = 0.13); improve law enforcement (15%, S = 0.14); raise conservation awareness (15%, S = 0.14); and improve local people’s relations with reserve managers (12%, S = 0.54).

5.3.2 Lessons learnt

Paulo Wilfred’s research in Ugalla started nearly a decade ago with the overarching objective of informing conservation management. This long-term research suggests that local communities are knowledgeable about illegal activities and keen to participate in conservation efforts. For example, during household surveys, villagers from unsampled households sometimes expressed their desire to share their views and experiences about natural resources. Accordingly, researchers can facilitate liaison between reserve managers and local people.

Although Paulo’s research exposes the situation on the ground, we are not yet able to connect these observations to a good understanding of the drivers of illegal behaviour or the governance context framing reserve management. To fulfil such an objective, more targeted research is required. Ideally, this should focus on individual activities, rather than trying to investigate all illegal activities at once. Different activities are conducted by different groups of people with different rationales and link to different governance issues.

The methods applied in Ugalla were resource-intensive. For example, Unmatched Count techniques typically require high sample sizes (see Nuno et al., Reference Nuno, Bunnefeld and Naiman2013); more than 500 households were surveyed in this study, which was all that time and funding allowed. Surveying for illegal activity signs was also challenging, because it was difficult to estimate the time the signs had been present in the environment and different signs have different biases (e.g. rangers remove snares during their normal anti-poaching patrols, potentially leading to underestimates).

The main lessons learnt from Ugalla were as follows.

  • Be interdisciplinary! Don’t be afraid to use ecological survey methods, for example incorporating a field-based survey into the research design. This can provide a great opportunity to cross-validate findings from social research.

  • Conservation researchers preferring mixed methods should not be overambitious. Instead, they should be realistic, choosing techniques carefully and planning activities based on the resources available, following a robust pilot study.

  • While doing household surveys and focus groups, it is critical to use experienced research assistants who are neutral in the community but familiar with the study area. A survey of illegal activity signs also requires experienced field assistants, so information is collected accurately and consistently.

  • Both focus group discussions and household surveys should be kept relatively short and simple to minimise participant fatigue.

5.4 Developing a Theory of Change for an intervention

It is vitally important to be clear about why we think that our intervention is the right thing to do, and what barriers there might be to success, before we start. This understanding needs to be set out in a logical way, so that it is understandable and appealing to project staff and donors, and so that it can later be tested. There are a number of approaches which can be used, falling under a general heading of causal chain models (Qiu et al., Reference Qiu, Game and Tallis2018). One such approach is Theory of Change (Center for Theory of Change, 2018), which shows how a project can reach its desired impact and goals through different pathways of change. It provides indicators that can be tested, thereby supporting evaluation of a project’s success or failure. This is useful both for internal and external users, to understand what works, and to guide the allocation of project resources.

5.4.1 Case study: Vijana na Mazingira

In 2016, Hans Cosmas Ngoteya designed a retrospective Theory of Change for Vijana na Mazingira (VIMA), the local conservation project which he runs in the Katavi–Rukwa ecosystem of western Tanzania. The project targets youths aged 12–35, with the goal of reducing pressure on natural resources from poaching, deforestation and encroachment. The Theory of Change was designed to support an evaluation of the effects of the project on attitudes, awareness and conservation behaviours by youths aged 18–35 participating in VIMA’s conservation education and alternative livelihood projects (Figure 5.4).

Figure 5.4 Hans Cosmas Ngoteya (second from right) setting up a beehive with local youths, as an alternative livelihood project.

In order to achieve a project’s desired impact, it is necessary first to understand the motivations for engaging in the behaviour that the project is aiming to modify. There are a number of frameworks available from social psychology that represent the factors that interact to influence behaviours. One of the most widely used in conservation is the Theory of Planned Behaviour (St John et al., Reference St John, Keane and Milner-Gulland2013). Hans used the Theory of Planned Behaviour to identify the different factors underlying the motivations of the VIMA project’s recipients (Figure 5.5). Based on Hans’ local knowledge and understanding of the project, a Theory of Planned Behaviour framework was developed for four desired project impacts, each of which represented a desired behavioural change. The Theory of Planned Behaviour was then used to identify how VIMA’s activities might tackle the different motivations underlying each behaviour.

Figure 5.5 A Theory of Planned Behaviour diagram illustrating the factors underlying the poaching behaviours of individuals targeted by the VIMA project.

A clear understanding of the motivations behind the behaviour, engendered by the Theory of Planned Behaviour exercise, can enable conservationists to map out the pathways of change the project should focus on, thereby generating a Theory of Change. The Theory of Planned Behaviour gives a representation of what underlies an individual’s behaviour, and this can be used to develop a Theory of Change for the planned intervention. In this case, the Theory of Planned Behaviour exercise highlighted that, typically, youths in Katavi–Rukwa viewed poaching as a way to feed their families and generate an income through bushmeat or ivory sales. Therefore, an intervention that developed alternative livelihood programmes could be an effective approach. This could include training youths in new income-generating activities (input), thereby providing alternative income sources (output), which will reduce their dependence on natural resources (outcome) and ultimately reduce their poaching behaviour (impact; Figure 5.6). At each step of this pathway lie assumptions; for example, that any alternative income source will replace, rather than supplement, income from hunting (Table 5.1).

Figure 5.6 Theory of Change for VIMA project showing interventions at the bottom and different pathways to reach the desired impacts. Numbers 1–10 are assumptions along the pathways of change (listed in Table 5.1).

Table 5.1 Assumptions underlying the Theory of Change

1.Participants understand the education they are given
2.If someone is educated about environmental issues it will improve their attitude towards conservation
3.Knowledge about conservation issues leads to a decrease in acceptance of environmentally harmful behaviours
4.There is dissemination of information from VIMA participants to the remainder of the community
5.If someone’s attitude towards conservation improves, they will reduce their unsustainable resource-use behaviour
6.If communities are against unsustainable resource use, illegal resource exploitation will decrease
7.VIMA’s alternative livelihood programmes can be put into practice and generate income
8.There is opportunity for the rest of the community to become involved in the alternative livelihood projects
9.Alternative livelihoods will be used to reduce unsustainable use of natural resources
10.Decreasing dependency on natural resources will reduce poaching, encroachment and deforestation

Baseline surveys, focused on the elements of the Theory of Planned Behaviour (attitudes, knowledge, social norms), provide a set of indicators against which change engendered by the intervention can be measured. Progress through the Theory of Change can also be monitored, using a set of more process-based indicators. For example, an input indicator might be the percentage of VIMA’s target audience engaged in the alternative income activities, an output indicator might be the income generated from the alternative livelihood, the outcome might be measured as improvements in household livelihood security and the impact might be measured using an indirect questioning technique such as the Unmatched Count Technique to quantify change in poaching prevalence.

5.4.2 Lessons learnt

The requirement for robust evaluations of the effectiveness of conservation interventions is becoming more and more apparent (Sutherland et al., Reference Sutherland, Bardsley and Bennun2011). Practitioners are required to ensure that their activities are based upon the best available evidence and designed for accountability and learning. However, many small NGOs (such as Hans’ organisation, Landscape and Conservation Mentors’ Organisation) may not feel that they have the capacity to design and implement evaluations that are both user-friendly and robust enough to be useful for adaptive management. Lacking a rigorous framework for articulating goals and assumptions, it is easy to drift through interventions without having either a strategic plan or a means of measuring success. This can lead to ineffective interventions and failure to capture the changes engendered in order to learn and adapt and demonstrate impact to funders. The development of a Theory of Change for the VIMA project enabled Hans to identify his assumptions, and develop methods to collect information which can be used to monitor future impact and test assumptions against a relevant baseline.

5.5 Exploring alternatives to illegal behaviour

One of the key lessons learnt in the VIMA project was the importance of having a clear understanding of the motivations behind behaviour. Unfortunately, not all conservation projects that involve communities take this approach when designing an intervention. For example, alternative livelihood projects have long been used as strategy for reducing local threats to species, habitats or resources of conservation concern. Alternative livelihood projects are designed to reduce the prevalence of behaviours that are considered environmentally damaging and unsustainable (Wright et al., Reference Wright, Hill and Roe2016). However, a systematic review of alternative livelihood projects conducted by Roe et al. (Reference Roe, Booker and Day2015) found insufficient evidence to understand when, where or why alternative livelihood projects work. Even though there is uncertainty regarding the effectiveness of alternative livelihood projects, they continue to be a key strategy in both terrestrial and marine conservation. However, the assumptions on which they are based are often unrealistic; for example, that the alternative livelihood projects will substitute for the undesirable behaviour, that the resource users are a homogeneous group, and that targeting interventions at individuals will scale up to population-level change in pressure on resources (Wright et al., Reference Wright, Hill and Roe2016).

In marine conservation, a common response to perceived over-fishing is to provide alternative employment for existing fishers. This requires that the assumption of substitutability holds, so that fishers will willingly and happily settle into a new way of making a living (Pollnac et al., Reference Pollnac, Pomeroy and Harkes2001; Pollnac & Poggie, Reference Pollnac and Poggie2008). Pollnac et al. (Reference Pollnac, Pomeroy and Harkes2001) added that this is based on the assumption that fishing is a hard and undesirable occupation and hence an employment of last resort, that fishers are among the poorest of the poor and that the poor care little about the type of job they have as long as they make enough to live.

5.5.1 Case study: shark fishers in Tanjung Luar

Fishing pressure is generally considered to be the main cause of the decline of shark populations globally (Stevens et al., Reference Stevens, Bonfil and Dulvy2000; Robbins et al., Reference Robbins, Hisano and Connolly2006; Dharmadi et al., Reference Dharmadi and Satria2015). Indonesia is the world’s largest shark producer, with annual average production of 106,000 tons in 2000–2011, contributing 13% of global shark production (Dent & Clarke, Reference Dent and Clarke2015). Although the exact number is unknown, it is assumed that many Indonesian fishers are heavily dependent on shark fisheries as a source of income and food. However, shark production in Indonesia has been declining in recent years (Sub Directorate of Capture Fisheries Data and Statistics, 2016), which could be leading to a decline in income and livelihood security for fishers.

From 2014 to the present, the Wildlife Conservation Society (WCS) Indonesia Programme has carried out a study of shark fishers in Tanjung Luar, a shark-fishing community in East Lombok, in order to understand whether providing alternative livelihoods could help to reduce fishing pressure on sharks. Tanjung Luar is one of the main shark landing sites in Indonesia. It is home to a targeted shark fishery comprising approximately 50 boats employing surface and bottom longlines and one of the biggest fish markets on Lombok Island, with more than 5000 fishers using it to sell their catch. Fishing is the main livelihood of Tanjung Luar’s population and there are at least 150 households heavily dependent on the shark industry, either as fishers, meat processors or traders. Shark fishers in Tanjung Luar use 4–25 gross tonnage boats, with three or four crew members, and the average fishing trip is 14 days.

Due to growing international concern regarding their conservation status, several shark and ray species have been listed on CITES Appendix II. As a CITES member, Indonesia is required to implement management measures, such as quotas, size limits and export bans to ensure that international trade in these species is not detrimental to wild populations. These measures could have negative impacts on the income and livelihood security of Tanjung Luar’s fishers, who are already vulnerable to market fluctuations, particularly in export markets (Jaiteh et al., Reference Jaiteh, Loneragan and Warren2017). WCS Indonesia Programme’s study aimed to: (1) collect data on biological and operational characteristics of the fishery (Figure 5.7), (2) understand shark fishers’ current socioeconomic status and aspirations, (3) explore alternative livelihood options and (4) create dialogue between fishers and the management authorities.

Figure 5.7 WCS Indonesia team members measuring guitarfish at Tanjung Luar port.

Photo provided by WCS-Indonesia.

Livelihoods options explored with the fishers included diversifying the target catch to more resilient species (e.g. squid, tuna and reef fish) and tourism, yet WCS Indonesia Programme’s surveys showed that shark fishing offered higher revenues than other fisheries. An independent fisheries assessment by Masyarakat dan Perikanan Indonesia also showed similar results (MDPI, 2017). Tanjung Luar was known for its squid fishery in the 1980s, but the number of squid fishermen has increased rapidly, increasing competition and making the addition of new fishers unsustainable (MDPI, 2017). Some fishers in Tanjung Luar who catch tuna or skipjack mentioned that their catch is also declining, and The Indian Ocean Tuna Commission classifies yellowfin tuna as overfished (IOTC, 2017). Some shark fishers have already started to fish for groupers and snappers on the side, but the value of this catch is far less than their earnings from sharks. Tourism is promising, but the industry is still under-developed. To date, identifying feasible alternatives that provide economic incentives to shift away from shark fishing has proven challenging, as there are no legal or sustainable marine alternatives that offer similar profits.

Our research showed that shark fishers wish to remain shark fishers. Fishing is the only skill they know, and most of them said that they would continue to fish as usual even if their catch declined by 50%. Our landings survey showed that some commonly caught sharks are over-exploited. When findings were shared with fishers, although not all agreed with the results, shark fishers acknowledged that it is now harder to catch sharks and the sharks that are caught are smaller, a view also shared by shark fishers in eastern Indonesia (Jaiteh et al., Reference Jaiteh, Loneragan and Warren2017). The Tanjung Luar fishers’ response is not surprising, as similar reactions were also reported by Pollnac and Poggie (Reference Pollnac, Pomeroy and Harkes2006), with fishermen refusing to leave their existing fishery even though their incomes were declining; it is potentially their best option in the short run if they are still making a profit.

5.5.2 Lessons learnt

Based on the results of this research, instead of deploying alternative livelihood projects for shark fishers in Tanjung Luar, WCS’s Indonesia Programme chose to:

  1. (1) strengthen the existing fisher institutions, which focus on tourism development, in order to help that industry to develop, become more attractive and profitable;

  2. (2) maintain close interaction with shark fishers by regular home visits and conducting informal meetings; and

  3. (3) facilitate formal meetings between shark fishers and the management authorities, to foster dialogue on developing management measures that ensure the sustainability of both shark and ray populations and fishers’ livelihoods.

It is challenging to establish a direct connection between livelihood interventions and conservation. Rather than trying to find new livelihoods, sometimes it is more appropriate to focus on enhancing existing livelihood strategies which do not involve exploiting the natural resource of concern, targeting those most vulnerable to conservation-imposed resource access restrictions (Wright et al., Reference Wright, Hill and Roe2016). It may also be possible to establish a clearer link between livelihood sustainability and conservation as a means of building good community relations, as we opted to do. It is important to have a clear pathway demonstrating how an intervention is expected to lead to the desired outcome, e.g. by using theory of change to design the intervention after gaining a thorough understanding of community dynamics.

5.6 Discussion: interlacing research and practice

The four case studies presented here take us from research to practice; in so doing, they illustrate how integrated the two are. By starting with a strong theoretical framework (such as the Theory of Planned Behaviour) underpinning an intervention’s Theory of Change, unwarranted assumptions can be avoided, such as those which plague alternative livelihoods projects. Engaging with resource users before embarking on interventions can reveal dead ends, as illustrated in Tanjung Luar, where plans for an alternative livelihood project needed to be replaced by a more indirect process of advocacy and engagement with different parties, while building capacity for a livelihoods shift. A clear understanding of what the actual problem is, based on evidence rather than supposition, is vital; the example from Cambodia suggested that hunting was actually not a major threat to floricans, enabling conservation practitioners to focus on other threats.

Although a range of techniques is available for collecting information to underpin management, these should not be applied lightly. As the Ugalla example showed, the ideal of using mixed methods to gain a nuanced understanding takes time and resources, as well as expertise. Approaches such as the Unmatched Count Technique can look superficially appealing and easy to administer, but there are technical challenges in developing appropriate item lists, administering the questions in a way that makes respondents comfortable, and in data analysis. Even then, as the Cambodian example shows, the results may not be as informative as might be hoped. Time invested in foundational studies is well spent, but not all small NGOs can afford extensive research. Even then, however, it is possible to develop a robust Theory of Change, as a tool for exposing assumptions and supporting ongoing monitoring and evaluation, as the VIMA example showed.

Our case studies have specific lessons, but they also tell universal stories. The role of research in facilitating positive interactions between managers and local people is an interesting observation that was seen in both Ugalla and Tanjung Luar, while both the Cambodian and Ugalla case studies highlighted the importance of good local research assistants. All four case studies emphasised how research and practice need to intertwine more often and more routinely. This will enable conservationists (whether from governments or NGOs) to think through their interventions in advance, use appropriate methods to understand existing behaviour and local perspectives on ways forward, and thereby design locally appropriate, participatory interventions that support adaptive management.

References
Birdlife International. 2015. Houbaropsis bengalensis. The IUCN Red List of Threatened Species 2015: e.T22692015A82252874. http://dx.doi.org/10.2305/IUCN.UK.2015–4.RLTS.T22692015A82252874.en (accessed 19 April 2016).
Brockington, D. & Scholfield, K. 2010. Expenditure by conservation nongovernmental organizations in sub‐Saharan Africa. Conservation Letters, 3, 106113.
Center for Theory of Change. 2018. Theory of Change. Available from www.theoryofhcange.org.
Dent, F. & Clarke, S. 2015. State of the global market for shark products. FAO Fisheries and Aquaculture Technical Paper. Rome: FAO.
Dharmadi, Fahmi & Satria, F. 2015. Fisheries management and conservation of sharks in Indonesia. African Journal of Marine Science, 37, 249258.
Fairbrass, A., Nuno, A., Bunnefeld, N., et al. 2016. Investigating determinants of compliance with wildlife protection laws: bird persecution in Portugal. European Journal of Wildlife Research, 62, 93101.
Gandiwa, E. 2011. Preliminary assessment of illegal hunting by communities adjacent to the northern Gonarezhou National Park, Zimbabwe. Tropical Conservation Science, 4, 445467.
Gavin, M. C., Solomon, J. N. & Blank, S. G. 2010. Measuring and monitoring illegal use of natural resources. Conservation Biology, 24, 89100.
Gore, M. L. 2011. The science of conservation crime. Conservation Biology, 25, 659661.
Harrison, M., Baker, J., Twinamatsiko, M., et al. 2015. Profiling unauthorized natural resource users for better targeting of conservation interventions. Conservation Biology, 29, 16361646.
Ibbett, H., Lay, C., Phlai, P., et al. 2019. Conserving a globally threatened species in a semi-natural, agrarian landscape. Oryx, 53, 181191.
Imai, K. 2011. Multivariate regression analysis for the item count technique. Journal of the American Statistical Association, 106, 407416.
IOTC. 2017. 2017 Status summary for species of tuna and tuna like species under the IOTC mandate. Available from www.iotc.org/sites/default/files/Summary_of_Stock_Status.pdf
Jaiteh, V. F., Loneragan, N. R. & Warren, C. 2017. The end of shark finning? Impacts of declining catches and fin demand on coastal community livelihoods. Marine Policy, 82, 224233.
Kahler, J. S. & Gore, M. L. 2012. Beyond the cooking pot and pocket book: factors influencing noncompliance with wildlife poaching rules. International Journal of Comparative and Applied Criminal Justice, 36, 103120.
Kalumanga, E. 2015. How elephants utilize a miombo-wetland ecosystem in Ugalla landscape, Western Tanzania. Doctoral dissertation, Department of Physical Geography, Stockholm University.
Kiffner, C., Peters, L., Stroming, A., et al. 2015. Bushmeat consumption in the Tarangire–Manyara ecosystem, Tanzania. Tropical Conservation Science, 8, 318332.
Larrosa, C., Carrasco, R. L. & Milner-Gulland, E. J. 2016. Unintended feedbacks: challenges and opportunities for improving conservation effectiveness. Conservation Letters, 9, 316326.
Martin, A., Caro, T. & Mulder, M. B. 2012. Bushmeat consumption in western Tanzania: a comparative analysis from the same ecosystem. Tropical Conservation Science, 5, 352364.
MDPI. 2017. Tanjung Luar Fishery Assessment – internal report for Wildlife Conservation Society Indonesia Program. Denpasar, Indonesia.
Mgawe, P., Mulder, M. B., Caro, T., et al. 2012. Factors affecting bushmeat consumption in the Katavi–Rukwa ecosystem of Tanzania. Tropical Conservation Science, 5, 446462.
Milner-Gulland, E. J. 2012. Interactions between human behaviour and ecological systems. Philosophical Transactions of the Royal Society Biological Sciences, 367, 270278.
Nuno, A. N. A., Bunnefeld, N., Naiman, L. C., et al. 2013. A novel approach to assessing the prevalence and drivers of illegal bushmeat hunting in the Serengeti. Conservation Biology, 27, 13551365.
Nuno, A. & St John, F. A. V. 2015. How to ask sensitive questions in conservation: a review of specialized questioning techniques. Biological Conservation, 189, 515.
Nuno, A., Blumenthal, J. M., Austin, T. J., et al. 2018. Understanding implications of consumer behavior for wildlife farming and sustainable wildlife trade. Conservation Biology, 32, 390400.
Packman, C. E. 2011. Seasonal landscape use and conservation of a critically endangered bustard: Bengal florican in Cambodia. Doctorate of Philosphy thesis, University of East Anglia.
Packman, C. E., Showler, D. A., Collar, N. J., et al. 2014. Rapid decline of the largest remaining population of Bengal Florican Houbaropsis bengalensis and recommendations for its conservation. Bird Conservation International, 24, 429437.
Papworth, S., Milner-Gulland, E. J. & Slocombe, K. 2013. The natural place to begin: the ethnoprimatology of the Waorani. American Journal of Primatology, 75, 11171128.
Pollnac, R. B., Pomeroy, R. S. & Harkes, I. H. T. 2001. Fishery policy and job satisfaction in three southeast asian fisheries. Ocean and Coastal Management, 44, 531544.
Pollnac, R. B. & Poggie, J. J. 2006. Job satisfaction in the fishery in two Southeast Alaskan towns. Human Organization, 65, 329339.
Pollnac, R. B. & Poggie, J. J. 2008. Happiness, well-being and psychocultural adaptation to the stresses associated with marine fishing. Human Ecology Review, 15, 194200.
Qiu, J., Game, E. T., Tallis, H., et al. 2018. Evidence-based causal chains for linking health, development, and conservation actions. BioScience, 68, 182192.
Riggio, J. & Caro, T. 2017. Structural connectivity at a national scale: wildlife corridors in Tanzania. PLoS ONE, 12, 116.
Robbins, W. D., Hisano, M., Connolly, S. R., et al. 2006. Ongoing collapse of coral-reef shark populations. Current Biology, 16, 23142319.
Roe, D., Booker, F., Day, M., et al. 2015. Are alternative livelihood projects effective at reducing local threats to specified elements of biodiversity and/or improving or maintaining the conservation status of those elements? Environmental Evidence, 4, 18.
St John, F. A. V., Edwards-Jones, G., Gibbons, J. M., et al. 2010. Testing novel methods for assessing rule breaking in conservation. Biological Conservation, 143, 10251030.
St John, F. A. V., Keane, A. M. & Milner-Gulland, E. J. 2013. Effective conservation depends upon understanding human behaviour. Key Topics in Conservation Biology, 2, 344361.
St John, F.A. V., Mai, C. & Pei, K. J. C. 2015. Evaluating deterrents of illegal behaviour in conservation: carnivore killing in rural Taiwan. Biological Conservation, 189, 8694. Elsevier Ltd.
Stevens, J. D., Bonfil, R., Dulvy, N. K., et al. 2000. The effects of fishing on sharks, rays, and chimaeras (chondrichthyans), and the implications for marine ecosystems. ICES Journal of Marine Science, 57, 476494.
Sub Directorate of Capture Fisheries Data and Statistics. 2016. Capture Fisheries Statistics of Indonesia by Province, 2015. Jakarta: Directorate General of Capture Fisheries.
Sutherland, W. J., Bardsley, S., Bennun, L., et al. 2011. Horizon scan of global conservation issues for 2011. Trends in Ecology and Evolution, 26, 1016.
Thomas, A. S., Gavin, M. C. & Milfont, T. L. 2015. Estimating non-compliance among recreational fishers: insights into factors affecting the usefulness of the randomized response and item count techniques. Biological Conservation, 189, 2432.
UGR. 2006. A checklist of plants, animals and birds in Ugalla Game Reserve. Unpublished report, Ugalla Game Reserve Project, Tabora, Tanzania.
Wilfred, P. & MacColl, A. D. C. 2015. Local perspectives on factors influencing the extent of wildlife poaching for bushmeat in a game reserve, Western Tanzania. International Journal of Conservation Science, 6, 99110.
Wilfred, P., Milner-Gulland, E. J. & Travers, H. 2017. Attitudes to illegal behaviour and conservation in western Tanzania. Oryx, 110.
Woodhouse, E., de Lange, E. & Milner-Gulland, E. J. 2016. Evaluating the Impacts of Conservation Interventions on Human Well-being. Guidance for Practitioners. London: IIED.
Wright, J., Hill, Ni., Roe, D., et al. 2016. Reframing the concept of ‘alternative livelihoods’. Conservation Biology, 30, 713.
Young, J. C., Rose, D. C., Mumby, H. S., et al. 2018. A methodological guide to using and reporting on interviews in conservation science research. Methods in Ecology and Evolution, 9, 1019.

Chapter Six Mobilisation of indigenous and local knowledge as a source of useable evidence for conservation partnerships

Pernilla Malmer , Masterson Vanessa
Swedbio, Stockholm Resilience Centre
Beau Austin
Charles Darwin University
and Maria Tengö
Swedbio, Stockholm Resilience Centre
6.1 Introduction

Rapid and interlinked changes in the biosphere, including degradation of the biodiversity and ecosystems that underpin human well-being, are reported with increasing regularity. As such, there is an urgent need for conservation initiatives that are capable of countering the speed and veracity of change, while meeting the needs of human societies on a crowded planet. While significant advancements in scientific knowledge in the fields of sustainability and conservation continue to be achieved, the forecasted rate of rapid ecological and social change requires the production of innovative mechanisms for management and policy.

One way of contributing to new solutions in a timely manner is to more effectively mobilise multiple knowledges, values and governance systems that can complement Western approaches to science. Together these can extend the collective knowledge base and contribute to collaboratively designing ways forward for looking after people and the biosphere. Compared with Western-based approaches, indigenous and local knowledge systems represent alternative ways of learning from and with the environment, through close and continuous observation framed by distinct worldviews with particular strengths and limitations (like all knowledge systems). Knowledge is embodied by the actors and in their practices, tools, and technologies, as well as in the institutions that organise the production, transfer and use of knowledge (Cornell et al., Reference Cornell, Berkhout and Tuinstra2013). There has recently been more attention focused on the urgent need for science and policy to recognise and mobilise the knowledge of indigenous people and local communities who steward substantial biodiversity across the globe (Brondizio & Le Tourneau, Reference Brondizio and Le Tourneau2016; Mistry & Berardi, Reference Mistry and Berardi2016). Collaborative ways for mobilising knowledge and learning across diverse knowledge systems can contribute complementary knowledge, innovations and new solutions. Involvement of multiple actors and knowledges can strengthen usefulness and legitimacy in decision-making and implementation (Sterling et al., Reference Sterling, Betley and Sigouin2017a; Gavin et al., Reference Gavin, McCarter and Berkes2018).

In this chapter, we draw attention to the potential for mobilising local and indigenous knowledge systems, institutions and actors in ways that allow meaningful use of their knowledge about landscapes and their functions as evidence for conservation. By doing this, we propose that innovative and collaborative mechanisms can be designed and implemented that will create opportunities for long-term sustainable governance and conservation of biodiversity.

We introduce the Multiple Evidence Base (MEB) approach to guide the design and implementation of conservation partnerships that enable engagements with indigenous and local knowledge as evidence as an entry point to promote sustainable governance of interrelated ecosystems and human well-being (Tengö et al., Reference Tengö, Brondizio and Elmqvist2014, Reference Tengö, Hill and Malmer2017). The approach was developed to guide inclusive processes for collaborations across knowledge systems, based on equity and usefulness for all actors involved. It emphasises that indigenous, local and scientific knowledge systems are complementary, equally valid and useful for informing sustainable governance of biodiversity and ecosystems. The MEB focuses on the theoretical and practical potential for collaborative knowledge-weaving processes to mobilise indigenous and local actors, institutions and practices to achieve long-term conservation and sustainability targets. We argue that collaborative approaches to conservation must be equitable and fair to be effective in the long term (Brondizio & Le Tourneau, Reference Brondizio and Le Tourneau2016; Sterling et al., Reference Sterling, Betley and Sigouin2017a; Gavin et al., Reference Gavin, McCarter and Berkes2018).

The utility and value of the MEB approach will be discussed in light of its aim to support more informed and efficient local, national and international policy processes and governance decisions for the integrated benefits of conservation, sustainable use and human well-being. We describe the current and potential role that a MEB approach may have in enhancing the efficacy of conservation science and policy by clarifying and strengthening synergies with indigenous knowledges and practices. To achieve this, we first review the peer-reviewed and grey literature to reflect on the extent of uptake of the MEB and how it has been applied in both science and policy-practice processes. Second, to illustrate the approach and reflect on successes and practical challenges, we take a deeper look at three case studies of piloting a MEB approach. The cases demonstrate the potential for the MEB approach to be used as both a framing tool for collaborative partnerships and a practical guide to weaving multiple knowledge systems. Lastly, we discuss ways forward to nurture conservation and mobilise partnerships that build on knowledge collaborations. We find that a MEB approach has potential to support the inclusion of a wider range of evidence in conservation practice, strengthen active participation of local actors and improve conservation partnerships through the recognition and revitalisation of local knowledge systems and governance.

6.2 The need for new approaches to collaborative conservation

There is a long history of attempts to reconcile conservation objectives with local livelihoods in integrated development and conservation processes, which have often been framed as ‘win–win’ opportunities with social–ecological benefits (Adams et al., Reference Adams, Aveling and Brockington2004). In the conservation literature, the importance of involving local people is well established, with mounting evidence that processes that meaningfully engage local people are more likely to succeed in protecting biodiversity (Waylen et al., Reference Waylen, Fischer and McGowan2010; Sterling et al., Reference Sterling, Betley and Sigouin2017a) and that failure to do so can lead to lack of trust and commitment, project failure, and in the worst case, lingering conflicts (Oldekop et al., Reference Oldekop, Holmes and Harris2015). While many indigenous peoples and local communities continue to be evicted from their ancestral lands and experience colonisation in the name of conservation, there is now a move towards recognising their connections to land and endogenous obligations to care for it as synergetic with biodiversity conservation outcomes (Knox, Reference Knox2017). This provides a foundation for enabling local people and conservation organisations to be strategic allies. Furthermore, there is increasing evidence that involving local actors in monitoring enhances management responses at local spatial scales, and increases the speed of decision-making to tackle environmental challenges at operational levels of resource management (Danielsen et al., Reference Danielsen, Burgess and Jensen2010; Sterling et al., Reference Sterling, Betley and Sigouin2017a).

Despite these generally acknowledged realities about the usefulness of engaging with indigenous peoples and local communities, they are often included as stakeholders in conservation, without recognition of their knowledge and expertise (Danielsen et al., Reference Danielsen, Burgess and Jensen2010). In the literature much attention is given to the uniqueness and utility of indigenous and local knowledge systems, which is often holistic, providing an understanding of integrated social–ecological systems, biocultural values and belief systems (Sheil et al., Reference Sheil, Boissière and Beaudoin2015; Sterling et al., Reference Sterling, Betley and Sigouin2017a). However, in practice, there often exists scepticism about the contemporary existence and/or effectiveness of indigenous and local knowledge as useful evidence in conservation. Similarly, holders of indigenous and local knowledge can be sceptical of the claims generated through western scientific approaches due both to the unfamiliarity of the epistemic practices employed and recent or ongoing experiences of colonisation and disempowerment (Nadasdy, Reference Nadasdy1999; Johnson et al., Reference Johnson, Alessa and Behe2015; Kealiikanakaoleohaililani & Giardina, Reference Kealiikanakaoleohaililani and Giardina2016; Mistry & Berardi, Reference Mistry and Berardi2016).

6.3 The multiple evidence base approach: connecting knowledge systems for the benefit of conservation and human well-being

The need to engage with diverse sources of knowledge for conservation has been recognised in high-level science–policy processes, such as the Convention on Biological Diversity, and the Intergovernmental Science Policy Platform on Biodiversity and Ecosystem Services (IPBES). From the outset, IPBES had the ambition to recognise and respect the contribution of indigenous and local knowledge to the conservation and sustainable use of biodiversity and ecosystems (Díaz et al., Reference Díaz, Demissew and Carabias2015). This was used as a window of opportunity to start an open dialogue to explore current divides between indigenous, local and scientific knowledge systems, and to elicit methods for collaborations based on equity, reciprocity and usefulness for all involved (see Tengö et al., Reference Tengö, Brondizio and Elmqvist2014). A science–policy–practice dialogue process brought together knowledge-holders and experts from diverse knowledge systems, convened by SwedBio at Stockholm Resilience Centre in collaboration with key partners representing indigenous peoples and local communities, such as the International Indigenous Forum on Biodiversity and the African Biodiversity Network. The active engagement from these networks, representing a diversity of knowledge systems and linking practices on the ground with global policy and science, created legitimacy and recognition of outcomes from the dialogues. The starting point was the pivotal dialogue meeting prior to the establishment of IPBES in the indigenous territory of Guna Yala, Panama, where essential principles for exchange across knowledge systems were identified: trust, respect, reciprocity, equity, transparency and free prior and informed consent (Tengö & Malmer, Reference Tengö and Malmer2012). Since then, the MEB approach has developed in parallel to the IPBES, while carefully paying attention to other interests and needs of the partners.

The MEB can be understood as a deep approach to collaborative knowledge-sharing that explicitly acknowledges that challenges are fundamentally due to different perspectives and practices concerning human–nature relationships, approaches to knowledge validation, knowledge governance and who qualifies as an ‘expert’. Also, it recognises that scientists have tended to dominate the design and implementation of collaborations across knowledge systems both historically and contemporarily (Nadasdy, Reference Nadasdy1999; Mistry & Berardi, Reference Mistry and Berardi2016). Another key component of an MEB approach is its emphasis on the need for mobilisation and validation of knowledge within knowledge systems themselves. That is, if scientific methods that often are specific and partial are applied to local knowledge that is practical, multidimensional and holistic, there is a risk of omission, misinterpretation and rejection of critical and useable knowledge.

The MEB approach views different knowledge systems as complementary and emphasises that joint analysis assists in working both with convergence and divergence (e.g. Molnár et al., Reference Molnár, Kis and Vadász2016a; Hohenthal et al., Reference Hohenthal, Räsänen and Minoia2018). For example, Molnár et al. (Reference Molnár, Kis and Vadász2016a) highlight that when discussing approaches to conservation in the Hungarian steppe, local herders focus on primarily utilitarian purposes, such as how they can manage the behaviour of their grazing animals in order to promote the health and diversity of grass assemblages for production. In comparison, conservationists working in the same landscapes focus almost solely on the protection of the plants themselves, with little regard to the impact on grazing animals. If this difference is ignored, or framed as a problem, it has the potential to create tension when attempting to collaboratively design and implement conservation initiatives in the region. Conversely, these different perspectives can be worked together to provide an enriched picture of exactly what is necessary for maintaining and enhancing biodiversity and social–ecological system function in the steppe.

In order to build evidence – whether new knowledge or existing – that is legitimate and useful for all actors in such collaborations for conservation, there is a need to engage with local knowledge systems and knowledge-holders from the outset, co-defining a common problem and facilitating equitable engagement through all activities, including mobilising and assessing knowledge. This process is outlined in the three phases of the MEB approach (Figure 6.1a). Collaboratively analysing and interpreting the complementary evidence from diverse sources is a way to triangulate information, strengthen legitimacy and relevance of existing knowledge and build a base for further learning.

Figure 6.1 The Multiple Evidence Base approach in action. (a) The three phases of a MEB approach: joint problem formulation, generating an enriched picture with contribution from multiple sources of evidence and joint analysis and evaluation of knowledge (Tengö et al., Reference Tengö, Brondizio and Elmqvist2014). (b) Actors, institutions and processes are at the core of the five tasks required for successful collaboration across diverse knowledge systems. The different colours of the lines and dots in parts (a) and (b) represent different knowledge systems, or streams of knowledge within knowledge systems (Tengö et al., Reference Tengö, Hill and Malmer2017).

As guidance for how to implement an MEB approach, five tasks were identified as critical (Figure 6.1b; Tengö et al., Reference Tengö, Hill and Malmer2017). First, to mobilise knowledge – to ensure that the knowledge is articulated, validated internally and free to be shared with others. Second, to translate knowledge, reciprocally, so that all actors can comprehend each others’ knowledge and where it is derived from. Third, to negotiate, to jointly address convergence and divergence between knowledge systems, and the extent to which the latter can be resolved, for example by understanding differences in underlying assumptions and values (Gagnon & Berteaux, Reference Gagnon and Berteaux2009; Molnár et al., Reference Molnár, Kis and Vadász2016a). Fourth, to synthesise. Here we emphasise synthesis based on a joint process that does not require that all knowledge is validated by one knowledge system (e.g. empirical validation by science). Lastly, to apply – and this is where we iterate the need to recognise the different needs and interests by different actors. Knowledge collaborations need to be designed in a way that is is perceived as useful and leads to constructive outcomes for all involved. The bridging of knowledge systems therefore requires the creation of settings for exchange of multiple forms of knowledge and learning across key aspects of the system (Figure 6.1b). We view the outcome as weaving – knowledge collaborations that respect the integrity of each knowledge system while working them together for practical collaboration (Johnson et al., Reference Johnson, Howitt and Cajete2016; Tengö et al., Reference Tengö, Hill and Malmer2017). In the next section, we use literature and our own experience to evaluate and discuss implementation of the MEB approach, with a specific focus on describing the outcomes in terms of evidence applied in conservation partnerships.

6.4 Reviewing the impact of the MEB in conservation and sustainability

The literature on knowledge collaborations for conservation and sustainability is wide-ranging. To focus on collaborations across knowledge systems (indigenous, local and scientific knowledge systems) and to generate further insights into the application, challenges and usefulness of a MEB approach, we reviewed articles that cite Tengö et al. (Reference Tengö, Brondizio and Elmqvist2014) or that mention ‘Multiple Evidence Base’ in the academic literature, represented by Scopus (123 articles), and the grey literature (219 results), represented by Google Scholar (as of 2018–02-01).

The results of this review demonstrate that the MEB approach has contributed to a general move towards broader participation of knowledge-holders in multi-level ecosystem assessments (Díaz et al., Reference Díaz, Demissew and Carabias2015; Nesshöver et al., Reference Nesshöver, Vandewalle and Wittmer2016), as well as citizen science, the importance of the plurality of knowledge systems in conservation (Prado & Murrieta, Reference Prado and Murrieta2015) and knowledge application in public policy and resource management (Bruckmeier, Reference Bruckmeier2016). This is part of a ‘shift that has occurred in the science–policy–society interface with a move towards greater inclusivity, and efforts to transcend traditional reductionist approaches’ (Jabbour & Flachsland, Reference Jabbour and Flachsland2017, p. 196).

The MEB approach is finding traction in diverse discussions including citizen science (Buytaert et al., Reference Buytaert, Zulkafli and Grainger2014) and community-based monitoring (Johnson et al., Reference Johnson, Alessa and Behe2015; Lyver et al., Reference Lyver, Timoti and Jones2017), collaborative management and decision-making (Mathevet et al., Reference Mathevet, Thompson and Folke2016), community-based conservation (Nkambule et al., Reference Nkambule, Buthelezi and Munien2016; Sterling et al., Reference Sterling, Betley and Sigouin2017a), measuring resilience (Quinlan et al., Reference Quinlan, Berbes-Blazquez and Haider2016; Sterling et al., Reference Sterling, Ticktin and Kipa Kepa Morgan2017b), approaches to modelling global change processes (Verburg et al., Reference Verburg, Dearing and Dyke2016), indigenous autonomy and cultural revitalisation (Gonzales, Reference Gonzales2015), value pluralism in ecological economics (Martín-López & Montes, Reference Martín-López and Montes2015; Kenter, Reference Kenter2016; Pascual et al., Reference Pascual, Balvanera and Diaz2017), biocultural values and diversity (Gavin et al., Reference Gavin, McCarter and Mead2015; Sterling et al., Reference Sterling, Ticktin and Kipa Kepa Morgan2017b) and political ecology, law and environmental justice (Gambon & Rist, Reference Gambon and Rist2018; Hohenthal et al., Reference Hohenthal, Räsänen and Minoia2018).

The majority of articles reviewed (51 percent) engage with the MEB approach in a relatively superficial manner to illustrate that combining multiple knowledge systems is a sustainability challenge. The literature is awash with programmatic articles with calls to include, combine and integrate knowledges to find solutions to sustainability problems (e.g. Balvanera et al., Reference Balvanera, Calderón-Contreras and Castro2017; Vasseur et al., Reference Vasseur, Horning and Thornbush2017). However, still very little attention is paid to exactly how this will be done. Additionally, 20 percent of articles reviewed represent collaborative processes in practice but do not apply a MEB approach. Many articles view actors as stakeholders and talk about ‘open participation and open consultation’ (e.g. Livoreil et al., Reference Livoreil, Geijzendorffer and Pullin2016) rather than addressing their role as knowledge-holders and experts and the need for equitable platforms for engagement, mobilisation and translation of indigenous and local knowledge.

The MEB approach has also received significant attention in the grey literature and science–policy–practice community. For example, it is called for as a way of ensuring equitable participation for indigenous, local and scientific knowledge in monitoring of the Convention on Biological Diversity. For example the Convention’s Aichi target 18 on traditional knowledge, innovation and practices, along with the Community-Based Monitoring and Information Systems, is a bottom-up approach developed by indigenous peoples and local communities to contribute their experiences and observations through monitoring (CBD, 2014; Farhan Ferrari et al., Reference Farhan Ferrari, de Jong and Belohrad2015). Further, a MEB approach has been encouraged in traditional knowledge inventories, as well as in the development of safeguards for biodiversity financial mechanisms and Reducing Emissions from Deforestation and Degradation (REDD+) under the United Nations Framework Convention on Climate Change.

To illustrate the implementation of the MEB approach in the literature, we have selected a small set of pertinent case studies. Table 6.1 presents an analysis using key features of the MEB approach – joint problem formulation, validation within knowledge system and the five tasks illustrated in Figure 6.1b.

Table 6.1 Articles applying a multiple evidence base in literature

Article citationIssue investigated including locationMultiple evidence baseEvidence of joint problem formulation and usefulness for all (Tengö et al., Reference Tengö, Brondizio and Elmqvist2014)Evidence of validation within knowledge systems (Tengö et al., Reference Tengö, Brondizio and Elmqvist2014)Evidence of application of the five tasks for successful collaboration across diverse knowledge systems (Tengö et al., Reference Tengö, Hill and Malmer2017). 1 = mobilise, 2 = translate, 3 = negotiate, 4 = synthesise, 5 = apply
Austin et al., Reference Austin, Vigilante and Cowell2017MEB approach to enable enriched picture of progress of an Indigenous Land and Sea Management programme run by the Wunambal Gaambera people in the Kimberley, AustraliaInforming the evidence base of the Wunambal Gaambera Healthy Country Plan using western scientific and local indigenous knowledge. Parallel integration of western and indigenous monitoring data/information to support co-production of enriched picture of country and management activities by the Uunguu Monitoring and Evaluation (M&E) CommitteeCollaborative and multiple evidence-based M&E committee designed the approach to conducting evaluation of progress and assessment of key targetsYes, each stream of knowledge was internally validated and cross-checked through collaborative self-assessment by Uunguu M&E Committee
  • 1, 2, 3, 4, 5

  • Local indigenous knowledge was mobilised through the planning process and the M&E committee; translation of information from various sources via monitoring methods; further negotiation and translation occurred within M&E committee; all knowledge streams synthesised through M&E committee meetings and reporting processes; and applied through adaptive management of the Healthy Country Plan

Nguyen et al., Reference Nguyen, Luom and Parnell2017Sustainable management of eroding mangrove-dominated muddy coasts in Vam Ray, Hon Dat district, Kien Giang Province, VietnamPartnership between government agencies, scientists and local communities. Methods included literature review, semi-structured interviews, participatory community meetings, participatory diagramming and thematic analysis. The introduction and analysis of different knowledge systems are undertaken in participatory community meetings, semi-structured interviews, field visits, photovoice and debriefingsAll parties agreed to co-formulate the problems and use local and scientific knowledge to generate and pilot new knowledge for solving them. It was agreed to build local capacity and to utilise as many local resources as possible for developing the fence and nursery construction, to solve serious erosion problems affecting the communityLocal knowledge of, e.g. Melaleuca fence construction was validated by local experts based on their experience. However, facing new challenges in the community created interest in other knowledge such as scientific knowledge. The collaboration led to new knowledge about fencing for controlling coastal erosion
  • 1, 2, 5

  • Local knowledge held by individuals regarding traditional Melaleuca fences, and local contexts were systematically collected and brought together with the relevant scientific knowledge in relation to sedimentation and coastal dynamics in Kien Giang, Vietnam into ecologically based, cost-effective strategies for successfully controlling coastal erosion

Robinson et al., Reference Robinson, Maclean and Hill2016Water management in territories of aboriginal people connected to the Girringun Indigenous Corporation (Girringun) in northern AustraliaParticipatory maps created in workshops of Girringun support staff, Aboriginal rangers, some Girringun elders who are also artists and some of the authors to determine the values, knowledge and management aspirations of participants for their ‘fresh water country’. Second workshop to discuss the values that the participants had for native plants and trees and to identify risks to those values and the attributes of partnerships that support these valuesCo-research approach, in which Girringun representatives worked with the researchers to select participants and design the participatory mapping workshops, advising on an appropriate focus, location and design for each workshopYes, the integrity of each indigenous knowledge system was maintained throughout process
  • 1, 2, 3, 4, 5

  • Girringun representatives and scientists created individual maps to mobilise and translate the knowledge needed for Girringun and its associated tribal groups to assess two distinct issues of concern. Collective watershed maps were also used to negotiate knowledge and although there was some variety in the information shared by different participants, the integrity of each indigenous knowledge system was maintained throughout the process. Synthesis of themes occurred through creation of targeted research ‘products’, including a one-page summary, that could be used by the research team and the Girringun Indigenous Community to translate the results of the project in a way that was useful to the participants, Girringun and the wider natural resource management community

Smith et al., Reference Smith, Basu and Chatterjee2017Developing conservation strategies for pollinators in the context of pollinator decline in Orissa, IndiaPeer-to-peer validation of trends and statements distilled from focus groups including 50 smallholder subsistence farmers, including tribal people, who have personal and procedural knowledge of crop production, and rural advisors; anecdotal network. This was in preparation for integration with scientific knowledge from other regionsThe problem (a potential pollinator crisis) was defined by scientists, who recognised the dearth of information on diversity of crops and pollinators and together with farmers and rural advisors collated traditional and local knowledge on the sameYes. Peer-to-peer validation of indigenous knowledge of trends and statements distilled from focus groups including 50 farmers and rural advisors
  • 1, 2, 4

  • Traditional knowledge of crop diversity and pollinators was elicited and internally validated, providing a consensus on knowledge which was collated for integration with scientific knowledge

Molnár et al., Reference Molnár, Kis and Vadász2016a, Reference Molnár, Sáfián, Máté, Roué and Molnár2016bMitigation of conflicts between cattle herding and conservation management of salt-steppe and wood pastures in HungaryAn inventory of objectives and practices of herders (representing traditional knowledge) and conservationists and ethnobotanists (scientific knowledge) were collected by participatory knowledge co-production in teamwork with the co-authors. Possible resolutions to potential conflicts were suggested. Methods include: (1) participatory observation, (2) semi-structured interviews with herders and conservationists, (3) co-author herders and conservation managers completed and clarified the contents of the tables in two roundsHerders and ethno-ecologists jointly formulated the problem, and potential solutions were suggested by all partiesYes. Herder interviews colleagues that explain their observations and experiences and jointly validate the relevance for the issues. Less focus on conservation manager’s validation. However, importance of integrity, equity and reciprocity in their knowledge-based interactions highlighted
  • 1, 2, 3, 4

  • Herders’ and conservationists’ knowledge of practices were elicited in interviews. Herders’ perspectives were mobilised and translated through film. Data were negotiated among diverse author group and synthesised for joint publication

Strangway et al., Reference Strangway, Dunn and Erless2016A registry monitoring an aboriginal subsistence fishery in the Cree community of Waskaganish (Waskaganish Voluntary Anadromous Cisco Catch Registry) within the Environmental Impact Assessment (EIA) Follow-up Phase after diversion of the Rupert River for hydroelectric plant at Nûtimesânân in Northern Quebec, CanadaCollaboration between Hydro-Quebec and the impacted community. Bringing together different monitoring reports of cisco including the Voluntary Catch Registry of the Crees of Waskaganish First Nation, biological monitoring and complementary studiesUnclear. State-owned utility, Hydro-Quebec, proposed a monitoring programme after authorisation of river diversion which would include studies on cisco spawning success, as cisco harvesting is key for Waskaganish First Nation community’s cultural identity and subsistence economy. Four community members hired as monitors to collect data on cisco catchYes. In the voluntary cisco catch registry programme, compiled data were presented to land users at the end of the fishing season for interpretation and validation. Shared observations regarding the fishing season, fishing success and any other comments, as well as how the data will be presented outside of the community, were discussed and included in the final Registry reports
  • 1, 2, 4, 5

  • Fishers propose mitigation measures to increase fishing success under new flow rates, and stakeholders assess their potential, with the results of the various cisco monitoring programmes, including the Voluntary Registry, featuring prominently in the decision. Once measures are implemented, the Registry programme is used to evaluate their effectiveness by collecting catch data

Notes: Examples assessing the experience of applying a MEB approach, showing the issue investigated, the location and the multiple evidence base, in literature that either quoted Tengö et al. (Reference Tengö, Brondizio and Elmqvist2014) or referred to MEB. The review examined evidence of joint problem formulation and usefulness for all stakeholders (defined by Tengö et al., Reference Tengö, Brondizio and Elmqvist2014), evidence of internal validation within knowledge systems (defined by Tengö et al., Reference Tengö, Brondizio and Elmqvist2014) and evidence of application of each of the five tasks for successful collaboration across diverse knowledge systems (as defined by Tengö et al., Reference Tengö, Hill and Malmer2017).

The cases illustrate that, in different contexts, specific phases of the MEB approach presented by Tengö et al. (Reference Tengö, Brondizio and Elmqvist2014, Reference Tengö, Hill and Malmer2017) are more or less useful, and are operationalised in different ways. The process of co-defining the problem and questions together with all knowledge-holders appears to be a challenge not taken up in all cases, often with scientists or project proponents defining a problem, and then approaching indigenous and local knowledge-holders and local communities through consultation sessions to join and support the collaboration (e.g. Strangway et al., Reference Strangway, Dunn and Erless2016; Lyver et al., Reference Lyver, Timoti and Jones2017; Smith et al., Reference Smith, Basu and Chatterjee2017). However, other papers do emphasise the critical role of joint problem formulation for the success of conservation interventions (Brondizio et al., Reference Brondizio, Foufoula-Georgiou and Szabo2016; Galvin et al., Reference Galvin, Reid and Fernández-Giménez2016).

Maintaining the integrity of diverse knowledge systems throughout collaborative knowledge processes also appears to be a particular challenge in science-driven processes. Actively thinking about what validation of knowledge within knowledge systems means (rather than using science to validate local knowledge) and how it may be embedded in practice is absent from most papers. There are notable exceptions that explicitly reflect upon this challenge (e.g. Austin et al., Reference Austin, Vigilante and Cowell2017) and suggest new approaches, such as peer-to-peer validation by farmers (Smith et al., Reference Smith, Basu and Chatterjee2017; Table 6.1). Other papers do not address this explicitly, but still engage with how local knowledge systems evaluate knowledge (e.g. through interactions with internally acknowledged experts and their local institutions) (Molnár et al., 2016; Nguyen et al., Reference Nguyen, Luom and Parnell2017; see Table 6.1). Additionally, joint discussion and analysis of data across knowledge systems has sometimes been incorporated through formal consultation structures or committees (e.g. Strangway et al., Reference Strangway, Dunn and Erless2016; Austin et al., Reference Austin, Vigilante and Cowell2017; Reed & Abernethy, Reference Reed and Abernethy2018). The articles also illustrate the progress in development of methods to facilitate the phases and activities defined in Tengö et al. (Reference Tengö, Brondizio and Elmqvist2014, Reference Tengö, Hill and Malmer2017) to combine and relate multiple data through e.g. participatory scenario planning, focus groups (Danielsen et al., Reference Danielsen, Jensen and Burgess2014), fuzzy cognitive maps and community monitoring with digital devices (Brammer et al., Reference Brammer, Brunet and Burton2016). The use of art (Rathwell & Armitage, Reference Rathwell and Armitage2016; Polfus et al., Reference Polfus, Simmons and Neyelle2017), participatory maps (Robinson et al., Reference Robinson, Maclean and Hill2016) or film (Molnár et al., 2016) to mobilise, translate and present knowledge on an equitable platform has facilitated joint analysis and negotiation. Articles also illustrate practical ways of maintaining equity, such as creating research agreements or protocols concerning intellectual property; free, prior and informed consent; the roles and responsibilities of each member of the project team (Robinson et al., Reference Robinson, Maclean and Hill2016); and recognising indigenous and local knowledge-holders as authors on scientific articles (Molnár et al., Reference Molnár, Kis and Vadász2016a; Smith et al., Reference Smith, Basu and Chatterjee2017; Table 6.1).

The citations suggest that the mobilisation and translation activities suggested by Tengö et al. (Reference Tengö, Hill and Malmer2017) have had particular resonance in the conservation and sustainability literature. There has been consistent progress towards the explicit mobilisation and translation of indigenous knowledge and worldviews (Gonzales, Reference Gonzales2015; Vogt et al., Reference Vogt, Pinedo-Vasquez and Brondízio2016; Horstkotte et al., Reference Horstkotte, Utsi and Larsson-Blind2017; Timoti et al., Reference Timoti, Lyver and Matamua2017). In this way, the mobilisation of multiple knowledge systems contributes to a movement towards environmental justice and pluralism in decision-making (Hohenthal et al., Reference Hohenthal, Räsänen and Minoia2018), as well as recognising indigenous peoples’ autonomous actions towards dealing with climate change (Gonzales, Reference Gonzales2015).

In the next section, we use three in-depth case studies to further explore the value of a MEB approach to contribute to conservation partnership based on diverse sources of knowledge.

6.5 Exemplifying MEB cases and reflecting on lessons learned

Here we present three case studies that have explicitly implemented a MEB approach (Table 6.2). The first two set out processes to address local conservation and development issues. The third is an international dialogue meeting where the aim was to create a platform to discuss a fundamental crux in conservation globally – how to realise synergies between human rights and biodiversity conservation, and support local people and conservation organisations in becoming strategic allies.

Table 6.2 Summary of MEB tasks to guide knowledge collaborations (Tengö et al., Reference Tengö, Hill and Malmer2017) as applied in the three case studies

MEB phasesMultiple evidence-base case examples
4.1. Piloting the MEB: Tharaka’s river is running dry4.2. Mobilising indigenous knowledge systems for saltwater country across the Kimberley, Australia4.3. Justice and conservation: Global Dialogue on Human Rights and Biodiversity Conservation
  • 1. Mobilise

  • Develop knowledge-based products through a process of innovation and/or engaging with past knowledge and experience

  • The preparatory process for the ecocultural mapping, where the elders of the clans start to engage and document their experiences

  • The process of making ecocultural maps and calendars, which mobilised and synthesised knowledge on the landscape and how it has changed over time

  • Project objectives and research activities identified by an intercultural collaborative Working Group (WG) to ensure focus on local priorities. At the individual community workshop level, each of the indigenous ranger groups designed the specific activities, venue and participants. Focus group discussions and knowledge-holder interviews were selected as appropriate methods for indigenous people to use their knowledge to inform the process. Ranger groups were all equally resourced to facilitate and participate in research activities

Participation occurred before the actual dialogue, through interactions over internet. Preparation of indigenous community representatives to present and mobilise knowledge about ecology as well as human rights among the participants. The contributions from indigenous communities were planned to be presented during walking workshops in the Oigek Community. However, an outbreak of Marburg virus meant the dialogue was moved to Eldoret. The stories were told by community representatives attending the workshop
  • 2. Translate

  • Adapt knowledge products or outcomes into forms appropriate to enable mutual comprehension in the face of differences between actors

  • Occurred together with mobilisation in the ecocultural mapping event, where representatives from local authorities, regional authorities and national institutions were present

  • Also later in the process, in the documentation of customary laws that were considered along with modern law, and in the gazetting of sacred sites led by the National Museums of Kenya

  • All research results generated by indigenous workshop participants and knowledge-holders were collated and provided in short, simple reports to relevant indigenous communities for validation. A period of one month was provided to give feedback, make amendments, add anything that was missing or embargo content

  • Once the reports were validated they were presented to the WG who discussed how to analyse and represent results from the perspective of both indigenous people and their non-indigenous partners in collaborative management of saltwater country

The core focus for the dialogue and concerns articulated was what indigenous knowledge, practice and belief systems mean for indigenous peoples, in relation to how it is perceived by scientists and government representatives. But also, what human rights mean if applied to biodiversity conservation decisions. Dialogue was designed to encompass the very diverse ways of expression, experiences and perspectives among the participants
  • 3. Negotiate

  • Interact among different knowledge systems to develop mutually respectful and useful representations of knowledge

  • All actors accepted evidence of the critical situation for the river brought up by the ecocultural mapping, complemented with technical data from research and government institutions provided by regional authorities

  • Negotiation also happened in the development of action plans following the mapping process, where the community and local authorites agreed upon actions to improve the condition of the river

  • WG and research team had regular contact to ensure a collaborative research approach and facilitate discussions on saltwater research and monitoring at a regional scale

  • WG provided an important conduit between indigenous communities, their staff and the research community

  • WG held a final workshop attended by indigenous people, indigenous rangers, indigenous representative bodies, scientists and federal and state governments to raise awareness and seek final feedback on project outputs

Negotiations included how to interpret biodiversity data from different approaches for management and governance of ecosystems, along with knowledge about human rights principles and legislation, and cultural and socio-economic use of biodiversity
  • 4. Synthesise

  • Shape broadly accepted common knowledge bases for a particular purpose

  • Synthesis occurred in compiling customary laws and conventional laws together with authorities, when gazetting the sacred sites and in the development of action plans for protecting the river

  • Indigenous participants engaged in regular synthesis of results, from scoping, defining research questions and conducting fieldwork, to analysing results and communicating outcomes. The use of the MEB approach was a result of the WG’s capacity to consider a range of possible tools and processes and choose the ones that work best for the project

  • Indigenous people had the opportunity to continuously monitor to ensure that project frameworks and tools fitted into the holistic, contextual and current situation in the Kimberly Saltwater Country

Agreement in place, based on evidence, that synergies are possible between conservation and human rights. In policy and practice, more efforts are needed to synthesise ‘how’ this can happen. The dialogue did not aim for a synthesis, that is for a later stage, with policy decisions leading to application. When presenting a summary of evidence, it was considered important to recognise convergence, but also identify and recognise where there were still disagreements
  • 5. Apply

  • Use common knowledge bases to make decisions and/or take actions and to reinforce and feed back into the knowledge systems

  • The process led to applications to improve river conditions at multiple levels:

  • Revitalisation of rituals and enforcement of customary law at sacred sites

  • Government recognition of the custodians as protecting the sites

  • Enforcement of regulations of water extraction and riparian zone protection by regional authorities

  • The primary outcome: a regional network of indigenous people who have negotiated as regional knowledge brokers with their elders and knowledge-holders

  • Short-term funding secured for the WG to support implementation, modification and compliance of the best-practice approaches developed

  • Tools developed:

    • Regional saltwater monitoring framework based on indigenous knowledge identified social, cultural, economic and environmental values

    • Digital research protocol and application systems

    • Set of guidelines to describe simple processes for knowledge collaborations

The evaluation showed that knowledge for conservationists about Human Rights law and implementation representing a strand of research was considered useful. There was potential for application of insights around equal benefit of conservation and human rights in all the cases brought up in the dialogue
6.5.1 Piloting the MEB approach: Tharaka’s river is running dry
6.5.1.1 Context

Drought is a recurring challenge to the livelihoods of the people in Tharaka, Kenya. Kathita River is the main water source and of paramount importance, economically, culturally and spiritually. Fourteen sacred natural sites along the river are protected by the communities for their cultural and spiritual values. In recent years, the government’s policy guidelines and regulations for protecting the river have not been upheld and traditional ecological law has not been enforced either. This has led to excessive and often illegal abstraction along the river’s course, degradation of the riverine vegetation and destruction of the catchment area. The local people, led by clan-based custodians of the sacred sites, decided to come together to find ways of protecting the river using their indigenous and local knowledge and practices and customary laws. A non-governmental organisation in the area offered to facilitate an eco-cultural mapping process to enhance the eroded local capacity to govern the river.

A preparatory process brought together custodians of the sacred sites along the river. Local community organisations, county leaders and government institutions, including the National Museums of Kenya, were successively engaged in the process. In August 2014, community members jointly developed eco-cultural maps and calendars of the past and present, which illustrated changes in the integrity of their social–ecological system. Based on these, maps of the future envisioning different scenarios were drafted, creating a collective understanding and describing alternative pathways for the future. The maps and insights were shared and discussed with different actors beyond the local community.

A couple of years after the initial process, several of the problems identified with river governance have been addressed: strategies have been formulated for local authorities to reach out to land owners to safeguard riparian reserves. Tree seedlings are raised and distributed to land owners for planting in order to protect the riparian zone. The National Museums of Kenya have, together with the communities, gazetted the sacred sites along Kathita River, which has given them a government-recognised status. Rituals are again carried out at the sacred sites and the customary rules are enforced (Mburu, Reference Mburu2016).

6.5.1.2 Role of the MEB

The local non-governmental organisation convening the eco-cultural mapping in Tharaka is a member of a bridging organisation who were engaged in the initial dialogue across knowledge systems and volunteered to pilot a MEB approach. Eco-cultural mapping emerged as a culturally appropriate tool for knowledge mobilisation to enhance ecosystem governance for the society at large, beyond the community benefits expected by the clans that initiated the process. This also led to a greater understanding of the roles that different actors play in the local community and who to approach, how to formulate proposals and the utility of referring to established facts from community-based monitoring of the river.

The process contributed to unifying actors towards an enriched picture of understanding that could be shared and discussed with decision-makers outside the community. The eco-cultural mapping activity focused on how knowledge can be translated and negotiated to benefit an official process of conservation of sacred sites, and better ecosystem management of the Kathita river at large, through collaboration to protect the landscape (see Table 6.2). For this step, it was important to engage with actors with the authority to act in the customary governance system. Thus, the clans that were managing the sacred sites had a critical role in mobilising other community members.

6.5.1.3 Challenges and opportunities

The power imbalance between farmers with resources to extract and use water, and the majority of the community who did not have such resources, but were still exceptionally dependent on Kathita River as a water source, proved a challenge. Community research groups have been formed to solve specific emerging problems defined by the community.

The initiative for the eco-cultural mapping process came from the communities and the local non-governmental organisation, who contacted government and later also the Natural Museums of Kenya in order to catalyse change and ensure impact. The local actors as initiator created a solid base for trustful collaborations across knowledge systems.

6.5.2 Mobilising indigenous knowledge systems for saltwater country across the Kimberley region, Australia
6.5.2.1 Context

The Kimberley region in tropical north-western Australia is globally significant for its biodiversity, relatively intact ecosystems and its aesthetic and recreational values. Indigenous peoples comprise almost half of the region’s population and have ownership or management rights over most of the land and sea. They are caretakers of a diverse cultural landscape dating back at least 60,000 years. The Australian public places high value on the cultural and natural assets of the Kimberley. The Western Australian Government concluded in 2011 that to ensure the best possible outcomes of conservation efforts in the Kimberley, a combination of indigenous knowledge and scientific knowledge was needed.

The Kimberley Indigenous Saltwater Science Project (KISSP) was established by a group of indigenous peoples and their organisations, research institutes, corporations and government organisations to investigate ways of co-producing collaborative monitoring, management and research regionally. A working group was established in 2014 with representatives from seven indigenous groups (Balangarra, Bardi Jawi, Dambimangari, Karajarri, Nyul Nyul, Wunambal Gaambera and Yawuru peoples) and key staff from local indigenous organisations. The working group recruited a team of researchers to assist the project. In total, there were 103 indigenous participants in five Traditional Owner workshops and one Knowledge-Holder interview.

6.5.2.2 Role of the MEB

Although not intentionally applied at the commencement of the project, the MEB process was followed intuitively by the experienced practitioners involved. Midway through the project, the MEB approach was formally introduced to participants, who immediately recognised its value in describing their practice. The working group agreed to adopt the MEB as an overarching framework for the KISSP project and to design regional frameworks for collaborative knowledge production, monitoring, research and management of Kimberley Saltwater Country (Table 6.2).

6.5.2.3 Challenges and opportunities

The biggest challenge faced by the KISSP was to establish engagement with indigenous peoples in the Kimberley. Prior to the formation of the working group, the project struggled for many years to create dialogue with indigenous peoples. Finally, a workshop was held to identify collaborative pathways towards project goals. The intervention of the indigenous-led working group demonstrated the potential for MEB approaches to ensure useful outcomes through intercultural and interdisciplinary projects.

Initially, lack of investment in the capacity of indigenous peoples and their organisations to engage in the research process limited progress. This should not be understood as a lack of knowledge or capacity to care for saltwater country, but rather as a need for support to mobilise their knowledges and practices to contribute to the KISSP as a collaborative, intercultural project. There was a prior assumption that indigenous peoples and their knowledge and practice could easily fit into a regional project that comprised indigenous and scientific knowledge systems side by side. There was no insight of the need for recognition and equity, and for explicit usefulness of the research products for all involved in collaborative practices. For example, there was consistently a subconscious assumption that flows of knowledge produced throughout the project would be channelled in a unilateral direction to scientists in the regional capital in the form of ‘data’ to be analysed so as to suitably inform the policy and decision-making processes of the state. The communication of this new information back to indigenous peoples in the Kimberley was more of an afterthought and, presumably, seen more as a bureaucratic demand than a practical mechanism for improving collaborative management of Saltwater Country. This assumption ignored the practical, and fairly reasonable, requirement of local indigenous peoples that any knowledge shared or co-produced through collaborative research and monitoring be made available for informing their own local decision-making and practice for looking after Saltwater Country. The indigenous-led KISSP Working Group made this point patiently and constructively and, thus, ensured that the project could produce several locally useful outputs and outcomes for indigenous peoples in Kimberley Saltwater Country.

6.5.3 Justice and conservation: Global Dialogue on Human Rights and Biodiversity Conservation
6.5.3.1 Context

The Global Dialogue on Human Rights and Biodiversity Conservation was an international meeting initiated to address the conflicts that have often emerged across the globe between conservation agencies and indigenous peoples with longstanding relationships with their ancestral territories, co-organised by SwedBio at Stockholm Resilience Centre, Forest Peoples Programme, Natural Justice and the Chepkitale Indigenous Peoples Development Project as the local host in Kenya. The organisers represented actors engaged from different scales and perspectives, which created confidence and legitimacy for the dialogue. The dialogue started from the conviction that local people and conservation organisations could be strategic allies. It was attended by conservation agencies, social justice and human rights advocates, biodiversity conservation and sustainable use experts, legal and human rights professionals, members of community-based organisations, government officials, UN organisations and academics. It was designed in a global policy-setting context, while also aiming to contribute to local ways forward. The venue for the dialogue, Eldoret, Kenya, is situated between two biodiversity-rich areas conserved by indigenous peoples as their ancestral lands. The Ogiek people are an indigenous hunter-gatherer community on Mt Elgon, at the border of Uganda, while the Sengwer people are traditionally living with and taking care of the Embobut Forests. Both Ogiek and Sengwer have been faced with repeated attempts of eviction over decades in the name of conservation. In 2011, through a conservation-related mediation method called the Whakatane mechanism, the Ogiek communities in Mt Elgon reached an initial agreement to live in and govern parts of their ancestral lands. However, the Sengwer have rather experienced increased tensions in later years.

6.5.3.2 Role of the MEB

Globally there is an increased recognition that human rights protection can, and should, be complementary to safeguarding biodiversity and ecosystems (Knox, Reference Knox2017), but there is a need to mainstream how, through good case examples and methods in policy and practice. A MEB approach was introduced in the preparatory process before the dialogue as part of the multi-actor dialogue method. The design process started with informal discussions between conservation agencies, indigenous peoples, human rights professionals and the organisers a year before the dialogue took place. The long preparatory process helped mobilise knowledge and confidence as a base for common understanding of the overarching ecological, legal, institutional and political challenges among participating actors. Through the dialogue process, the MEB approach provided guidance to ensure equity, reciprocity and usefulness for all actors. In the evaluation, the community representatives stressed they had never before had experience of being recognised and presenting their stories as evidence on an equal footing with science and governments.

6.5.3.3 Challenges and opportunities

Establishing a collaboration among different actors at national level in Kenya representing government, indigenous peoples and conservation agencies that generally do not meet was the greatest challenge. Thanks to the global context of the meeting, the presence of international actors with diverse experiences contributed to a constructive dialogue. Interactions among indigenous peoples and scientists were successful because a common understanding of the MEB approach had been established during the preparation. Persistent barriers between indigenous peoples and governments still exist in local cases, in particular the Sengwer people, and should be resolved through policy and legal processes. However, establishing MEB processes whenever governance of ecosystems and biodiversity can be enhanced through collaborative processes across multiple knowledge systems can be useful for all involved in the meantime.

6.6 Sharing lessons from the three cases

In the first case from Tharaka River, the importance of mobilising indigenous and local knowledge as a solid base for translation and negotiation phases was very clear. This then helped people speak about their knowledge, and also catalysed the revitalisation of eroded institutions and rules that previously served to protect the river, including the recognition and protection of the sacred sites. As the problem formulation was owned by the community, this enabled articulation of the importance of the sacred sites for understanding previous river governance, and motivated local people to restore the river. Later, they contacted the Natural Museums of Kenya, to provide support in gazetting their biodiversity-rich sacred sites for formal national recognition. This illustrates the important role that values and beliefs in diverse knowledge systems can play for conservation, how they may be identified, and how knowledge and governance capacity is embedded in the belief systems.

In the case from the Kimberley, the use of the MEB and the role of indigenous and local knowledge in collaborative management, created space and enthusiasm for experimenting with new ways of combining knowledge systems for management and governance of Saltwater Country. The KISSP demonstrated that working with multiple knowledge systems and disciplines in the context of unequal power relations requires design, support and monitoring of mechanisms that can maintain constant dialogue (e.g. the KISSP working group). Thinking of the collaboration as ‘intercultural’ was useful for understanding what capacity development was required for all actors. No single party had capacity deficits, but the collective needed to build joint capacity for weaving knowledge systems in ethical and equitable ways.

In the Global Dialogue on Human Rights and Conservation, reaching synergistic solutions between conservation and human rights once again was about overcoming power imbalances. The dialogue was an opportunity for key actors with different knowledges, experiences, worldviews and power to meet in a neutral context. Diversity of experiences (positive and negative) across scales and a careful mix of actors helped to overcome these imbalances during the dialogue. Mobilisation of indigenous and local knowledge and strengthening confidence among participating community representatives, but also knowledge about human rights and other legal aspects before the dialogue, was critical for deliberations. The recognition of indigenous rights and the value of their knowledge and practices for conservation expressed by researchers contributed to trust followed by constructive proposals. Positive experiences from successful collaborations in conservation of indigenous lands contributed to exploring ways forward in cases where conflicts persist. The learning across different sectors and scales, such as ecologists learning about human rights aspects, was appreciated in the evaluations. It also became clear that the deepest conflicts may not relate to conflicting evidence from different knowledge systems regarding ecology, but to controversial policy, such as the eviction of people from conservation areas.

In all three cases, the main challenge of the collaborative process was to overcome power imbalances and build trust and confidence. The focus on recognising, mobilising and discussing evidence from diverse knowledge systems was an entry point that contributed to the development of strong collaborative partnerships. Designing a process that was considered useful for all involved was critical to securing successful and sustainable outcomes, new and useful ways to combine and apply knowledge from diverse knowledge systems, and sometimes the generation of new knowledge. In all cases, the aim of creating synergies across knowledge systems for providing evidence on sustainable governance could be realised when all holders of knowledge gained from collaborations. A MEB approach, on whatever level it is conducted, emphasises the importance of collaborative processes that value multiple knowledges and practices needed to sustain the social–ecological landscape to the double benefit of sustainable livelihoods and conservation over the long term. Further, the collaborative relationships of trust developed provide new opportunities to align multiple modes of governance of ecosystems, to ensure decisions and policy are based on all available knowledge.

6.7 Discussion

In this chapter, we review the use of one recent and important approach to combining the knowledge of indigenous peoples, local communities and scientists for sustainability and conservation partnerships. We have focused on the MEB and its potential for building more inclusive understanding of multiple sources of evidence, how it is generated and how it is transmitted among diverse conservation actors. We argue that such an approach is important for better understanding of interlinked social–ecological systems, strengthening conservation partnerships and identifying new evidence-based pathways towards sustainability. Our review and the three case studies show examples of different ways to move forward that recognise the complementarity and integrity of knowledge systems in addressing specific problems (Molnár et al., 2016; Smith et al., Reference Smith, Basu and Chatterjee2017), create conditions (and methodologies) for full and open dialogue on how to move ahead, overcome power inequalities and navigate cultural differences (Robinson et al., Reference Robinson, Maclean and Hill2016; Reed & Abernethy, Reference Reed and Abernethy2018). We demonstrate reciprocal synergies between indigenous and local knowledge and conservation science and rich cases of how cross-fertilisation leads to stronger partnerships and better outcomes. The three case studies also show that the MEB requires partnerships that are underpinned by recognition, respect and understanding of diverse knowledge systems, and that the process for producing and applying common knowledge to problems cannot be viewed separately from the outcomes of partnerships. That is to say, much like the concept of adaptive management in conservation, the diversity and dynamism of knowledge systems dictate that the process of collaboration be taken as seriously as the achievement of conservation outcomes themselves (Gavin et al., Reference Gavin, McCarter and Berkes2018).

More work is needed to further elaborate how to implement a MEB approach in different processes and contexts. The IPBES process has struggled with the tension between open collaboration and the demands for structure set by the scientific knowledge governance. There is yet some way to go to better acknowledge and solve epistemic challenges, such as diverse modes of validation across knowledge systems (Löfmarck & Lidskog, Reference Löfmarck and Lidskog2017; Obermeister, Reference Obermeister2017). There is also a need to continue developing tools and approaches for bridging knowledge systems that are connected to local, cultural, social and ecological conditions. Our review illustrates that indigenous peoples, local communities and scientists have begun to tackle this challenge (Molnár et al., 2016; Robinson et al., Reference Robinson, Maclean and Hill2016; Smith et al., Reference Smith, Basu and Chatterjee2017), but further dialogue is required, both horizontally across local scales and vertically through local to global institutions.

We have shown that a MEB approach has been particularly effective in dialogues where there are power imbalances among actors and historical bias concerning the validity or usability of knowledge systems other than western approaches to science (see also Klenk & Meehan, Reference Klenk and Meehan2015). Building trust and respect is especially pertinent in the context of ongoing and historical injustices and abuse of indigenous rights, and requires the recognition of indigenous peoples as rights-holders and defenders of biodiversity, who maintain management and governance systems of vast ecosystems (Brondizio & Le Tourneau, Reference Brondizio and Le Tourneau2016; Mistry & Berardi, Reference Mistry and Berardi2016).

Tengo et al. (Reference Tengö, Hill and Malmer2017) suggest five tasks that can guide processes that build trust and agency (see Figure 6.1b), while at the same time building a stronger evidence base for action. We find in our review that the mobilisation task is often neglected, or that documentation of indigenous and local knowledge is not fully recognised. More research is needed, but mobilisation of knowledge and empowerment of knowledge-holders may be critical steps for successful knowledge collaborations that also contribute to strengthening collaborative governance capacity. We also find that explicit joint problem formulation and analysis across knowledge systems is absent from many processes and is clearly a challenge in regional and global assessments with rigid scientific formats (Livoreil et al., Reference Livoreil, Geijzendorffer and Pullin2016; Nesshöver et al., Reference Nesshöver, Vandewalle and Wittmer2016; Oubenal et al., Reference Oubenal, Hrabanski and Pesche2017). Our case examples clearly show the importance of creating the right conditions for joint problem formulation.

It should be acknowledged that the implementation of a MEB approach is demanding, in terms of time and other resources, and requires strong commitment from all parties. However, we reiterate that there is mounting evidence of the potential positive outcomes in terms of novel indicators, more efficient responses to and implementation of findings, as well as for synergies between conservation and human well-being, including human rights (Danielsen et al., Reference Danielsen, Burgess and Jensen2010; Johnson et al., Reference Johnson, Alessa and Behe2015; Sterling et al., Reference Sterling, Ticktin and Kipa Kepa Morgan2017b, Reference Sterling, Filardi and Toomey2017c). As found by the participants in our third case example on reconciling conservation and human rights, conservation initiatives can play a positive role by engaging with communities and increasing their recognition as actors and partners who hold important and useful knowledge.

Our experiences derive mainly from dialogues and collaborations with indigenous and local knowledge-holders who have deep connections, obligations to care for and a duty to fight for their rights to actively govern their ancestral territories. We are aware that in many other contexts, local knowledge-holders may be less empowered and traditional governance systems and cultural connections may be displaced and eroded. However, we believe that insights about dialogue and partnership between indigenous peoples, local communities and scientists can also be applied in western, urban and developing settings, where local knowledge and experience may be less evident but remains critical for nurturing effective stewardship of biodiversity and ecosystems. Ultimately, the MEB approach contributes to a much-needed conceptual mind shift to mobilise all knowledge that is useful for maintaining the life-supporting ecosystems in our world.

References
Adams, W. M., Aveling, R., Brockington, D., et al. 2004. Biodiversity conservation and the eradication of poverty. Science (New York, N.Y.), 306(5699), 11461149. doi:10.1126/science.1097920
Austin, B. J., Vigilante, T., Cowell, S., et al. 2017. The Uunguu Monitoring and Evaluation Committee: intercultural governance of a land and sea management programme in the Kimberley, Australia. Ecological Management and Restoration, 18(2), 124133. doi:10.1111/emr.12257
Balvanera, P., Calderón-Contreras, R., Castro, A. J., et al. 2017. Interconnected place-based social–ecological research can inform global sustainability. Current Opinion in Environmental Sustainability, 29, 17. doi:10.1016/j.cosust.2017.09.005
Brammer, J. R., Brunet, N. D., Burton, A. C., et al. 2016. The role of digital data entry in participatory environmental monitoring. Conservation Biology, 30(6), 12771287. doi:10.1111/cobi.12727
Brondizio, E. S., Foufoula-Georgiou, E., Szabo, S., et al. 2016. Catalyzing action towards the sustainability of deltas. Current Opinion in Environmental Sustainability, 19, 182194. doi:10.1016/j.cosust.2016.05.001
Brondizio, E. S. & Le Tourneau, F.-M. 2016. Environmental governance for all. Science (New York, N.Y.), 352(6291), 12721273. doi:10.1126/science.aaf5122
Bruckmeier, K. 2016. Social–Ecological Transformation Reconnecting Society and Nature. London: Palgrave McMillan.
Buytaert, W., Zulkafli, Z., Grainger, S., et al. 2014. Citizen science in hydrology and water resources: opportunities for knowledge generation, ecosystem service management, and sustainable development. Frontiers in Earth Science, 2(October), 121. doi:10.3389/feart.2014.00026
Convention on Biological Diversity (CBD). 2014. Decision XII/12. Article 8(j) and related provisions.
Cornell, S., Berkhout, F., Tuinstra, W., et al. 2013. Opening up knowledge systems for better responses to global environmental change. Environmental Science & Policy, 28, 6070. doi:10.1016/j.envsci.2012.11.008
Danielsen, F., Burgess, N. D., Jensen, P. M., et al. 2010. Environmental monitoring: the scale and speed of implementation varies according to the degree of peoples involvement. Journal of Applied Ecology, 47, 11661168. doi:10.1111/j.1365-2664.2010.01874.x
Danielsen, F., Jensen, P. M., Burgess, N. D., et al. 2014. Testing focus groups as a tool for connecting indigenous and local knowledge on abundance of natural resources with science-based land management systems. Conservation Letters, 7, 380389. doi:10.1111/conl.12100
Díaz, S., Demissew, S., Carabias, J., et al. 2015. The IPBES Conceptual Framework – connecting nature and people. Current Opinion in Environmental Sustainability, 14, 116. doi:10.1016/j.cosust.2014.11.002
Díaz, S., Pascual, U., Stenseke, M., et al. 2018. Assessing nature’s contributions to people. Science, 359(6373), 270272. doi:10.1126/science.aap8826
Farhan Ferrari, M., de Jong, C. & Belohrad, V. S. 2015. Community-based monitoring and information systems (CBMIS) in the context of the Convention on Biological Diversity (CBD). Biodiversity, 16(2–3), 5767. doi:10.1080/14888386.2015.1074111
Gagnon, C. A. & Berteaux, D. 2009. Integrating traditional ecological knowledge and ecological science: a question of scale. Ecology and Society, 14(2), 19.
Galvin, K. A., Reid, R. S., Fernández-Giménez, M. E., et al. 2016. Co-design of transformative research for rangeland sustainability. Current Opinion in Environmental Sustainability, 20, 814. doi:10.1016/j.cosust.2016.03.003
Gambon, H. & Rist, S. 2018. Moving territories: strategic selection of boundary concepts by indigenous people in the Bolivian Amazon – an element of constitutionality? Human Ecology, 46, 2740. doi:10.1007/s10745-017-9960-z
Gavin, M. C., McCarter, J., Mead, A., et al. 2015. Defining biocultural approaches to conservation. Trends in Ecology & Evolution, 30, 140145. doi:10.1016/j.tree.2014.12.005
Gavin, M., McCarter, J., Berkes, F., et al. 2018. Effective biodiversity conservation requires dynamic, pluralistic, partnership-based approaches. Sustainability, 10, 1846. doi:10.3390/su10061846
Gonzales, T. 2015. An indigenous autonomous community-based model for knowledge production in the Peruvian Andes. Latin American and Caribbean Ethnic Studies, 10, 107133. doi:10.1080/17442222.2015.1034433
Hohenthal, J., Räsänen, M. & Minoia, P. 2018. Political ecology of asymmetric ecological knowledges: diverging views on the eucalyptus–water nexus in the Taita Hills, Kenya. Journal of Political Ecology, 25, 119.
Horstkotte, T., Utsi, T. A., Larsson-Blind, Å., et al. 2017. Human–animal agency in reindeer management: Sámi herders’ perspectives on vegetation dynamics under climate change. Ecosphere, 8(9), e01931. doi:10.1002/ecs2.1931
Jabbour, J. & Flachsland, C. 2017. 40 years of global environmental assessments: A retrospective analysis. Environmental Science & Policy, 77, 193202. doi:10.1016/j.envsci.2017.05.001
Johnson, J. T., Howitt, R., Cajete, G., et al. 2016. Weaving Indigenous and sustainability sciences to diversify our methods. Sustainability Science, 11, 111. doi:10.1007/s11625-015-0349-x
Johnson, N., Alessa, L., Behe, C., et al. 2015. The contributions of community-based monitoring and traditional knowledge to Arctic observing networks: reflections on the state of the field. Arctic, 68(5), 113. doi:10.14430/arctic4447
Kealiikanakaoleohaililani, K. & Giardina, C. P. 2016. Embracing the sacred: an indigenous framework for tomorrow’s sustainability science. Sustainability Science, 11(1), 5767. doi:10.1007/s11625-015-0343-3
Kenter, J. O. 2016. Editorial: Shared, plural and cultural values. Ecosystem Services, 21(October), 175183. doi:10.1016/j.ecoser.2016.10.010
Klenk, N. & Meehan, K. 2015. Climate change and transdisciplinary science: problematizing the integration imperative. Environmental Science & Policy, 54, 160167. doi:10.1016/j.envsci.2015.05.017
Knox, J. 2017. Report of the Special Rapporteur on the issue of human rights obligations relating to the enjoyment of a safe, clean, healthy and sustainable environment. UN OHCHR A/HRC/34/49.
Livoreil, B., Geijzendorffer, I., Pullin, A. S., et al. 2016. Biodiversity knowledge synthesis at the European scale: actors and steps. Biodiversity and Conservation, 25, 12691284. doi:10.1007/s10531-016-1143-5
Löfmarck, E. & Lidskog, R. 2017. Bumping against the boundary: IPBES and the knowledge divide. Environmental Science & Policy, 69, 2228. doi:10.1016/j.envsci.2016.12.008
Lyver, P. O. B., Timoti, P., Jones, C. J., et al. 2017. An indigenous community-based monitoring system for assessing forest health in New Zealand. Biodiversity and Conservation, 26, 31833212. doi:10.1007/s10531-016-1142-6
Martín-López, B. & Montes, C. 2015. Restoring the human capacity for conserving biodiversity: a social–ecological approach. Sustainability Science, 10, 699706. doi:10.1007/s11625-014-0283-3
Mathevet, R., Thompson, J. D., Folke, C., et al. 2016. Protected areas and their surrounding territory: social–ecological systems in the context of ecological solidarity. Ecological Applications, 26, 516. doi:10.1890/14-0421
Mburu, G. 2016. Reviving Indigenous and Local Knowledge for Restoration of Degraded Ecosystems in Kenya A. Contribution to the Piloting of the Multiple Evidence Base Approach. Report. Stockhom: SwedBio, Stockholm Resilience Centre.
Mistry, B. J. & Berardi, A. 2016. Bridging indigenous and scientific knowledge. Science, 352, 1274–5.
Molnár, Z., Kis, J., Vadász, C., et al. 2016a. Common and conflicting objectives and practices of herders and conservation managers: the need for a conservation herder. Ecosystem Health and Sustainability, 2(4), e01215. doi:10.1002/ehs2.1215
Molnár, Z., Sáfián, L., Máté, J., et al. 2016b. “It does matter who leans on the stick”: Hungarian herders’ perspectives on biodiversity, ecosystem services and their drivers. In Roué, M. & Molnár, Z., editors, Indigenous and Local Knowledge of Biodiversity and Ecosystem Services in Europe and Central Asia (pp. 4256). Paris: UNESCO.
Nadasdy, P. 1999. The politics of TEK: power and the ‘integration’ of knowledge. Arctic Anthropology, 36(1–2), 118. doi:10.2307/40316502
Nesshöver, C., Vandewalle, M., Wittmer, H., et al. 2016. The Network of Knowledge approach: improving the science and society dialogue on biodiversity and ecosystem services in Europe. Biodiversity and Conservation, 25(7), 12151233. doi:10.1007/s10531-016-1127-5
Nguyen, T. P., Luom, T. T. & Parnell, K. E. 2017. Developing a framework for integrating local and scientific knowledge in internationally funded environment management projects: case studies from Kien Giang Province, Vietnam. Local Environment, 22, 12981310. doi:10.1080/13549839.2017.1342617
Nkambule, S. S., Buthelezi, H. Z. & Munien, S. 2016. Oppportunities and constraints for community-based conservation: the case of the KwaZulu-Natal Sandstone Sourveld grassland, South Africa. Bothalia, 46(2), 18. doi:10.4102/abc.v46i2.2120
Obermeister, N. 2017. From dichotomy to duality: addressing interdisciplinary epistemological barriers to inclusive knowledge governance in global environmental assessments. Environmental Science & Policy, 68, 8086. doi:10.1016/j.envsci.2016.11.010
Oldekop, J. A., Holmes, G., Harris, W. E., et al. 2015. A global assessment of the social and conservation outcomes of protected areas. Conservation Biology, 30, 133141. doi:10.1111/cobi.12568
Oubenal, M., Hrabanski, M. & Pesche, D. 2017. IPBES, an inclusive institution? Challenging the integration of stakeholders in a science–policy interface. Ecology and Society, 22(1), art11. doi:10.5751/ES-08961-220111
Pascual, U., Balvanera, P., Diaz, S., et al. 2017. Valuing nature’s contributions to people: the IPBES approach. Current Opinion in Environmental Sustainability, 26–27, 716. doi:10.1016/j.cosust.2016.12.006
Polfus, J. L., Simmons, D., Neyelle, M., et al. 2017. Creative convergence: exploring biocultural diversity through art. Ecology and Society, 22(2), Art4. doi:10.5751/ES-08711-220204
Prado, H. M. & Murrieta, R. S. S. 2015. Ethnoecology in perspective: the origins, interfaces and current trends of a growing field. Ambiente & Sociedade, 18(4), 133154. doi:10.1590/1809-4422ASOC986V1842015
Quinlan, A. E., Berbes-Blazquez, M., Haider, L. J., et al. 2016. Measuring and assessing resilience: broadening understanding through multiple disciplinary perspectives. Journal of Applied Ecology, 53, 677687. doi:10.1111/1365-2664.12550
Rathwell, K. J. & Armitage, D. 2016. Art and artistic processes bridge knowledge systems about social–ecological change: an empirical examination with Inuit artists from Nunavut, Canada. Ecology and Society, 21(2), art21. doi:10.5751/ES-08369-210221
Reed, M. G. & Abernethy, P. 2018. Facilitating co-production of transdisciplinary knowledge for sustainability: working with Canadian Biosphere Reserve practitioners. Society and Natural Resources, 31, 3956. doi:10.1080/08941920.2017.1383545
Robinson, C. J., Maclean, K., Hill, R., et al. 2016. Participatory mapping to negotiate indigenous knowledge used to assess environmental risk. Sustainability Science, 11, 115126. doi:10.1007/s11625-015-0292-x
Sheil, D., Boissière, M. & Beaudoin, G. 2015. Unseen sentinels: local monitoring and control in conservation’s blind spots. Ecology and Society, 20(2), art39. doi:10.5751/ES-07625-200239
Smith, B. M., Basu, P. C., Chatterjee, A., et al. 2017. Collating and validating indigenous and local knowledge to apply multiple knowledge systems to an environmental challenge: a case-study of pollinators in India. Biological Conservation, 211(Part A), 2028. doi:10.1016/j.biocon.2017.04.032
Sterling, E. J., Betley, E., Sigouin, A., et al. 2017a. Assessing the evidence for stakeholder engagement in biodiversity conservation. Biological Conservation, 209, 159171. doi:10.1016/j.biocon.2017.02.008
Sterling, E. J., Filardi, C., Toomey, A., et al. 2017c. Biocultural approaches to well-being and sustainability indicators across scales. Nature Ecology and Evolution, 1, 17981806. doi:10.1038/s41559-017-0349-6
Sterling, E., Ticktin, T., Kipa Kepa Morgan, T., et al. 2017b. Culturally grounded indicators of resilience in social–ecological systems. Environment and Society, 8, 6395. doi:10.3167/ares.2017.080104
Strangway, R. E., Dunn, M. & Erless, R. 2016. Monitoring Nûtimesânân following the diversion of our river: a community-led registry in Eeyou Istchee, Northern Québec. Journal of Environmental Assessment Policy and Management, 18, 121. doi:10.1142/S1464333216500010
Tengö, M., Brondizio, E. S., Elmqvist, T., et al. 2014. Connecting diverse knowledge systems for enhanced ecosystem governance: the multiple evidence base approach. Ambio, 43, 579591. doi:10.1007/s13280-014-0501-3
Tengö, M., Hill, R., Malmer, P., et al. 2017. Weaving knowledge systems in IPBES, CBD and beyond – lessons learned for sustainability. Current Opinion in Environmental Sustainability, 26–27, 1725. doi:10.1016/j.cosust.2016.12.005
Tengö, M. & Malmer, P. 2012. Dialogue workshop on ‘Knowledge for the 21st Century: Indigenous knowledge, traditional knowledge, science and connecting diverse knowledge systems’. Usdub, Guna Yala, Panama, 10–13 April, 2012. Workshop Report. SwedBio at Stockholm Resilience Centre.
Timoti, P., Lyver, P. O., Matamua, R., et al. 2017. A representation of a Tuawhenua worldview guides environmental conservation. Ecology and Society, 22(4), 20. doi:10.5751/ES-09768-220420
Vasseur, L., Horning, D., Thornbush, M., et al. 2017. Complex problems and unchallenged solutions: bringing ecosystem governance to the forefront of the UN sustainable development goals. Ambio, 46, 731742. doi:10.1007/s13280-017-0918-6
Verburg, P. H., Dearing, J. A., Dyke, J. G., et al. 2016. Methods and approaches to modelling the Anthropocene. Global Environmental Change, 39, 328340. doi:10.1016/j.gloenvcha.2015.08.007
Vogt, N., Pinedo-Vasquez, M., Brondízio, E. S., et al. 2016. Local ecological knowledge and incremental adaptation to changing flood patterns in the Amazon delta. Sustainability Science, 11, 611623. doi:10.1007/s11625-015-0352-2
Waylen, K. A., Fischer, A., McGowan, P. J. K., et al. 2010. Effect of local cultural context on the success of community-based conservation interventions. Conservation Biology, 24, 11191129. doi:10.1111/j.1523-1739.2010.01446.x

Chapter Seven Informing conservation decisions through evidence synthesis and communication

Andrew S. Pullin
Collaboration for Environmental Evidence
Samantha H. Cheng
Center for Biodiversity and Conservation
Steven J. Cooke
Canadian Centre for Evidence-Based Conservation
Neal R. Haddaway , Biljana Macura
Stockholm Environment Institute
Madeleine C. Mckinnon
Bright Impact
and Jessica J. Taylor
Canadian Centre for Evidence-Based Conservation
7.1 Introduction

The volume of evidence from scientific research and wider observation is greater than ever. Approximately 2.5 million articles are published annually (Plume & van Weijen, Reference Plume and van Weijen2014) and this rate is increasing at around 3–3.5% per year (Ware & Mabe, Reference Ware and Mabe2015). Conservation is no exception to this trend and the result is a rapidly expanding body of potentially useful information for decision-makers (Li & Zhao, Reference Li and Zhao2015). While the expansion of research represents an important increase in knowledge generation, much of this information is scattered in fragments over increasingly diverse sources. This, along with the sheer volume, makes it harder for decision-makers to find, access and digest all of the relevant information on a particular topic, resolve seemingly contradictory results or simply identify a lack of evidence. Evidence synthesis is the process of searching for, and summarising, a body of research on a specific topic in order to inform decisions. The extent of relevant research may range from nothing, or one or two primary studies, to many hundreds. Despite the obvious potential value of synthesising findings from multiple studies (where two studies may be all that is needed to add value through synthesis), methods of rigorous evidence synthesis have been largely neglected until recently. We argue that it is time to place evidence synthesis as a central pillar of evidence-informed decision-making in conservation and environmental management.

As an enterprise, evidence synthesis is very broad and includes many and diverse methodologies, some more rigorous than others. For example, syntheses labelled as ‘literature reviews’ often lack standardised methodology, fail to report their methods and therefore lack transparency or the potential for repeatability (O’Leary et al., Reference O’Leary, Kvist and Bayliss2016). Additionally, these literature reviews do not deal with the risk of bias in either the primary research (e.g. poor-quality experimental design and conclusions that may not be supported by a given study) or the synthesis process (e.g. selective use of information). Meta-analysis approaches have become popular where significant amounts of quantitative data are available, but they are often biased in the way they select and include studies in their analysis (Koricheva & Gurevitch, Reference Koricheva and Gurevitch2014). In response to these problems, more rigorous methodologies, such as systematic reviews, have been developed. These were first used in the health sector through the work of the Cochrane Collaboration (Higgins & Green, Reference Higgins and Green2011), and have subsequently been applied to conservation and environmental management by the Collaboration for Environmental Evidence (Pullin & Knight, Reference Pullin and Knight2009; Collaboration for Environmental Evidence, 2018).

In this chapter we make a case for rigorous evidence synthesis: we explain why these methods are appropriate, how they can benefit wider society and how evidence can be synthesised, shared and used as a public good. Although evidence synthesis can inform a broad range of decision-making contexts, we focus here on two major aspects of conservation where evidence might be useful. First, in measuring the direct and indirect impacts of human activity on the natural world, and second, the effectiveness of conservation efforts to mitigate those impacts.

7.2 The central role of evidence synthesis in informing decisions in conservation policy and practice

Many factors can contribute to making a decision. In contexts where social and political stakes are high, as is common for conservation policy, scientific evidence will likely only inform decisions, rather than act as the primary driving force behind them. Although evidence is sometimes crucial, it may equally be ignored or overruled by other factors, such as political context, infrastructure and capacity. Ideally, evidence synthesis should play a central role in providing reliable evidence and enabling the wider society to understand or challenge decisions that might affect them. Making decisions without considering all available evidence might perpetuate biases, increase the likelihood of taking a wrong or costly action, or lead to missed opportunities to achieve faster or more cost-efficient outcomes. In a democratic society, comprehensive and rigorous evidence synthesis and open communication makes ‘sidelining’ (i.e. deliberately ignoring evidence) and/or biased (i.e. selective) use of evidence by authorities more difficult without challenge and transparent justification.

Unfortunately, evidence synthesis is itself often ‘bypassed’ completely or manipulated to get the answer required (i.e. policy-based evidence) (Dicks et al., Reference Dicks, Walsh and Sutherland2014). There may be significant resistance to the use of transparent evidence synthesis in the face of vested interests, and this may partly explain why organised and independent evidence synthesis receives so little attention or funding. Rigorous scientific evidence could also be seen as a threat to those with entrenched beliefs. Beyond outright opposition, complacency or inaccessibility of evidence might inhibit adoption of synthesis findings even when good intentions towards informed decision-making exist.

Fortunately, most decision-makers in conservation want practical advice that is grounded in the best available evidence (Cook et al., Reference Cook, Mascia and Schwartz2013). Leveraging syntheses and integrating their findings into decision-making processes requires an understanding of how and when evidence is necessary, and what level of confidence is needed to inform a decision. Such considerations will determine the choice of synthesis method(s), which should reflect practical needs to guide management decisions or future research. Syntheses can be used either to generate a new theory, conceptual framework or hypothesis (e.g. applying existing theory to a different context) or to test an existing hypothesis (e.g. evaluating the effectiveness of an intervention). In the context of effectiveness of interventions, evidence syntheses are relevant to decisions at several critical stage points in the life cycle of a programme or initiative: (1) initial scoping of a new topic early on in strategic planning (e.g. informing a new strategy on land use for a philanthropic foundation (Snilstviet et al., Reference Snilstviet, Stevenson and Villar2016)); (2) identification or validation of specific intervention designs (e.g. understanding how gender composition affects outcomes of resource management groups (Leisher et al., Reference Leisher, Temsah and Booker2016)); (3) benchmarking of institutional outcomes against other programmes (e.g. investments in community forest management by the Global Environment Facility (Bowler et al., Reference Bowler, Buyung-Ali and Healey2010)); (4) evaluation of overall effectiveness of an intervention across multiple contexts or applications (e.g. effects of property regimes in different biomes (Ojanen et al., Reference Ojanen, Zhou and Miller2017)). Understanding the purpose of the syntheses for informing the different stages of decision-making will ensure selection of a suitable method, appropriate engagement of stakeholders and relevant communication of findings.

Some evidence synthesis methods, such as systematic review, have been described as following the ‘information deficit model’ (Owens, Reference Owens2000); that is to say, they follow the assumption that the simple production and push delivery of evidence that fills a gap will be sufficient to achieve uptake. However, this perception misrepresents the full process behind the methodology. Systematic reviews can be socially inclusive, with extensive stakeholder contribution to formulating a question and approach, including setting the scope of the topic. This engagement attempts to ensure the findings of a review will fill a real and important synthesis gap (a knowledge need where sufficient primary research exists to allow synthesis) and respond to stakeholder demand. When engaging with stakeholders, a balance needs to be struck between involving them in the design of the review and independence from undue vested interest (Haddaway & Crowe, Reference Haddaway and Crowe2018). In the field of conservation, this balance is very much dependent on the nature of the question and the extent of vested interests (Kløcker Larsen & Nilsson, Reference Kløcker Larsen and Nilsson2017). Many aspects of evidence synthesis are collective, with stakeholders having shared motivation to benefit from the findings. In other cases, evidence synthesis is conducted in contested areas, with stakeholders that hold opposing views and may be hostile to the process and its findings. In the latter case, it is important to have a process that allows consultation when appropriate but also provides independence when necessary. For example, for some key steps, such as initial formulation of the question, engagement with stakeholders is usually essential (Land et al., Reference Land, Macura and Bernes2017), while other steps may need to be conducted free of such vested interests. To date, systematic reviews have engaged with a spectrum of stakeholders at different levels. Some reviews, for example those that are more academic or have specific commissioners (e.g. private goods reviews (Oliver & Dickson, Reference Oliver and Dickson2016)), have only passively engaged stakeholders by informing or consulting them (typically only at the beginning of the review process), while others have employed more in-depth engagement, extending to co-design of review methods and scope (Land et al., Reference Land, Macura and Bernes2017).

Alongside the purpose of syntheses, the level of confidence required to make a decision determines their method and scope. In some instances, where evidence of effectiveness is key, uncertainty in the evidence base hampers decision-making. In such instances one might ask ‘How much evidence is enough?’ or ‘How much uncertainty is acceptable?’ (Salafsky & Redford, Reference Salafsky and Redford2013). The need for evidence synthesis in the conservation sector may also vary depending on aspects of spatial scale, complexity and controversy. For example, decisions regarding inexpensive and low-risk local-scale interventions (e.g. applied to improve biodiversity or habitat conditions in nature reserves) may benefit most from locally generated, rigorous evidence, or more commonly from primary research studies conducted in similar contexts. This evidence could be provided by a single, self-generated study (as in adaptive management), be internally generated by the relevant organisation, or come from collating evidence from similar case studies. In contrast, decisions regarding expensive, often large-scale, high-risk programmes (e.g. to eradicate poaching and illegal trade in wildlife), where stakeholders are likely to be global and might hold conflicting views, may benefit from an independent global-scale, multi-context evidence synthesis. This might require a rigorous analysis of what works, where and when and for whom, involving analysis of heterogeneity in outcome and identification of effect modifiers. Often within conservation, a broader set of evidence types (e.g. controlled trials, case studies, quantitative and qualitative research) is needed to fully capture the complexity of conservation contexts.

7.3 Key aspects of rigorous evidence synthesis and why they are needed

To be reliable, evidence syntheses should consider all available evidence and attempt to provide the most accurate and precise estimation of the truth. A suite of methodologies has been developed that maximises transparency and repeatability while minimising subjectivity, susceptibility to bias or influence of vested interest. The most widespread of these, systematic reviews and systematic maps, are well-documented secondary research methods that follow detailed guidance (e.g. Collaboration for Environmental Evidence, 2018) and use step-wise processes set out in an a-priori protocol to comprehensively identify and collate all available evidence (Table 7.1).

Table 7.1 Overview of systematic evidence synthesis stages and the issues they address. For an explanation of bias see Collaboration for Environmental Evidence (2018) or Bayliss and Beyer (Reference Bayliss and Beyer2015)

Systematic review stageDescriptionDefining featuresType of issue addressed
Review question identification and formulation (with stakeholder engagement)Question is carefully identified and formulated with help of stakeholdersSocial acceptance, relevance and legitimacy of the review process
ProtocolProtocol outlines the intended method in detail. Protocol is peer-reviewed and published on an open-access platformPublic acceptance, peer reviewReview bias, question creep
Searching for relevant literatureComprehensive searches for grey and commercially published literature from a variety of sourcesComprehensiveness, repeatability (through transparency)Publication bias
Eligibility screeningCareful screening of all identified articles according to pre-determined inclusion criteriaConsistencySelection bias, review bias
Critical appraisal of study validity (optional for systematic maps)A detailed assessment of the susceptibility to bias and generalisability of each studyAccount for variability in internal validity and power of individual studiesSusceptibility to bias in individual studies and in study weighting by reviewers
Data coding and extractionTransparent coding and, in case of systematic reviews, extraction of study findingConsistency, repeatability (through transparency), minimising subjectivitySelection bias
Qualitative and/or qualitative data synthesis (not required for systematic maps)Well-documented and comprehensive synthesis of qualitative and/or quantitative study findingsComprehensiveness, repeatability (through transparency)Selection bias, vote-counting, publication bias
Reporting and communication of review findingsTransparent reporting of the review results with extensive supplementary informationRepeatability (through transparency), avoiding overreachDiscussion bias

Systematic reviews in conservation and environmental management have most commonly aimed to answer specific cause-and-effect type questions, for example relating to the effect of a management intervention or exposure on a subject of concern. (e.g. ‘What is the impact of a specific factor x on a subject z?’). In contrast, systematic maps collate and catalogue available evidence on a relatively broad subject, describing the nature of the evidence base and highlighting evidence clusters and gaps, along with methodological patterns in primary research (Collaboration for Environmental Evidence, 2018). Systematic maps can be used as an initial step of an evidence synthesis pathway to identify subtopics suitable for a systematic review and subtopics where there is insufficient evidence to make synthesis of primary data worthwhile. In such latter cases, which are common in conservation, the map may identify individual primary studies that provide useful evidence (for an example of a systematic review question generated from a map, see www.eviem.se/en/projects/SR15-Prescribed-forest-burning/).

Systematic reviews were originally developed in response to an absence of easily accessible and rigorous synthesis of available evidence. However, recent assessments have shown that non-systematic reviews that aim to inform environmental policy and practice are still prevalent, but have low methodological reliability, suffering from lack of transparency and methodological rigour, and are consequently highly susceptible to bias (Woodcock et al., Reference Woodcock, Pullin and Kaiser2014, Reference Woodcock, O’Leary and Kaiser2017; O’Leary et al., Reference O’Leary, Kvist and Bayliss2016). Moreover, the term ‘systematic review’ is often used by authors (and not challenged by editors or peer reviewers) when the reviews are in no way systematic. The production of substandard and ‘fake’ systematic reviews is increasing in all fields, from public health to environmental management and education (Haddaway et al., Reference Haddaway, Land and Macura2016; Ioannidis, Reference Ioannidis2016; Haddaway, Reference Haddaway2017; Pussegoda et al., Reference Pussegoda, Turner and Garritty2017); they are ‘fake’ in the sense that they lack necessary comprehensiveness, transparency and reliability (Haddaway, Reference Haddaway2017). This further confuses the issue for potential readers, with only a handful of environmental journals requiring authors to follow accepted standards of conduct and reporting (see Collaboration for Environmental Evidence, 2018). A potential evidence user can use keywords like ‘systematic review’ in their search and have it return documents that claim to be such, when in fact they are not. The misuse of the term ‘systematic review’ can undermine efforts towards effective decision-making and is a key reason for establishing independent standards.

Stakeholders, including scientists, rarely have the time or training to differentiate between a ‘true’ systematic review and one that misses critical components of the method (resulting in increased risk of bias and lack of transparency) especially when published in an outlet such as a peer-reviewed journal. To enhance the uptake of more rigorous and reliable synthesis methodologies and maximise the potential of evidence to inform decisions, independent coordinating bodies have been founded in different sectors of society to provide guidelines and standards for evidence synthesis. In the field of medicine this process began in the 1990s with the establishment of the Cochrane Collaboration, which aimed to conduct systematic reviews in order to provide healthcare professionals with the best available evidence on the effectiveness of clinical interventions (Higgins & Green, Reference Higgins and Green2011). The methods were transferred to the field of conservation and environmental management in the early 2000s (Pullin & Stewart, Reference Pullin and Stewart2006) and are now under the coordination of the Collaboration for Environmental Evidence. These independent coordinating bodies provide guidelines for and training in the conduct of systematic reviews and systematic maps, as well as registering, endorsing and publishing such evidence syntheses. Syntheses registered through the coordinating bodies are scrutinised by methodology experts, guaranteeing a level of reliability and rigour (Collaboration for Environmental Evidence, 2018).

In circumstances where vested interests might potentially influence the outcome of an evidence synthesis, these independent organisations provide a framework and platform to assist the review team to achieve and demonstrate independence of the synthesis process. The framework allows for full engagement of commissioners and other stakeholders in formulation of the review question and planning of the review protocol, followed by independent peer review and publication of the protocol prior to the conduct of the review. In cases where conflict or the risk of undue influence from particular stakeholders is high, the review process should be conducted by an independent review team and the report submitted for independent peer review. Following this process, the review findings may be endorsed by the independent organisation.

7.4 New developments that address barriers to evidence synthesis and communication

There are persistent barriers to the conduct of environmental evidence syntheses and communication of their findings. First, the high resource costs required have been a major disincentive to producing high-quality syntheses, despite their critical value for effective conservation. Second, efficient and effective means of communicating results and facilitating their use for real-life decision-making scenarios are haphazardly applied. These barriers limit the ability of evidence synthesis to dynamically and adaptively respond to conservation challenges. However, new developments in big data and deep learning approaches are offering exciting opportunities to harness evidence syntheses and promote them to broader audiences.

Conducting rigorous evidence syntheses, such as systematic reviews, can carry both significant monetary and human resource costs (Dicks et al., Reference Dicks, Walsh and Sutherland2014). These costs are particularly prohibitive for organisations with critical needs for evidence, but who have limited time and resources to engage in such synthesis efforts or even to glean needed information from lengthy synthesis reports (Elliott et al., Reference Elliott, Turner and Clavisi2014). Moreover, high costs make updating syntheses to create a dynamic evidence base with the most up-to-date knowledge effectively impossible using current technology (Garritty et al., Reference Garritty, Tsertsvadze and Tricco2010). Additionally, the window of opportunity for decision-making may be shorter than the time in which a credible synthesis can be completed. Thus, to be useful to conservation, evidence syntheses must be optimised to efficiently find, collate and communicate existing evidence (Boyack & Klavans, Reference Boyack and Klavans2014).

In a policy space where decision-making timelines are short and demands for rigorous, reliable evidence are high, methods assisted by advances in computing can support rapid evidence collation as well as increase cost efficiency (Shemilt et al., Reference Shemilt, Khan and Park2016). Computer-assisted approaches range from tools that manage data and streamline the synthesis process to tools powered by machine learning algorithms that allow rapid screening and extraction of evidence with reduced human intervention (Kohl et al., Reference Kohl, McIntosh and Unger2018). Promising computer-assisted approaches, including automatic term recognition, document clustering, automatic document classification and document summarisation (Frantzi et al., Reference Frantzi, Ananiadou and Mima2000; O’Mara-Eves et al., Reference O’Mara-Eves, Thomas and McNaught2015) have been trialled in medical and health topics (Ananiadou et al., Reference Ananiadou, Rea and Okazaki2009) and are beginning to be tested in ecological topics (Westgate et al., Reference Westgate, Barton and Pierson2015; Grubert & Siders, Reference Grubert and Siders2016; Roll et al., Reference Roll, Correia and Berger-Tal2018).

These developments are encouraging for increased efficiency of the synthesis processes and potentially enabling dynamic syntheses that continuously update with new evidence as it becomes available. However, there are certain caveats and limitations that must be considered prior to widespread employment of computer-assisted tools. First, unlike medicine and fields such as economics, the semantics of conservation are highly heterogeneous and non-standardised (Westgate & Lindenmayer, Reference Westgate and Lindenmayer2017), posing difficulties for both efficient and comprehensive searching, and reliable application of machine learning algorithms to sort and mine text for desired patterns. Second, thus far, the performance of these approaches remains largely untested empirically, particular for conservation and environmental topics. As the value of evidence synthesis methods is in their transparency and credibility, reliable data on the efficacy of different computer-assisted approaches are important for uptake and expansion. Third, many existing computer-assisted platforms are fee-based or require programming skills, limiting their utility to a broader field of users. To improve global ability to address pervasive environmental threats, we need to democratise access to the tools that can help decision-making worldwide, not solely in countries or among researchers with means.

7.5 Mainstreaming evidence synthesis for decision support

Efforts to engage in open science and collaborative practice between conservation and technology fields will require forming collaborative partnerships and fostering conversation between evidence producers, evidence users and data scientists, to build a cohesive and engaged community of practice to open channels of communication to all users (Joppa, Reference Joppa2015). This will allow the broader community to use existing efforts as a starting point and avoid reinventing the wheel and wasting already limited resources (Lowndes et al., Reference Lowndes, Best and Scarborough2017). Furthermore, collaborative partnerships and creative funding can foster the long-term sustainability of tools that can live on to serve users. Too often, tools and platforms are created in good faith but require maintenance and updating and lack the ongoing funding and personnel to do so. This is particularly important as tools are most useful when they can dynamically respond to user needs and emerging technologies. This is a critical stepping stone for breaking down barriers to understanding and using evidence synthesis methodologies, as without a dynamic toolbox, synthesis methods will reman aloof from the needs of a diversifying and widening audience.

Evidence synthesis conducted to Collaboration for Environmental Evidence standards generates systematic reviews and systematic maps that are theoretically accessible to all. Yet, simply because something is available does not mean that the potential user is aware of it, knows where to find it, or even how to make sense of it. This is particularly the case for those new to the concept of evidence synthesis. Indeed, many practitioners and policy-makers rely on past experience or consult colleagues, rather than make use of the full suite of evidence (Pullin et al., Reference Pullin, Knight and Stone2004; Young et al., Reference Young, Corriveau and Nguyen2016). These issues create a number of inherent challenges for those decision-makers seeking to be evidence-informed and also broader potential audiences, such as stakeholders and wider society.

One of the mantras of science communication is ‘know your audience’ (Wilson et al., Reference Wilson, Ramey and Donaldson2016; Cooke et al., Reference Cooke, Gallagher and Sopinka2017) and to have impact, the findings of an evidence synthesis need to be effectively tailored and communicated to different groups of people in different ways and through different media. Communication efforts should, for example, be sensitive to the fact that different groups vary in their ‘trust’ of the science they encounter from different sources (e.g. academic journals, colleagues, social media) (Wilson et al., Reference Wilson, Ramey and Donaldson2016; Cooke et al., Reference Cooke, Gallagher and Sopinka2017).

A study that surveyed the willingness of practitioners to use a synopsis of relevant literature on bird conservation found that participants were more likely to use the evidence to inform decisions if it was easily accessible and in a clearly summarised format (Walsh et al., Reference Walsh, Dicks and Sutherland2014). Similar summaries are needed to complement evidence syntheses. These summaries may then need to be further refined and transformed into policy briefs. Policy briefs are often written through the cultural lens of a given organisation and a given issue, meaning that these are unlikely to be useful if prepared in a generic format. Sundin et al. (Reference Sundin, Andersson and Watt2018) recently proposed the use of storytelling as a tool to effectively communicate the results of evidence syntheses. This method could give meaning to the evidence and can be communicated through videos (e.g. see https://youtu.be/4uPowxn2skg), presentations or public forums (e.g. newspapers, magazines). Nevertheless, uptake of these methods in science communication is generally slow and also could still rely on poorly conducted syntheses (McKinnon et al., Reference McKinnon, Cheng and Dicks2018).

There has also been a rise in various knowledge management platforms and data-visualisation tools to explore underlying data that support evidence synthesis (e.g. www.3ieimpact.org/en/evaluation/evidence-gap-maps/, or www.cedar.iph.cam.ac.uk/resources/evidence/). These platforms present data from synthesis projects using interactive features and intuitive visualisations. For example, the Evidence for Nature and People Data Portal (www.natureandpeopleevidence.org) allows users to filter data according to desired parameters – such as diving into a data set to examine a specific intervention or outcome or geographic region, and visualising resultant trends. Syntheses, and in particular systematic maps, can be multi-layered and complex, precipitating a need for an interface that is graphical and intuitive, allowing a broader audience to use it (Figure 7.1).

Figure 7.1 An example of an evidence ‘heat map’ linking conservation interventions with human well-being outcomes. The map allows the user to assess the evidence base for gaps and gluts as well as clicking on each box to further examine the relevant studies.

If reported responsibly, these platforms and visualisations can play an important role in how stakeholders access evidence. A challenge for these approaches is to communicate that evidence syntheses are only estimates of the truth, which depend on the reliability of the evidence with which they were made. There is potential for evidence to be misinterpreted if the relative weight or reliability of a given element is misconstrued when visualised. Regardless of the output, it is important that authors of evidence syntheses communicate any uncertainty in the evidence and the risks associated with relying on studies that have high risk of bias.

Although it is laudable to communicate the findings of a topical evidence synthesis, additional efforts are also needed to communicate to practitioners the value of systematic reviews or maps, how they differ from other evidence synthesis methods and how they can be integrated with existing science advice and decision-making processes within different regions or institutions. Writing academic papers and delivering presentations at scientific conferences is unlikely to reach the typical practitioner, so creative approaches to outreach are needed to access and inform them.

Without use of rigorous evidence synthesis, policies and practice claiming to be ‘evidence-informed’ can be meaningless. For conservation and the environmental sector in general, the value of evidence synthesis has yet to be fully realised and we have the feeling that its time is yet to come. However, the recent methodological developments, awareness-raising and capacity development, together with new technologies for faster and more efficient conduct, suggest this time is not far away. Conservation is an interdisciplinary field and cannot remain for long in a state of relative evidence synthesis deficit in comparison with other sectors with which it seeks to be relevant. Although still marginalised, the methodology and infrastructure to build conservation’s evidence base through rigorous synthesis now exist at a global level. A commitment to evidence-informed decision-making that recognises the central role of rigorous evidence synthesis is required by key actors in the sector if these potential benefits are to be achieved.

References
Ananiadou, S. S., Rea, B. B., Okazaki, N. N., et al. 2009. Supporting systematic reviews using text mining. Social Science Computer Review, 27, 509523. https://doi.org/10.1177/0894439309332293
Bayliss, H. R. & Beyer, F. R. 2015. Information retrieval for ecological syntheses. Research Synthesis Methods, 6, 136148.
Bowler, D., Buyung-Ali, L., Healey, J. R., et al. 2010. The Evidence Base for Community Forest Management as a Mechanism for Supplying Global Environmental Benefits and Improving Local Welfare: A STAP Advisory Document. Washington, DC: Scientific and Technical Advisory Panel. Global Environment Facility.
Boyack, K. W. & Klavans, R. 2014. Creation of a highly detailed, dynamic, global model and map of science. Journal of The Association for Information Science and Technology, 65, 670685. https://doi.org/10.1002/asi.22990
Collaboration for Environmental Evidence. 2018. Guidelines and Standards for Evidence Synthesis in Environmental Management. Version 5.0. Available from www.environmentalevidence.org/information-for-authors (accessed 8 March 2018).
Cook, C. N., Mascia, M. B., Schwartz, M. W., et al. 2013. Achieving conservation science that bridges the knowledge–action boundary. Conservation Biology, 27, 669678. https://doi.org/10.1111/cobi.12050
Cooke, S. J., Gallagher, A. J., Sopinka, N. M., et al. 2017. Considerations for effective science communication. FACETS, 2, 233248. https://doi.org/10.1139/facets-2016–0055
Dicks, L. V., Walsh, J. C. & Sutherland, W. J. 2014. Organising evidence for environmental management decisions: a ‘4S’ hierarchy. Trends in Ecology and Evolution, 29, 607613. https://doi.org/10.1016/j.tree.2014.09.004
Elliott, J. H., Turner, T., Clavisi, O., et al. 2014. Living systematic reviews: an emerging opportunity to narrow the evidence–practice gap. PLoS Medicine, 11, 16. https://doi.org/10.1371/journal.pmed.1001603
Frantzi, K., Ananiadou, S. & Mima, H. 2000. Automatic recognition of multi-word terms: the C-value/NC-value method. International Journal on Digital Libraries, 3, 115130. https://doi.org/10.1007/s007999900023
Garritty, C., Tsertsvadze, A., Tricco, A. C., et al. 2010. Updating systematic reviews: an international survey. PLoS ONE, 5, e9914. https://doi.org/10.1371/journal.pone.0009914
Grubert, E. & Siders, A. 2016. Benefits and applications of interdisciplinary digital tools for environmental meta-reviews and analyses. Environmental Research Letters, 11, 093001. https://doi.org/10.1088/1748–9326/11/9/093001
Haddaway, N. R., Land, M. & Macura, B. 2016. ‘A little learning is a dangerous thing’: a call for better understanding of the term ‘systematic review’. Environment International, 99, 356360. https://doi.org/10.1016/j.envint.2016.12.020
Haddaway, N. R. 2017. Response to ‘Collating science-based evidence to inform public opinion on the environmental effects of marine drilling platforms in the Mediterranean Sea’. Journal of Environmental Management, 203, 612614. https://doi.org/10.1016/j.jenvman.2017.03.043
Haddaway, N. R. & Crowe, S. 2018. Experiences and lessons in stakeholder engagement in environmental evidence synthesis: a truly special series. Environmental Evidence, 7, art11.
Higgins, J. & Green, S. 2011. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1. Cochrane Collaboration. Available from http://handbook-5–1.cochrane.org (accessed 8 March 2018).
Ioannidis, J. P. A. 2016. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. The Milbank Quarterly, 94, 485514. https://doi.org/10.1111/1468–0009.12210
Joppa, L. N. 2015. Technology for nature conservation: an industry perspective. Ambio, 44, 522526. https://doi.org/10.1007/s13280-015–0702-4
Kløcker Larsen, R. & Nilsson, A.E. 2017. Knowledge production and environmental conflict: managing systematic reviews and maps for constructive outcome. Environmental Evidence, 6. https://doi.org/10.1186/s13750-017–0095-x
Kohl, C., McIntosh, E. J., Unger, S., et al. 2018. Online tools supporting the conduct and reporting of systematic reviews and systematic maps: a case study on CADIMA and review of existing tools. Environmental Evidence, 7(8). https://doi.org/10.1186/s13750-018–0115-5
Koricheva, J. & Gurevitch, J. 2014. Uses and misuses of meta-analysis in plant ecology. Journal of Ecology, 102, 828844. https://doi.org/10.1111/1365–2745.12224
Land, M., Macura, B., Bernes, C., et al. 2017. A five-step approach for stakeholder engagement in prioritisation and planning of environmental evidence syntheses. Environmental Evidence, 6(25). https://doi.org/10.1186/s13750-017–0104-0
Leisher, C., Temsah, G., Booker F., et al. 2016. Does the gender composition of forest and fishery management groups affect resource governance and conservation outcomes? A systematic map. Environmental Evidence, 5(6). https://doi.org/10.1186/s13750-016–0057-8
Li, W. & Zhao, Y. 2015. Bibliometric analysis of global environmental assessment research in a 20-year period. Environmental Impact Assessment Review, 50, 158166. https://doi.org/10.1016/j.eiar.2014.09.012
Lowndes, J. S. S., Best, B. D., Scarborough, C., et al. 2017. Our path to better science in less time using open data science tools. Nature Ecology and Evolution, 1, 160. https://doi.org/10.1038/s41559-017–0160
McKinnon, M. C., Cheng, S. H., Dupre, S., et al. 2016. What are the effects of nature conservation on human well-being? A systematic map of empirical evidence from developing countries. Environmental Evidence, 5(8).
McKinnon, M. C., Cheng, S., Dicks, L., et al. 2018 Seek higher standards to honestly assess conservation effectiveness. Mongabay. Available online on 17 April 2018.
O’Leary, B. C., Kvist, K., Bayliss, H. R., et al. 2016. The reliability of evidence reviews in environmental science and conservation. Environmental Science Policy, 64, 7582. https://doi.org/10.1016/j.envsci.2016.06.012
O’Mara-Eves, A., Thomas, J., McNaught, J., et al. 2015. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Systematic Reviews, 4(5). https://doi.org/10.1186/2046–4053-4–5
Ojanen, M., Zhou, W., Miller, D. C., et al. 2017. What are the environmental impacts of property rights regimes in forests, fisheries and rangelands? Environmental Evidence, 6(12). https://doi.org/10.1186/s13750-017–0090-2
Oliver, S. & Dickson, K. 2016. Policy-relevant systematic reviews to strengthen health systems: models and mechanisms to support their production. Evidence and Policy, 12(2), 235259. https://doi.org/10.1332/174426415X14399963605641
Owens, S. 2000. ‘Engaging the public’: information and deliberation in environmental policy. Environment and Planning A, 32, 11411148. https://doi.org/10.1068/a3330
Plume, A. & van Weijen, D. 2014. Publish or perish? The rise of the fractional author …. Research Trends, 38. www.researchtrends.com/issue-38-september-2014/publish-or-perish-the-rise-of-the-fractional-author/ (accessed 12 December 2019).
Pullin, A. S., Knight, T. M., Stone, D. A., et al. 2004. Do conservation managers use scientific evidence to support their decision-making? Biological Conservation, 119, 245252. https://doi.org/10.1016/j.biocon.2003.11.007
Pullin, A. S. & Stewart, G. B. 2006. Guidelines for systematic review in conservation and environmental management. Conservation Biology, 20, 16471656. https://doi.org/10.1111/j.1523–1739.2006.00485.x
Pullin, A. S. & Knight, T. M. 2009. Doing more good than harm: building an evidence-base for conservation and environmental management. Biological Conservation, 142, 931934. https://doi.org/10.1016/j.biocon.2009.01.010
Pussegoda, K., Turner, L., Garritty, C., et al. 2017. Systematic review adherence to methodological or reporting quality. Systematic Reviews, 6(131). https://doi.org/10.1186/s13643-017–0527-2
Roll, U., Correia, R. A. & Berger-Tal, O. 2018. Using machine learning to disentangle homonyms in large text corpora. Conservation Biology, 32, 716724. https://doi.org/10.1111/cobi.13044
Salafsky, N. & Redford, K. 2013. Defining the burden of proof in conservation. Biological Conservation, 166, 247253. https://doi.org/10.1016/j.biocon.2013.07.002
Shemilt, I., Khan, N., Park, S., et al. 2016. Use of cost-effectiveness analysis to compare the efficiency of study identification methods in systematic reviews. Systematic Reviews, 5, 140. https://doi.org/10.1186/s13643-016–0315-4
Snilstviet, B., Stevenson, J., Villar, P. F., et al. 2016. Land-use change and forestry programmes: evidence on the effects on greenhouse gas emissions and food security. www.3ieimpact.org/media/filer_public/2016/11/17/egm3-landuse-forest.pdf (accessed 8 March 2018).
Sundin, A., Andersson, K. & Watt, R. 2018. Rethinking communication: integrating storytelling for increased stakeholder engagement in environmental evidence synthesis. Environmental Evidence, 7(6). https://doi.org/10.1186/s13750-018–0116-4
Walsh, J. C., Dicks, L. V. & Sutherland, W. J. 2014. The effect of scientific evidence on conservation practitioners’ management decisions. Conservation Biology, 29, 8898. https://doi.org/10.1111/cobi.12370
Ware, M. & Mabe, M. 2015. An overview of scientific and scholarly journal publishing. International Association of Scientific, Technical and Medical Publishers, www.stm-assoc.org/2015_02_20_STM_Report_2015.pdf (accessed 8 March 2018).
Westgate, M. J., Barton, P. S, Pierson, J. C., et al. 2015. Text analysis tools for identification of emerging topics and research gaps in conservation science. Conservation Biology, 29, 16061614. https://doi.org/10.1111/cobi.12605
Westgate, M. J. & Lindenmayer, D. B. 2017. The difficulties of systematic reviews. Conservation Biology, 31, 10021007. https://doi.org/10.1111/cobi.12890
Wilson, M. J., Ramey, T. L., Donaldson, M. R., et al. 2016. Communicating science: sending the right message to the right audience. FACETS, 1(1), 127137. https://doi.org/10.1139/facets-2016–0015
Woodcock, P., Pullin, A. S. & Kaiser, M. J. 2014. Evaluating and improving the reliability of evidence in conservation and environmental science: a methodology. Biological Conservation, 176, 5462. https://doi.org/10.1016/j.biocon.2014.04.020
Woodcock, P., O’Leary, B. C., Kaiser, M. J., et al. 2017. Your evidence or mine? Systematic evaluation of reviews of marine protected area effectiveness. Fish and Fisheries, 18, 668–81. https://doi.org/10.1111/faf.12196
Young, N., Corriveau, M., Nguyen, V. M., et al. 2016. How do potential knowledge users evaluate new claims about a contested resource? Problems of power and politics in knowledge exchange and mobilization. Journal of Environmental Management, 184, 380388. https://doi.org/10.1016/j.jenvman.2016.10.006

Chapter Eight Aligning evidence for use in decisions: mechanisms to link collated evidence to the needs of policy-makers and practitioners

Lynn V. Dicks
University of Cambridge
Barbara Livoreil
Fondation pour la recherche sur la biodiversité
Rebecca K. Smith
University of Cambridge
Heidi Wittmer
Helmholtz Centre for Environmental Research – UFZ
and Juliette Young
Centre for Ecology and Hydrology and AgroSup Dijon
8.1 Introduction

We should not be surprised by the scale of the challenge when trying to link a body of scientific knowledge to the complex, shifting and seemingly unpredictable world of policy, or to the massively decentralised, globally distributed world of conservation practice (Young et al., Reference Young, Waylen and Sarkki2014). One side of the challenge is developing a consensual understanding of the science itself. By nature, scientific knowledge is continually progressing, with theories, empirical data and new interpretations emerging all the time. Even within a single discipline, it can be hard to convey what is known at a particular point in time, and this often involves presenting different scientific viewpoints. For instance, there is substantial variation around the world in public health advice regarding alcohol consumption, with ‘safe’ limits in the UK being 50% of those in the USA (Wood et al., Reference Wood, Kaptoge and Butterworth2018). In conservation, the challenge is even greater, as relevant research cuts across the natural, physical and social sciences.

The other side of the challenge is working out how, and when, to offer relevant scientific knowledge to decision-makers, in order to have the greatest impact on the decisions being made. This is the focus of our chapter. We argue that it is a question of correct alignment: of selecting the right knowledge to address the needs of decision-makers, ensuring that knowledge is accessible to them, and articulating it within their decision-making processes.

First, we consider how well current efforts to synthesise evidence in conservation align with the needs of decision-makers. Then we describe three mechanisms that might be used to enhance the alignment of available knowledge with decision-making, starting at small local scales and moving to the global scale: decision support tools, active knowledge exchange and large-scale scientific assessments. For each mechanism, we provide examples and draw out general guidelines regarding the circumstances in which it is likely to be most effective.

8.2 How well do current evidence synthesis activities align with policy and practice needs?

When scientific evidence is needed for decision-making, the process of obtaining and analysing the evidence is often demand-led. An organisation faced with a difficult management or policy decision will undertake or commission a review to answer a specific question. For example, the UK Government Department of Environment, Food and Rural Affairs (Defra) commissioned a review of evidence on the status of pollinators (Vanbergen et al., Reference Vanbergen, Heard and Breeze2014) before designing the National Pollinator Strategy for England (Defra, 2014). When this happens, the evidence synthesis is well-aligned with the policy and practice needs, summarising relevant material that can be found in the time available. However, it also puts immense time pressure on the evidence synthesis process, because decision-making can only happen once the evidence has been reviewed. This tends to lead to the selection of evidence synthesis methods such as rapid evidence assessments, traditional non-systematic literature reviews and expert consultations, which are not the most rigorous or unbiased approaches available (Dicks et al., Reference Dicks, Haddaway and Hernández-Morcillo2017).

The Collaboration for Environmental Evidence (www.environmentalevidence.org) and the Conservation Evidence project (www.conservationevidence.com) aim to address the needs of conservation practitioners and policy-makers with more rigorous methods of knowledge synthesis, namely systematic reviews, systematic maps (Collaboration for Environmental Evidence, 2013; see also Chapter 7) and subject-wide evidence syntheses (Sutherland et al., Reference Sutherland, Taylor and MacFarlane2019b; see also Chapter 4). They do so by actively involving stakeholders in the selection of topics to synthesise and the collation and subsequent evaluation of the evidence found (Dicks et al., Reference Dicks, Wright and Ashpole2016; Haddaway et al., Reference Haddaway, Kohl and Rebelo da Silva2017).

To evaluate the overall success of this alignment effort, we recently asked how well evidence collated by the Conservation Evidence project on the subject of sustainable food production matched the priority knowledge needs of decision-makers. Five independent exercises (Pretty et al., Reference Pretty, Sutherland and Ashby2010; Dicks et al., Reference Dicks, Abrahams and Atkinson2013a, Reference Dicks, Bardgett and Bell2013b; Ingram et al., Reference Ingram, Wright and Foster2013; Jones et al., Reference Jones, Mead and Kaiser2014), involving 240 people from across business, practice, policy-making and academia, had generated 286 priority questions faced by decision-makers. We sorted these into five categories, following the Driver–Pressure–State–Impact–Response (DPSIR) framework (Maxim et al., Reference Maxim, Spangenberg and O’Connor2009). This conceptual framework describes interactions between society and the environment in a way that is meaningful for policy. Social and economic developments (Driving Forces, D) exert Pressures (P) on the environment and, as a consequence, the State (S) of the environment changes. This leads to Impacts (I) on ecosystems, human health and society, which may elicit a societal Response (R) that feeds back on D, S or I. We added a category for questions about underlying science that did not fit the DPSIR categories (Figure 8.1).

Figure 8.1 Categorisation of 286 priority questions identified by stakeholders as relevant to sustainable food production (Pretty et al., Reference Pretty, Sutherland and Ashby2010; Dicks et al., Reference Dicks, Abrahams and Atkinson2013a, Reference Dicks, Bardgett and Bell2013b; Ingram et al., Reference Ingram, Wright and Foster2013; Jones et al., Reference Jones, Mead and Kaiser2014) according to the Driver–Pressure–State–Impact–Response framework. Examples of questions are provided for each category. The extracted segment represents questions already answered by evidence summaries provided by the Conservation Evidence project.

Of all the priority questions, 189 (66%) were about responses (R), which are the focus of the Conservation Evidence project. Evidence had already been summarised that could help answer 35 of these questions (12% overall; Smith et al., Reference Smith, Dicks and Sutherland2015; Sutherland et al., Reference Sutherland, Dicks and Ockendon2019a).

In a similar vein, Cook et al. (Reference Cook, Possingham and Fuller2013a) investigated the contribution of systematic reviews to conservation decision-making, finding that 35% of the 43 reviews considered practical on-the-ground management, while most addressed interventions relevant to policy. Cook et al. (Reference Cook, Possingham and Fuller2013a) argued that the benefits for conservation could be significantly enhanced by increasing the number of systematic reviews focused on questions of direct management relevance.

These two analyses show there is some alignment between high-quality evidence synthesis methods and the needs of conservation practitioners and policy-makers, but it could be improved. Below, we provide a series of examples of mechanisms to enhance this alignment at a range of scales.

8.3 Decision support systems

Decision support systems are tools designed to assist decision-makers, for example, by visually or numerically illustrating different possible outcomes to a question, or leading users through logical decision steps (Dicks et al., Reference Dicks, Walsh and Sutherland2014). Often software-based, they represent a link between relevant science and decision-making (Dicks et al., Reference Dicks, Walsh and Sutherland2014; Figure 8.2). Decision support systems are useful for incorporating evidence into decisions related to a specific question that has been widely and repeatedly addressed. It is also important that the evidence can be converted into simple numerical or visual formats.

Figure 8.2 A schematic showing how scientific information could support environmental decision-making (Dicks et al., Reference Dicks, Walsh and Sutherland2014). The triangle on the left shows an evidence hierarchy, in which summaries, such as those produced by the Conservation Evidence project, integrate evidence from across studies and systematic reviews, and form the basis for information flowing into decision support systems. In these circumstances, environmental decisions (shown by the ‘Decision’ diamond on the right) are based on the best-available evidence, combined with the expertise and local knowledge of the practitioner or policy-maker (described by the ‘Experience’ box). Dashed lines illustrate bypass routes currently taken to inform environmental decisions.

There are many decision support tools available covering various aspects of environmental science. For instance, Zasada et al. (Reference Zasada, Piorr and Novo2017) identified 60 research projects funded between 2002 and 2013 under the European Commission’s 6th and 7th Framework Programmes that had developed decision support tools for landscape and environmental management. Of these, only 61% still existed in 2014, and only half were updated after the projects that developed them ended, although this seems a pre-requisite for ongoing use. The uptake of decision support systems depends on a range of factors, including ease of use, performance, whether they are recommended by peers and the level of marketing (Rose et al., Reference Rose, Sutherland and Parker2016). Uptake can be enhanced by ensuring that users are closely involved in the conception and design of the tools (Rose et al., Reference Rose, Parker and Fodey2018).

While decision support systems are often designed by researchers as a way of incorporating scientific knowledge into practice, most are based on one particular model, study or approach to a scientific question and represent a ‘bypass’ of the evidence hierarchy (Figure 8.2 and see Dicks et al., Reference Dicks, Walsh and Sutherland2014). There are only a few examples where they represent the best-available scientific knowledge, based on rigorous synthesis of evidence.

One such decision support tool is the online biodiversity metric incorporated into the Cool Farm Tool (available at www.coolfarmtool.org), which provides scores for the likely benefits for biodiversity of a range of farm management actions. The actions that are included are selected according to a combination of expert judgement and assessments of summarised evidence conducted by the Conservation Evidence project. Each farm management action is assigned scores reflecting the benefit for overall biodiversity, and also for 11 species groups (e.g. woodland birds, beneficial invertebrates), weighted according to the evidence. Actions that are strongly supported by the evidence provided by the Conservation Evidence syntheses (Sutherland et al., Reference Sutherland, Dicks and Ockendon2019a) are scored more highly than those for which effectiveness is not known.

Another example is the set of greenhouse gas emission calculators used in agriculture to support mitigation by changing farm management. These tools incorporate models of greenhouse gas emissions and carbon storage according to vegetation type and farming practice (Richards et al., Reference Richards, Metzel and Chirinda2016). These calculators combine empirical models with emission factors collated by the Intergovernmental Panel on Climate Change (see ‘National and International Scientific Assessments’). Although the outputs from these tools are only as good as the data that they are based on, new information can be added to improve their performance as it becomes available. For example, Richards et al. (Reference Richards, Metzel and Chirinda2016) demonstrated that two widely used software tools tend to overestimate emissions from smallholder farms in tropical environments, but suggest that this is probably due to a systematic bias in literature, with most data coming from temperate regions, rather than bias in the models themselves. As empirical data are included from a wider range of environments, more accurate disaggregated emissions factors will become available for different parts of the world. If the decision support systems are maintained and updated, this new knowledge will directly influence decision-making at farm level.

8.4 Active knowledge exchange mechanisms

Active knowledge exchange mechanisms are the most diverse alignment mechanism of the three considered in this chapter. Our concept is similar to that of ‘boundary organisations’ identified by some other authors (Guston, Reference Guston2001; Cook et al., Reference Cook, Mascia and Schwartz2013b), in that they operate in both scientific and practical spheres, but retain distinct lines of accountability to both groups. They can take a variety of institutional forms, from a dedicated, self-funded or government-funded organisation to a network of people working together across organisations (see also Chapter 13).

The reputation of such a body depends on its ability to produce or broker knowledge that is salient, credible and legitimate (Cash et al., Reference Cash, Clark and Alcock2003; Sarkki et al., Reference Sarkki, Tinch and Niemelä2015) while maintaining transparency. Credibility refers to the scientific adequacy of the technical evidence and arguments. Salience is the relevance of the brokered knowledge to the needs of decision-makers. Legitimacy reflects the perceptions that the production of information has been respectful of stakeholders’ divergent values and beliefs, unbiased in its conduct and fair in its treatment of views and interests. Achieving all these values requires adequate attention to governance from the outset.

Here, we provide examples of knowledge exchange mechanisms operating at a subnational scale, related to a particular environmental issue or landscape (Wadden Sea case study); at a national or international scale but restricted to environmental science (EKLIPSE mechanism); and at a national or international scale ranging across all scientific knowledge (European Scientific Advice Mechanism, and UK Parliamentary Office of Science and Technology).

8.4.1 Management of the Wadden Sea

At a subnational scale, van Enst et al. (Reference van Enst, Runhaar and Driessen2016) provided a detailed case study of three contrasting knowledge exchange mechanisms that have been important in aligning scientific evidence with policy and management decisions around the Wadden Sea, a shallow estuarine sea in the Netherlands. Competing cockle-fishing, gas extraction and biodiversity conservation interests generate continuous debate over the scientific knowledge, and the strategic use or misuse of such knowledge has played a pivotal role in disputes (Floor et al., Reference Floor, van Koppen and Lindeboom2013). Knowledge exchange mechanisms were devised to improve the transparent use of evidence. Two of the knowledge exchange mechanisms were government-funded: the Wadden Academy, a science-led organisation that oversees monitoring and data-gathering, and the Netherlands Commission for Environmental Assessment, which produces official reports. The third, IMSA Amsterdam, is a commercial think-tank and consultancy, focused on mediating between stakeholders, science and policy. These three organisations worked together to improve the salience, credibility and legitimacy of the scientific knowledge that was available, allowing it to be influential in decision-making related to the cockle-fishery and gas-exploitation controversies. Their efforts ultimately reduced conflict and improved environmental outcomes for the Wadden Sea, for example by enabling more sustainable fishing methods to be adopted (van der Molen et al., Reference van der Molen, Puente-Rodríguez and Swart2015; van der Molen, Reference van der Molen2018).

8.4.2 The EKLIPSE mechanism

Knowledge exchange mechanisms focused on one environmental issue can develop deep, long-term relationships between a core set of stakeholders and researchers. When operating across many different issues at national or international scale, relationships with experts and other stakeholders are generally short-term and must continually be re-established as the topic of interest to policy changes. One possible approach to this is provided by the EKLIPSE mechanism (Watt et al., Reference Watt, Ainsworth and Balian2018; www.eklipse-mechanism.eu), which engages relevant actors from science, policy and society to identify evidence relevant to European policy. EKLIPSE accepts requests for knowledge synthesis on specific issues from policy-makers and other societal actors. A wide network of knowledge-holders can respond to the request, often through the formation of an expert working group (Wyborn et al., Reference Wyborn, Louder and Harrison2018). To give an example, the European Commission requested scientific knowledge on how to evaluate nature-based solutions (solutions inspired and supported by nature) for their ability to enhance sustainability in cities. In response, EKLIPSE convened a pan-European expert group to conduct a rapid evidence assessment and build a framework for evaluating the costs and benefits of nature-based solutions. This was disseminated as a policy report and an open-access scientific paper (Raymond et al., Reference Raymond, Frantzeskaki and Kabisch2017).

8.4.3 The European Scientific Advice Mechanism and UK Parliamentary Office of Science and Technology

At a larger scale, knowledge exchange mechanisms can provide an interface between science and policy across all scientific issues. Usually these are national or international, such as the UK Parliamentary Office for Science and Technology (POST; Norton, Reference Norton1997) and the European Union Scientific Advice Mechanism (ec.europa.eu/research/sam/index.cfm). At this level, knowledge exchange mechanisms have tended to settle on one particular way of doing things that works. At the POST, for instance, a Board selects subjects for briefing notes, known as POSTnotes, from among ideas gathered from a range of sources, including parliamentarians, the public and other stakeholders (www.parliament.uk/post). POSTnotes are generally researched through a series of interviews with key experts. Almost 600 POSTnotes have been published since 1989, on subjects ranging from the psychological health of military personnel to new plant-breeding technologies. All are freely available online and held in the House of Commons library.

The European Union Scientific Advice Mechanism, on the other hand, responds to requests for advice from the ‘College of European Commissioners’ through a group of government-appointed scientific advisers. It delivers evidence review reports on specific issues, drawing on a network of expertise from more than 100 European scientific academies in over 40 countries (e.g. The Royal Society in the UK, Hungarian Academy of Sciences). For both it and POST, adherence to a clearly defined process is a way of building credibility and assuring transparency. However, it does not necessarily provide the flexibility to address the diversity of issues and problems faced by environmental policy decision-makers.

To summarise, active knowledge exchange mechanisms can have a range of scales, formats and institutional arrangements. This plurality is the best approach to linking science and policy in decision-making contexts, where different types of questions continually arise.

8.5 National and international scientific assessments

A longer-term approach to aligning evidence synthesis with conservation policy decisions involves governments or international bodies mandating large-scale, scientific assessments in broad areas of strong policy interest. Examples include the assessment reports conducted by the Intergovernmental Panel on Climate Change (IPCC; www.ipcc.ch), Intergovernmental Science Policy Platform on Biodiversity and Ecosystem Services (IPBES; www.ipbes.net) and Millennium Ecosystem Assessment (www.millenniumassessment.org; see Chapter 16 for further details of mechanism and function of the Millennium Ecosystem Assessment and the IPBES science–policy platform). These global assessments involve hundreds or even thousands of scientists around the world, including indigenous and local knowledge-holders in the case of IPBES (Sutherland et al., Reference Sutherland, Gardner and Haider2014; see also Chapter 16).

Generally, governments define the scope of the assessment and identify or nominate a set of experts to conduct it (IPCC, 2015). The nominated experts form working groups and develop report texts, which are subject to extensive, transparent review, first by other experts and then by governments. Following review, the report texts are converted into concise summary documents (usually called ‘Summary for Policy-makers’), the final text of which is agreed by governments. Each statement in the summary document must be traceable back to the full scientific report and, from there, to individual pieces of research or sources of knowledge. Through this process, science and policy influence one another in a two-way exchange of knowledge over very large temporal and spatial scales.

The IPCC, which has been active for almost three decades, has built a strong reputation for providing an overview of climate science across a range of disciplines, from geophysics to economics. There are now clear links from the scientific understanding of human-induced climate change and its impacts to policies controlling greenhouse gas emissions at national and international levels. Most recently, the Paris Climate Agreement of December 2015 is a global accord under which nations have made pledges and set emissions targets to keep global temperature rise below 2°C (Clemencon, Reference Clemencon2016; Tobin et al., Reference Tobin, Schmidt and Tosun2018). A large quantity of scientific research underlies these policy pledges, which would likely not have happened, or not have been so extensive, without the IPCC assessment process. Forty-five different global climate models are now being used together to link levels of greenhouse gas emissions to long-term global temperature rise under different emissions scenarios (Collins et al., Reference Collins, Knutti, Arblaster, Stocker, Qin and Plattner2013). There is also a plethora of analyses and modelling connecting economic activity to greenhouse gas emissions (e.g. Vandyck et al., Reference Vandyck, Keramidas and Saveyn2016) and threshold temperate rises with specific impacts on environments, economies and human well-being (IPCC, Reference Field, Barros and Dokken2014).

The Millennium Ecosystem Assessment (2005) was the first global evaluation of the status of ecosystems, and developed the ecosystem services framework for understanding how nature can benefit people. The ecosystem services concept originated in the academic world (Potschin & Haines-Young, Reference Potschin, Haines-Young, Potschin, Haines-Young, Fish and Turner2016), but the Millennium Ecosystem Assessment formalised the thinking, providing a conceptual framework and nomenclature for ecosystem services. Since its publication, a growing number of countries have conducted their own national ecosystem assessments (Schrӧter et al., Reference Schröter, Albert and Marques2016) and the policy ground is being set for their results to be used in national natural-capital accounting. Both Aichi Biodiversity Target 2 from the Convention on Biological Diversity’s Strategy Plan 2011–2020 (Convention on Biological Diversity, 2010) and Action 5 of the EU Biodiversity Strategy to 2020 (European Commission, 2011) call for biodiversity values to be incorporated into national accounting.

Large-scale assessments are most effective at aligning scientific evidence with decisions when there is a broad issue of strong political interest, such as climate change or biodiversity loss. The assessments are expensive (see Table 8.1), so there must be substantial political commitment and a source of funds over the relatively long term.

Table 8.1 A summary of the costs associated with three mechanisms to align evidence synthesis with policy and practice in the environmental field, compared to the costs of individual evidence synthesis methods

ActivityWhen to applyCost (£)
Mechanisms to align evidence synthesis with the needs of policy and practice
Decision support toolsSpecific question, repeatedly addressed380,000–3.9 million per tool1
Knowledge exchange mechanismsMany questions arising600,000 per year2
International assessmentsOne big, broad issue~3 million per year3
Individual evidence synthesis methods
Systematic reviewMany studies address a single question19,000–190,0001
Subject-wide evidence synthesisMultiple sources of relevant evidence exist
  • Initial cost: 45,000–480,000

  • Update cost:

  • 20% of initial cost1

2 Cost of the EKLIPSE mechanism;

Given the obvious power of national and international scientific assessments to influence policy, it is now more important than ever to incorporate into them the transparent, unbiased repeatable methods that have been developed for evidence synthesis. Currently, the rigour and reliability of large-scale scientific assessments rely on extensive peer review, rather than systematic searching or careful elicitation methods that reduce bias. Evidence synthesis methods are usually not reported (with some exceptions, such as chapter 6 of the Intergovernmental Science Policy Platform on Biodiversity and Ecosystem Services pollination report; IPBES, 2016). However, such assessments are conducted over long timescales, with the IPCC, for example, producing a global assessment report every 5–10 years. With this amount of time and money available (see Table 8.1) there is a clear opportunity to develop rigorous processes of evidence synthesis within this framework. As a first step, we urge policy-makers and institutions involved in commissioning large-scale scientific assessments to require authors to report their underlying synthesis methods.

8.6 What does it all cost?

The cost of the alignment mechanisms outlined in this chapter varies considerably, both within and among the different activities (Table 8.1). These costs should be interpreted in the context of total spending on scientific research. For example, the budget of the European Commission’s flagship scientific research programme, Horizon 2020, is approximately £8 billion per year.

The organisations that fund research and aspire to be evidence-informed already invest heavily in improving interactions between science, policy and practice. Unfortunately, they frequently fund expensive decision support systems that are not maintained or used a few years later (Zasada et al., Reference Zasada, Piorr and Novo2017) and large-scale reviews or scientific assessments that do not follow clear protocols to reduce bias. The challenge in aligning evidence synthesis with decision-making is not to find the money, but to demand and enable improved rigour and continuity in activities that are already taking place.

No single mechanism will be best for aligning evidence with policy and practice in all contexts. Each has strengths and weaknesses, and can be applied in different circumstances and at different scales. International assessments have redirected policies and scientific endeavour on a very large scale, but would be unlikely to align specific scientific findings with conservation practice at smaller scales. At smaller scales, the potential of decision support systems to incorporate rigorously collated environmental evidence has hardly been tapped.

At every level, mechanisms to link synthesised evidence with policy and practice decisions need to be funded sufficiently to ensure salience, legitimacy, credibility and transparency. These linking mechanisms need access to methods of collating and communicating evidence that are well-developed, transparent and widely understood (Cook et al., Reference Cook, Nichols and Webb2017; Dicks et al., Reference Dicks, Haddaway and Hernández-Morcillo2017) and are just as important as the research itself, if not more so.

References
Cash, D. W., Clark W. C., Alcock, F., et al. 2003. Knowledge systems for sustainable development. Proceedings of the National Academy of Sciences, 100, 80868091.
Clemencon, R. 2016. The two sides of the Paris climate agreement: dismal failure or historic breakthrough? Journal of Environment & Development, 25, 324.
Collaboration for Environmental Evidence. 2013. Guidelines for Systematic Review and Evidence Synthesis in Environmental Management. Version 4.2. Available from www.environmentalevidence.org/Documents/Guidelines/Guidelines4.2.pdf.
Collins, M., Knutti, R., Arblaster, J., et al. 2013. Long-term climate change: projections, commitments and irreversibility. In: Stocker, T. F., Qin, D., Plattner, G.-K., et al., editors, Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge: Cambridge University Press.
Convention on Biological Diversity. 2010. Decision X/2. The Strategic Plan for Biodiversity 2011–2020 and the Aichi Biodiversity Targets. UNEP/CBD/COP/DEC/X/2, 29 October 2010.
Cook, C. N., Mascia, M. B., Schwartz, M. W., et al. 2013b. Achieving conservation science that bridges the knowledge-action boundary. Conservation Biology, 27, 669678.
Cook, C. N., Nichols, S. J., Webb, J. A., et al. 2017. Simplifying the selection of evidence synthesis methods to inform environmental decisions: a guide for decision-makers and scientists. Biological Conservation, 213(Part A), 135145.
Cook, C. N., Possingham, H. P. & Fuller, R. A. 2013a. Contribution of systematic reviews to management decisions. Conservation Biology, 27, 902915.
Defra. 2014. Supporting Document to the National Pollinator Strategy: For Bees and Other Pollinators in England. London: Department for Environment, Food and Rural Affairs.
Dicks, L. V., Abrahams, A., Atkinson, J., et al. 2013a. Identifying key knowledge needs for evidence-based conservation of wild insect pollinators: a collaborative cross-sectoral exercise. Insect Conservation and Diversity, 6, 435446.
Dicks, L. V., Bardgett, R. D., Bell, J., et al. 2013b. What do we need to know to enhance the environmental sustainability of agriculture? A prioritisation of knowledge needs for the UK food system. Sustainability, 5, 30953115.
Dicks, L. V., Haddaway, N., Hernández-Morcillo, M., et al. 2017. Knowledge synthesis for environmental decisions: an evaluation of existing methods, and guidance for their selection, use and development – a report from the EKLIPSE project. EKLIPSE D3.1, Version 1.0.
Dicks, L. V., Walsh, J. & Sutherland, W. J. 2014. Organising evidence for environmental management decisions: a 4S hierarchy. Trends in Ecology & Evolution, 29, 607613.
Dicks, L. V., Wright, H. L., Ashpole, J. E., et al. 2016. What works in conservation? Using expert assessment of summarised evidence to identify practices that enhance natural pest control in agriculture. Biodiversity and Conservation, 25, 13831399.
European Commission. 2011. Our Life Insurance, Our Natural Capital: An EU Biodiversity Strategy to 2020. 3.5.2011 COM(2011) 244. 2011. Brussels: European Commission.
Floor, J. R., van Koppen, C. S. A. & Lindeboom H. J. 2013. A review of science–policy interactions in the Dutch Wadden Sea – the cockle fishery and gas exploitation controversies. Journal of Sea Research, 82, 165175.
Guston, D. H. 2001. Boundary organizations in environmental policy and science: an introduction. Science, Technology, & Human Values, 26, 399408.
Haddaway, N. R., Kohl, C., Rebelo da Silva, N., et al. 2017. A framework for stakeholder engagement during systematic reviews and maps in environmental management. Environmental Evidence, 6, 11.
Ingram, J. S. I., Wright, H. L., Foster L., et al. 2013. Priority research questions for the UK food system. Food Security, 5, 617636.
IPBES. 2016. The Assessment Report of the Intergovernmental Science–Policy Platform on Biodiversity and Ecosystem Services on Pollinators, Pollination and Food Production. Bonn: IPBES.
IPCC. 2014. Summary for policy-makers. In: Field, C. B., Barros, V. R., Dokken, D. J., et al., editors, Climate Change 2014: Impacts, Adaptation, and Vulnerability. Part A: Global and Sectoral Aspects. Contribution of Working Group II to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (pp. 132). Cambridge: Cambridge University Press.
IPCC. 2015. IPCC Factsheet: How Does the IPCC Review Process Work? IPCC Secretariat, Switzerland. Available from www.ipcc.ch/news_and_events/docs/factsheets/FS_review_process.pdf (accessed 29 April 2018).
Jones, A. C., Mead, A., Kaiser, M. J., et al. 2014. Prioritization of knowledge needs for sustainable aquaculture: a national and global perspective. Fish and Fisheries, 16, 668683.
Lidskog, R. 2014. Representing and regulating nature: boundary organisations, portable representations, and the science–policy interface. Environmental Politics, 23, 670687.
Maxim, L., Spangenberg, J. H. & O’Connor, M. 2009. An analysis of risks for biodiversity under the DPSIR framework. Ecological Economics, 69, 1223.
Millennium Ecosystem Assessment. 2005. Ecosystems and Human Well-being: Synthesis. Washington, DC: Island Press.
Norton, M. 1997. The UK Parliamentary Office of Science and Technology and its interaction with the OTA. Technological Forecasting and Social Change, 54, 215231.
Potschin, M. & Haines-Young, R. 2016. Ecosystem services in the twenty-first century. In: Potschin, M., Haines-Young, R., Fish, R. & Turner, R.K., editors, Routledge Handbook of Ecosystem Services (pp. 19). London:Routledge.
Pretty, J., Sutherland, W. J., Ashby, J., et al. 2010. The top 100 questions of importance to the future of global agriculture. International Journal of Agricultural Sustainability, 8, 219236.
Raymond, C. M., Frantzeskaki, N., Kabisch, N., et al. 2017. A framework for assessing and implementing the co-benefits of nature-based solutions in urban areas. Environmental Science & Policy, 77, 1524.
Richards, M., Metzel, R., Chirinda, N., et al. 2016. Limits of agricultural greenhouse gas calculators to predict soil N2O and CH4 fluxes in tropical agriculture. Scientific Reports, 6, 26279.
Rose, D. C., Parker, C., Fodey, J. O. E., et al. 2018. Involving stakeholders in agricultural decision support systems: improving user-centred design. International Journal of Agricultural Management, 6, 8089.
Rose, D. C., Sutherland, W. J., Parker, C., et al. 2016. Decision support tools for agriculture: towards effective design and delivery. Agricultural Systems, 149, 165174.
Sarkki, S., Tinch, R., Niemelä, J., et al. 2015. Adding iterativity to the credibility, relevance, legitimacy: a novel scheme to highlight dynamic aspects of science–policy interfaces. Environmental Science and Policy, 54, 505512.
Schröter, M., Albert, C., Marques, A., et al. 2016. National ecosystem assessments in Europe: a review. BioScience, 66, 813828.
Smith, R. K., Dicks, L.V. & Sutherland, W. J., 2015. Scientific Evidence to Address Priority Knowledge Needs For Sustainable Agriculture. Conservation Evidence resources. Available from www.conservationevidence.com/synopsis/download/19 (accessed 28 April 2018).
Sutherland, W. J., Dicks L. V., Ockendon, N., et al. 2019a. What Works in Conservation 2019. Cambridge: Open Books Publishers.
Sutherland, W. J., Taylor, N. G., MacFarlane, D., et al. 2019b. Building a tool to overcome barriers in the research–implementation space: the Conservation Evidence database. Biological Conservation, 283, DOI:10.1016/j.biocon.2019.108199
Sutherland, W. J., Gardner, T. A., Haider L. J., et al. 2014. How can local and traditional knowledge be effectively incorporated into international assessments? Oryx, 48, 12.
Tobin, P., Schmidt, N. M., Tosun, J., et al. 2018. Mapping states Paris climate pledges: analysing targets and groups at COP 21. Global Environmental Change – Human and Policy Dimensions, 48, 1121.
Vanbergen J., Heard, M. S., Breeze, T., et al. 2014. Status and Value of Pollinators and Pollination Services. London: Department for the Environment, Food and Rural Affairs.
van der Molen, F. 2018. How knowledge enables governance: the coproduction of environmental governance capacity. Environmental Science & Policy, 87, 1825.
van der Molen, F., Puente-Rodríguez, D., Swart, J. A. A., et al. 2015. The coproduction of knowledge and policy in coastal governance: integrating mussel fisheries and nature restoration. Ocean & Coastal Management, 106, 4960.
Vandyck, T., Keramidas, K., Saveyn, B., et al. 2016. A global stocktake of the Paris pledges: implications for energy systems and economy. Global Environmental Change, 41, 4663.
van Enst, W. I., Runhaar, H. A. C. & Driessen, P. P. J. 2016. Boundary organisations and their strategies: three cases in the Wadden Sea. Environmental Science & Policy, 55, 416423.
Watt, A. D., Ainsworth, G., Balian, E., et al. 2018. Building a mechanism for evidence-informed European policy on biodiversity and ecosystem services through engagement of knowledge holders. Evidence and Policy, http://doi.org/10.1332/174426418X15314036194114
Wood, A. M., Kaptoge, S., Butterworth, A. S., et al. 2018. Risk thresholds for alcohol consumption: combined analysis of individual-participant data for 599 912 current drinkers in 83 prospective studies. The Lancet, 391, 15131523.
Wyborn, C., Louder, E., Harrison, J., et al. 2018. Understanding the impacts of research synthesis. Environmental Science & Policy, 86, 7284.
Young, J. C., Waylen, K., Sarkki, S., et al. 2014. Improving science–policy dialogue to meet the challenges of biodiversity conservation: having conversations rather than talking at one-another. Biodiversity and Conservation, 23, 387404.
Zasada, I., Piorr, A., Novo, P., et al. 2017. What do we know about decision support systems for landscape and environmental management? A review and expert survey within EU research projects. Environmental Modelling & Software, 98, 6374.