In chapter 5, we developed a logical framework for assessing impacts, including the necessity for control or reference areas, and for sampling to occur before and after the putative impact. In this chapter, we consider practical details of the monitoring, focusing on the formal design and statistical analysis appropriate to the detection of impacts, and on the practical details associated with executing these designs in running waters. It is important to realize that we can only translate general design principles into a specific plan to collect data if we specify the statistical model that is to be fitted to the data. Perhaps the most important message of this chapter is that apparently similar monitoring ‘questions’ can have quite different statistical models behind them. These different models, in turn, can lead to quite different advice about how to optimize a particular data collection program.

If we consider the two major tasks outlined earlier, the formal test for the existence of an unacceptable impact and the characterization of the spatial extent of any impact, the latter procedure is relatively straightforward in terms of the design and underlying statistical models. The detailed design is modified by practical considerations associated with stream environments, and the characteristics of the activity suspected of having an impact.

In contrast, there is a range of design options for detecting an impact.

Following the arguments and considering the issues presented in this book will allow us to design an effective and flexible monitoring program to detect and evaluate impacts in flowing waters. This includes the negotiations of what effect size is important to detect and what elements can be traded off or even sacrificed (as discussed in chapters 12 and 13). So now we've implemented the monitoring design and presumably detected (or not) some impact with known confidence – but the job is not yet finished. Further negotiations are in order to continue or refine assessment. Truly effective management of impacts requires that some action follows the well-designed studies we have so far advocated in this book. This chapter discusses issues that are central to what needs to be done after the main monitoring task has been completed.

LINKS WITH MANAGEMENT DECISIONS AS POINTS OF NEGOTIATION

We have emphasized the role that input from monitoring data (in terms of results and their interpretation) should have in management decision-making if we are to be engaged in a task that makes any difference. There is an imperative for environmental assessments to become more sophisticated and responsive to societal needs. Monitoring can be reactive (used only once an impact is clearly observed), proactive (seeking to assess impacts before they manifest themselves; see Fairweather 1993; Fairweather & Lincoln Smith 1993) or progress through adaptive learning. The latter means not just trying to benefit from mistakes but also combining elements of learning from both the scientific and management sides.

Statistical decision theory has a long history and can basically be viewed in two ways. Classical statistical hypothesis-testing in the Neyman–Pearson form (Neyman & Pearson 1928), which we described in chapter 4, emphasizes decision errors from the test of a null hypothesis, and these errors have a frequentist interpretation. In contrast, what is termed modern statistical decision theory has a strong Bayesian influence (Berger 1985; Hamburg 1985; Pratt et al. 1996) and has emphasized monetary costs and benefits from decisions in an economic and management context. Nonetheless, all statistical decision problems have certain characteristics in common (Box 12.1; Hamburg 1985; Neter et al. 1993). In this chapter, we will focus on errors associated with the components of the decision-making process and how the choice of criteria for making decisions interacts with the design of the monitoring program.

MAKING STATISTICAL DECISIONS

We need to examine the errors possible from a statistical decision-making process in an environmental monitoring context. In chapter 4 (see Table 4.4) we defined two possible types of error. These errors arise because we are making decisions about the truth or otherwise of hypotheses about unknown population parameters from imperfect samples. If we could record an entire population, such as all the possible locations upstream and downstream of the mine, then we could make decisions about the truth of hypotheses about those parameters without sampling error. Errors of inference may still arise, due to measurement error and confounding.

In the last chapter, we introduced the different sorts of analytical models we can use to detect impacts. There are distinct and important differences between these models, and yet understanding these differences is only the first step. The next step is to be able to apply these designs sensibly and usefully in real situations. In many cases as we will see, the ecology of streams and rivers (and, indeed, many other environments) is not sufficiently well understood to make perfect or even very good decisions about design in every instance. As we shall argue below, however, the critical issue is to understand how to make good design decisions, and, because monitoring designs always involve compromises, to be very clear and explicit about the reasoning upon which such decisions were based. The remaining part of the process, then, is to understand what monitoring designs can tell you definitively and what they cannot – what can be legitimately and logically interpreted from the data versus what will remain unclear.

Below we describe a hierarchy of decisions that will help define the nature of the problems faced in monitoring impacts in flowing waters properly. There are several problems we have to solve to apply good design principles. These problems are the location and character of control locations, and the frequency of sampling through time. In many places, we will suggest that a systematic and well-structured review of the literature will be necessary to solve these problems.

Chapter 5 described the logic behind the BACI approach and chapter 7 described the four basic BACI-type analytical models and the relative strengths of inference each provides (Table 7.1). We can make strong inferences (those with least uncertainty) about the effects of human impacts by examining differences between control and impact locations before and after the onset of human activity, most especially when we have replication of these design elements. However, what happens when one or more BACI elements are entirely missing or when we have no replication? Perhaps the most common problem facing environmental managers is where putative impacts have already occurred, tens or even hundreds of years before, and there is no scope for planning a Before period. There may be no control locations because all suitable locations have suffered the same human activity in question. The latter problem is particularly common when modern human activities are spread over large spatial scales because, as indicated earlier, it reduces the potential numbers of places we can search for controls. How should we proceed in these circumstances?

We must recognize first that the difficulties created here are ones of increased inferential uncertainty, not which analytical model to apply. When one or more of the four elements are missing, we lack the information that would otherwise allow us to distinguish, with some confidence, those changes caused by human impacts from those caused by alternative (natural) phenomena (Table 9.1; and see chapter 5).

In this chapter we discuss the basics of good monitoring design. ‘Design’ here means the stipulation of where, when and how many observations or sampling units are taken to provide the data from which we will make inferences against some specified objectives. We discuss here the underlying principles that we consider central to good design, and present an ideal case. In the interests of establishing an understanding of why elaborate designs are often presented, we ignore for the moment the ubiquitous compromises that are necessary for logistic, social or economic reasons. We do not focus here on particular variables (chapter 10), what sorts of changes are considered important (chapter 11) or the specifics of natural systems in the interests of presenting the general principles that underlie good monitoring for most variables in almost any system. Nor do we discuss here the analytical tools used to refine or optimize designs or analyse the resultant data (chapters 7–13). This chapter should be read, therefore, as a conceptual overview of the design principles that motivate us and which will be expanded in operational detail throughout later chapters.

We recognize that ‘ideal’ designs will rarely, if ever, be feasible (for a variety of reasons) and discuss in later chapters what compromises are most likely to be precipitated by the characteristics of streams (chapter 8) or because of accidents of history, money etc. In beginning here with an outline of the concepts behind an ‘ideal’ case, we seek to establish the principle that all these inevitable compromises are just that – compromises.

Chapter 5 dealt with the ideal design criteria appropriate for the logical detection of impacts. In this chapter, we illustrate why these design criteria are important by outlining the problems that have been faced by ecologists seeking to apply at least some of the principles that were discussed in chapter 5. Many studies were conducted before much was known about good design, but problems in applying monitoring designs have arisen partly from the nature of rivers themselves (chapter 2), partly from the nature of the variables that river ecologists deal with (chapter 10), and partly from institutional and political arrangements that affect the timing and scope of such studies.

Before describing these problems in detail, we first present an historical sketch of the previous attempts at implementing programs for impact assessment and monitoring using biological data in rivers and streams. This brief history lesson illustrates the apparent simplicity and hidden complexities of assessing and monitoring in rivers and streams. We will then elaborate on how each of the three problem areas makes it difficult to implement ideal experimental designs in rivers and streams.

A BRIEF HISTORICAL SKETCH

The vast majority of published field studies of impacts in streams have been directed at investigating pre-existing impacts (see reviews by: Hellawell 1986; Hynes 1960). Many of the earlier studies were aimed at least at corroborating conclusions drawn from observations of large, obvious human impacts and from physicochemical investigations (Fjerdingstad 1964; Forbes & Richardson 1919; Kolkwitz & Marsson 1908; Patrick 1949), and sometimes at establishing that biological variables were detecting changes that had been missed by using physicochemical data alone (reviewed by Wilhm 1975).