Controlled Experiments

One route to gaining information we need is through controlled experiments in which testable propositions, termed hypotheses (singular: hypothesis), are formulated, and information is gathered to test them. The investigator makes predictions about what will happen under certain circumstances if a particular hypothesis is true, and then determines whether those predictions are fulfilled. If the predictions are not fulfilled, the hypothesis is falsified.

To do a controlled experiment, scientists compare the responses of two or more identical groups to different treatments. If the responses to the treatments are accurately measured and the results are correctly recorded, then any differences in the responses of the different groups may be evidence that the treatment had an effect.

It is important, however, to be aware of potential bias, a consistent tendency to deviate in one direction from a true value. Bias occurs when one outcome or answer or piece of evidence is selected or encouraged over others. Biases might be caused by using faulty instruments or they might result if an observer allows their perceptions to be influenced by preconceived notions. Bias may be introduced deliberately, or it may be unintentionally caused by unacknowledged assumptions or by faulty sampling.

Even evidence that we think of as objective may be biased. For instance, we tend to regard photographs as objective records of reality. We say “seeing is believing” and “one photograph is worth a thousand words.” But the photographer makes decisions, consciously or unconsciously, about what to place in the frame, how much light to let into the picture, and how sharp to make the focus. These influence our response to a photo. If a photographer of wolves selects only wolves that are attacking prey, focuses on bloody wounds they inflict, and doesn’t allow a lot of light into their photos, that is likely to get an unfavorable response from viewers.

Because researchers at Michigan State University and the Michigan Department of Conservation suspected that nutrition affected the ability of fawns to survive their first winter, they tested the hypothesis that the amount of protein in the diet of white-tailed deer fawns during their first autumn would affect the rate at which they gained weight (Ullrey et al., Reference Ullrey, Youatt and Johnson1967). The research team conducted a controlled experiment in which 45 captive white-tailed deer fawns were assigned at random to receive one of three diets for 98 days after they were weaned. The three diets differed only in the percentages of corn and soybean meal they contained, which resulted in three different levels of crude dietary protein. At the beginning of the trial and 70 and 98 days after the start of the trial, all fawns were weighed, and rates of weight gain were calculated. The results showed a statistically significant effect of dietary protein on rates of weight gain in both males and females, with fawns that received more dietary protein gaining weight faster than those that got the lowest level of protein.

The use of statistics allows investigators to evaluate rigorously any differences between experimental groups. The term “significant” has a specific meaning in statistics. The statistically significant effect of dietary protein on rates of weight gain meant that there was a low probability (which is stated as a calculated P level) that the observed differences were due to chance alone rather than to the different treatments. Sample size is taken into consideration when significance levels are calculated in order to account for the greater likelihood that differences in small groups will be due to chance alone.

Regardless of whether a controlled experiment gives the expected results, an experiment should be repeated, or replicated. This is especially useful if the repeat experiment is done by different investigators. If the different groups of investigators get the same results, that further bolsters the original conclusions. If not, then the researchers on each team should look for reasons for the discrepancy, perhaps formulating new hypotheses about what is going on and designing experiments to test them.

Getting unexpected results is not a bad thing in science. In fact, unexpected results are a major driver of scientific progress because they spur scientists to try to find out why things didn’t turn out as expected. Because it provides a framework within which results can be compared to predictions, and conclusions can be modified as new evidence becomes available, science has the potential to be self-correcting.

Controlled experiments assume that the different treatments are the only difference between treatment groups. In the study of the effects of dietary protein on fawn weight gain, the three treatment groups were similar at the start of the experiment. They were the same approximate age, had been reared under the same conditions, and were weaned at approximately the same age. During the experiment, the three experimental groups were kept under identical conditions except for the different diets they received.

In studies of captive wildlife, ensuring that all experimental subjects are kept under the same conditions is relatively straightforward. That is a lot harder to do in field experiments. This is illustrated by an experiment about the effects of predation on prey populations in Australia. Drought is a major factor limiting populations of small mammals in Australia, but prey populations typically recover quickly after they collapse during a drought. Wildlife biologists at CSIRO, Australia’s national science agency, designed a controlled experiment to test the hypothesis that predators could prevent populations of rabbits from rebounding after a crash due to drought (Newsome et al., Reference Newsome, Parer and Catling1989). To do this, the researchers killed cats and foxes on study plots at Yathong Nature Reserve (Figure P.3) that had low populations of rabbits, and then compared estimated rabbit populations before, during, and after the predators were removed from those plots to estimated rabbit populations on similar plots where predators had not been removed. Fourteen months after the start of the experiment, estimated rabbit populations had rebounded on the treated plots (the plots where predators were removed) but not on the plots from which predators had not been removed. The researchers concluded that predation on the rabbits had delayed recovery from the drought-induced population crash.

The concept of testing the effects of predators on prey populations by conducting removal experiments is simple, but putting it into practice is not. In this study, fencing to keep mammalian predators from moving onto the treatment plots was impractical, so predators were shot throughout the experiment. It would have been even more challenging to cover the study area with something that would keep out bird predators. Fortunately, earlier studies had suggested that birds of prey would have little effect on rabbits in this setting, so they were not considered in the experiment.

It is relatively easy to measure changes in captive animals, like the fawns in the experiment described above, if good instruments and observers are available, but obtaining reliable measurements in the field is more complex. The most thorough way to determine population size is to count all individuals in an area. However, it is rarely possible to do this in studies of wild animals. Consequently, it is usually necessary to estimate populations of animals by making an approximation that is based on a sample. Many sampling methods – each with advantages and disadvantages – are available. When it is not possible to observe individual animals directly, it may be necessary to rely on counting signs of activity, such as calls, feces, nests, or burrows and to use this information as an indicator (index) of abundance. This technique is only useful if the number of individuals that produce a given amount of sign is known. In the predator-removal study, researchers used counts of active burrow entrances to estimate rabbit populations, as well as a second method – counts of rabbits observed with a spotlight at night along sampling transects.

Sometimes when researchers cannot answer a question through field experiments, they can get part of the answer they need by doing experiments in artificial settings and extrapolating the results to field conditions. For instance, populations of the American black duck declined in eastern North America at around the time when wetlands (habitats between terrestrial and aquatic environments) in that region became more acidic due to acid precipitation (Section 5.2.1.5). If someone wanted to test the hypothesis that acidification of the wetlands in black duck habitat caused a decrease in the growth rate of ducklings, there would have been substantial obstacles to doing the sort of controlled field experiment that might answer this question because such an experiment would risk serious negative impacts to the experimentally acidified habitats and the ducks within them. Instead, Dr. Barnett Rattner and his colleagues at the US Fish and Wildlife Service constructed six artificial wetlands where they studied this question. Three of these wetlands were randomly selected to receive a treatment of sulfuric acid, and the other three were left as controls (Rattner et al., Reference Rattner, Haramis and Chu1987). A captive black duck hen with three or four ducklings was placed in each wetland for 10 days. Initially, the ducklings in all groups were of similar age and weight. At the end of the trial, the ducklings reared on the acidified wetlands weighed significantly less than those reared on the untreated wetlands. Despite the limitations on experiments to test the influence of acidification on black ducks, this study provided evidence of an effect. However, because the experimental system used captive animals and constructed wetlands, we cannot be certain that the results applied to the wild.

Well-designed controlled experiments shed light on many important questions. They are particularly useful when questions of policy (What level of pesticide application should be permitted? Should predators be killed? Should naturally started fires be put out? Should drainage of wetlands be allowed? What about mining in the Amazon?) are at stake, because in matters where we need to evaluate alternative courses of action, this methodology defines a standard for evidence. However, the design, methods, and underlying assumptions of scientific studies should always be carefully evaluated.

The principal advantage of controlled experiments is that they reduce the workings of the world to a collection of understandable variables that can be manipulated. Fortunately, there are statistical techniques that allow us to evaluate in a single study the effects of many variables and the interactions between them. The conceptual simplicity of controlled experiments is also a disadvantage, however. Ecological systems are complex. In nature species interact with each other and their environment in myriad ways that are not easily mimicked by controlled experiments.

In addition, there are many situations in which it is impossible or unethical to do controlled experiments. Fortunately, we can sometimes take advantage of natural experiments to get information that is useful for testing hypotheses.

Comparative Studies (Natural Experiments)

If we want to know the effects of potentially harmful treatments – such as pesticides, radiation, oil spills, or acid rain – on populations of wild organisms, there are substantial practical and moral obstacles in the way of doing controlled experiments like the ones described above. This is particularly true when we study past environments (where it is not possible to experiment) or rare or sensitive species and ecosystems (where it is not ethical to deliberately cause exposure to something harmful).

The ozone layer in our upper atmosphere reduces the amount of harmful ultraviolet radiation that reaches the Earth (Section 5.2.1.3). This layer became depleted during the 1970s, and the thinning of the ozone layer that resulted is thought to be responsible for a variety of ecological changes ranging from altered food chains to cataracts and blindness in some species. Controlled field experiments to evaluate the effects of reduced ozone are not possible, because there is only one Earth.

Fortunately, there are other ways of getting relevant information. One approach is the comparative study, in which conditions are compared in two or more situations that differ in place, time, or another variable but are alike in many other respects. For instance, we might compare similar events among closely related organisms or in similar habitats or in the same place at different times. When we want to assess the effects of inadvertent environmental perturbations, such as the thinning of the ozone layer, studies of conditions before and after the change are useful. If a pronounced change in the value of a variable – such as a decrease in the amount of ozone or an increase in the concentration of carbon dioxide (CO₂) in the atmosphere – is observed after a certain date, investigators can search for events that preceded and might have caused the change.

This method can shed some light on what might have caused thinning of the ozone layer, but it too has limitations. First of all, in most studies of this type, the early data were not gathered in the same way as more recent data. Second, there are likely to be multiple variables that changed during the period of interest, and this will complicate interpretation of the data. Third, there are usually time lags between a cause and its effect, but most often we don’t know how long those lags are. Whatever caused ozone thinning might have begun changing a few years or a few hundred years before the resulting change in atmospheric composition was noticed.

Usually, we do not foresee the consequences of our actions, so we do not plan before-and-after comparisons ahead of time. If someone had suspected a hundred years ago that the ozone layer might wane, they could have tried to gather data on conditions before that happened (although they wouldn’t have had the technology to do this very well) and compared it to data gathered subsequently. On the other hand, if someone had foreseen this change far in advance, perhaps people would have taken steps to prevent it or slow its course.

So, scientists are often left scrambling to conduct the first phase of an unplanned comparative study. To do this, they may scour historical records and earlier studies to glean information about prior conditions. This kind of information is very useful, and it underscores the importance of keeping accurate records because one never knows what use data will be put to in the future. But frequently, this type of information was gathered using methods that differ from the ones we would choose. Investigators might have to use data from many different sources, or data that were gathered using different methods and by workers with different degrees of expertise and training and different ideas about what is important. Since this type of information was rarely compiled with the questions that concern contemporary researchers in mind, often the relevant information was simply not recorded. Consequently, researchers have to make inferences from scraps of information.

As with controlled experiments, the results of comparative studies can be used to test hypotheses. We can state hypotheses, make predictions derived from our hypotheses, and then look at evidence from the past to find out if our predictions are correct. Care must be taken, however, in interpreting the results of this type of study. When trying to disentangle cause and effect in the past, we often face situations where several variables changed simultaneously. Correlation, the association of variables with each other, does not equal causation. Comparative studies may identify certain factors that occurred together in time, and statistical methods can be used to evaluate the significance of these associations (that is, the likelihood that they are due solely to chance), but this does not prove a causal relationship between them. There may be other factors that changed at the same time and that actually caused or at least contributed to the effects we are interested in. (In Box 2.2, we will encounter an example of an unplanned before-and-after study in which it was difficult to determine causation because multiple variables changed simultaneously.)

Models

Scientists often seek to understand the behavior of systems under conditions that cannot be observed directly. This may be because the system is too small (an atom) or too large (the Earth’s atmosphere) to observe directly or because the phenomena of concern took place in the past or are still going on (climate change). In such situations, scientists often construct a model, a concrete or abstract representation of a system, that can be used to predict how a system behaves under specified conditions. A scientific model may take many forms: a physical structure, a description, an equation, an analogy, or a theoretical projection. A simulation is a type of model that predicts the changes a system undergoes given certain starting conditions and assumptions. Computers are very useful for this type of modeling because they allow researchers to manipulate many variables and to perform calculations rapidly under a wide range of scenarios (Section 10.3).

Models and simulations are used a great deal in conservation. If we want to predict what effects a proposed policy will have on habitats or populations, models are very useful. How long will the world’s tropical forests last if we continue clearing them at current rates? What will the average summer temperature be in London in 2,050 CE if we cut our production of greenhouse gases in half? How long will it take for a population to become extinct if its current population trends continue? If we introduce six wolves into an area of suitable habitat, what size will the wolf population attain in 20 years? We cannot answer these questions directly, but we can measure responses under certain conditions and use this information to predict the outcome under other conditions. If our predictions are borne out, we can develop simulations to predict parts of the system in more detail. If not, we can revise our simulations in an effort to come up with better predictions.

Although models and simulations usually represent systems that are not amenable to experimentation, experiments may be useful for examining how certain parts of a system work. For example, you might wish to conduct experiments to test the responses of plants and animals to several treatments for cleaning up spilled oil. The information obtained from the experiments could then be used to modify your model of how long it takes for an ecosystem to recover from oil spills under different condition. A model’s predictions should be repeatedly tested against reality, and the information generated in this way should be used to refine the model in order to make it more realistic.

Models are particularly useful where the risks of doing experimental studies are unacceptable, as in the case of research on rare organisms. Field studies inevitably involve a degree of disturbance to wild populations, while laboratory experiments require the removal of some individuals from the wild (and possible stress or mortality from handling). Both these outcomes should be avoided when dealing with sensitive populations. Models are one way to avoid these negative impacts.

Like other methods of getting information, simulations and other kinds of models have limitations as well as advantages. A model always incorporates certain assumptions about how a system behaves. We should therefore keep in mind the assumptions on which models are based. One reads a great deal these days about debates over models that predict global changes in environmental conditions, population growth, and resource availability. Much of the debate focuses on different assumptions about how the system in question behaves.

Scientists are influenced by their values and assumptions about the natural world, which guide their decisions about what to study and how to interpret their findings. These values and assumptions are derived from many sources including intuition, conviction, faith, ideology, experience, and other intangible states. In choosing what to study (in other words, what to focus on), the scientist is like the photographer choosing how to frame their shot. We need to be on the lookout for cases where models incorporate assumptions about how the world works that are at odds with the data that are used to construct the models (Botkin, Reference Botkin2012).

Models necessarily oversimplify the behavior of the systems they portray. However, a model that incorporates a lot of the important factors influencing a system and contains realistic assumptions about how the system changes is likely to do a good job of predicting that system’s behavior. An oversimplified model with unrealistic assumptions will not.

Natural and Historical Records

Ecologists often look to the past to get information that will help them to understand the present or plan for the future. If we want to know how much the climate varied in the last 2 million years, how often grassland fires occurred in Australia before policies of fire suppression were instituted, where wetlands used to occur in China, what the former geographic range of the snow leopard was, or what the extinction rate of native mammals of Canada was before Europeans arrived, we must study the past.

Natural records include (but are not limited to) ice; soils or sediments; tree rings; fossils, pollen, and artifacts preserved in sedimentary rocks; packrat middens (piles of accumulated objects); and the tissues of long-lived individuals. Museums maintain collections of specimens such as skeletons, study skins, and dried plants. The value of such natural records depends on whether information we are interested in was preserved.

Processes that occur in pulses often produce layered records that are very useful. Periodically deposited sediments and rings or layers that result from variations in the growth rates of wood, bone, fish scales, or coral are examples. These form where alternating cold and warm seasons produce marked differences in the seasonal growth rates of living tissues. In temperate climates, trees produce distinct annual rings, hibernating mammals deposit bone, and fish scales have layers that are correlated with periods of growth.

The position of material in a sequence of layers may provide information on its relative age. By comparing growth rings in the trunks of individual trees that have overlapping life spans, scientists can date tree rings over periods that are longer than the lifespan of an individual tree. When this information is combined with the position of tree scars that resulted from fires which occurred at known dates, chronologies can be constructed that cover thousands of years.

Plants and animals are useful indicators of environmental conditions because every species has a specific range of environmental conditions it can tolerate. Evidence of muskrats or cattails in the past indicates that surface water was present, and the past presence of cacti indicates that there was a hot, dry environment. (This only works if we find records of them where they lived, not if they were transported somewhere else after they died.)

The time span covered by natural records ranges from years to millennia depending on the type of record. Fossils cannot distinguish a year or even a decade within the fossil record, but they can give us information about what was going on millions of years ago. Tree ring chronologies do not go back that far, but they can allow us to pinpoint the year when an event occurred.

Some natural records are more likely to be preserved than others. In other words, the samples passed down to us by natural records are biased. For example, packrat middens are found only in rocky terrain. The absence of packrat middens in sandy soil does not mean that packrats never lived there; it means only that if they lived there, their middens were not preserved. In addition, the record of the past that natural processes provide is often too short or too fragmented to tell us what we want to know. Or the record may be extensive but not provide information for the places and time periods we are interested in.

Historical records are made by people. Journals, maps, notes, photographs, genealogies, censuses, books, newspaper articles, interviews, sketches, paintings, legal transcripts, and recordings can be valuable sources of information about historical ecology. Repeat measurements over time, known as time series data, are useful for reconstructing historical trends and evaluating variability in those trends. Weather stations, stream gauges, astronomical observatories, and satellites record time series data.

Documents are an inexpensive, easy-to-use source of information about the past. However, the value of historical documents depends on their condition and whether the information that is preserved is representative. Such documents provide valuable windows to the past, but the viewpoint of the observer must be taken into consideration. The decision about what to record is always subjective, and historical documents reflect the recorder’s assumptions about what was important. Because anecdotes (personal accounts) recorded in historical documents present the specifics of a particular time and place, they represent a small sample, but the details they capture are useful for understanding the larger context in which events occurred.

The usefulness of historical documents also depends on the accuracy of the recorded information, which in turn depends on the observational skills, memory, meticulousness, and honesty of the person who recorded the information and also on the technical capabilities of the equipment used. The time span covered by historical documents is relatively short (usually decades or centuries), but such documents often allow us to pinpoint when events occurred to the nearest month, week, day, and sometimes even hour.

Scientists have combined natural and historical records to reconstruct changes in vegetation in the Cerro Grande grasslands in northern New Mexico during the twentieth century (Figure P.2). Tree ring chronologies going back to the year 1480 and repeat aerial photographs between 1935 and 1979 showed that forest cover expanded and grassland shrank in that region during the twentieth century. Researchers used this data along with weather records and records of changes in grazing and fire management to evaluate the causes of this shift in vegetation. Subsequently, managers with the US National Park Service made use of this information when they planned nearby restoration programs (Swetnam et al., Reference Swetnam, Allen and Betancourt1999).

Oral Traditions

In many cultures information about the past is preserved in oral traditions such as narratives, songs, poems, or sayings that are passed down through generations. These sometimes describe events such as volcanic eruptions that occurred thousands of years ago, or they may transmit information about ecological relationships and insights about the effects of management. Oral traditions also embody traditional knowledge, attitudes, and insights about phenomena in the natural world.

The Indigenous Maori people in the Waikato region of New Zealand (Figure P.3) have at least 19 ancestral sayings that pertain to New Zealand flax, or harakeke, a culturally important plant of freshwater wetlands and coastal habitats. Some of these sayings relate to ecological relationships of the flax. One expresses the relationship between flax and the kākā, an endangered, nectar-eating parrot that pollinates it: “Your flax bush … has nurtured the fledgling, and the full-grown kākā.” Others describe the environmental conditions that favor the growth of flax (“When the flax plants are plentiful, it is a sign of much rain”; “the flax is nourished by the dead leaves that fall around its base”) or provide instructions for management (“Clear away the overgrowth so that the flax will put forth many young shoots”). These sayings provide information useful for restoration of wetland ecosystems impacted by drainage, invasions of non-native species, and fragmentation (Wehi, Reference Wehi2009).

Data Recorded by People without Formal Training in Science

Not everyone who contributes to scientific endeavors has professional training in science. Citizen scientists and parataxonomists are two examples. Parataxonomists (Section 12.4.2) – local, often Indigenous, people who are experts in identifying and classifying local flora and fauna on the basis of their observations and experience rather than academic training – are sometimes employed to collect, identify, and preserve specimens for further study by Western scientists. Citizen scientists are voluntary amateurs who participate in the collection of information that is integrated into a database for use in large, scientific studies. Participants contribute data about phenomena such as the identity, locations, and abundance of animals or plants, phenology (climate-related phenomena such as flowering or migration), measurements of water chemistry, or astronomical observations from around the world.

In other contexts, Indigenous peoples have assembled data that challenged scientists’ conclusions about wildlife abundance (Section 12.6). Data produced by Indigenous mapping projects as part of territorial claims have also provided baseline information for assessing ecological change (Nietschmann, Reference Nietschmann1994).

These different kinds of information about the natural world are not mutually exclusive. They can be used to complement one another, with each method suggesting fruitful areas of inquiry that can be pursued using other tools. Regardless of which tools we use to study the natural world, information should be evaluated carefully.