The importance of screening the data

The first essential of any statistical data analysis is to check that the data are appropriate for the analysis. For frequency analysis, the data collected at a site must be a true representation of the quantity being measured and must all be drawn from the same frequency distribution. An initial screening of the data should aim to verify that these requirements are satisfied.

The exact nature of the problems that may affect the data depend on the kind of data that were measured. For environmental data for which a frequency analysis is being attempted, two kinds of error are particularly important and plausible.

First, data values may be incorrect. Incorrect recording or transcription of data values is easily done and casts doubt on any subsequent frequency analysis of the data.

Second, the circumstances under which the data were collected may have changed over time. The measuring device may have been moved to a different location or trends over time may have arisen from changes in the environment of the measuring device. This means that the frequency distribution from which the data were sampled is not constant over time, and frequency analysis of the data will not be a valid basis for estimating the probability distribution of future measurements at the site.

Even though the data may reputedly be reliable, it is still important to check for errors. A sobering example was provided by Wallis, Lettenmaier, and Wood (1991), who compiled a set of daily precipitation and temperature records for 1009 sites in the United States from data supplied by the National Climatic Data Center (NCDC).