What are we trying to do?

In this chapter we are going to discuss the problem of obtaining the best description of our data in terms of some theory, which involves parameters whose values are initially unknown. Thus we could have data on the number of road accidents per year over the last decade; or we could have measured the length of a piece of metal at different temperatures. In either of these cases, we may be interested to see (i) whether the data lie on a straight line, and if so (ii) what are its gradient and intercept (see Fig. 2.1).

These two questions correspond to the statistics subjects known as Hypothesis Testing and Parameter Fitting. Logically, hypothesis testing precedes parameter fitting, since if our hypothesis is incorrect, then there is no point in determining the values of the free parameters (i.e. the gradient and intercept) contained within the hypothesis. In fact, we will deal with parameter fitting first, since it is easier to understand. In practice, one often does parameter fitting first anyway; it may be impossible to perform a sensible test of the hypothesis before its free parameters have been set at their optimum values.

Various methods exist for parameter determination. The one we discuss here is known as least squares. In order to fix our ideas, we shall assume that we have been presented with data of the form shown in Fig. 2.1, and that it corresponds to some measurements of the length of our bar yiobs at various known temperatures xi.