The purpose of this chapter is to consider various methods for constructing ‘good’ estimators for the unknown parameters θ. The methods to be discussed are the least-squares method, the method of moments and the maximum likelihood method. These three methods played an important role in the development of statistical inference from the early nineteenth century to the present day. The historical background is central to the discussion of these methods because they were developed in response to the particular demands of the day and in the context of different statistical frameworks. If we consider these methods in the context of the present-day framework of a statistical model as developed above we lose most of the early pioneers' insight and the resulting anachronism can lead to misunderstanding. The method developed in relation to the contemporary statistical model framework is the maximum likelihood method attributed to Fisher (1922). The other two methods will be considered briefly in relation to their historical context in an attempt to delineate their role in contemporary statistical inference and in particular their relation to the method of maximum likelihood.

The method of maximum likelihood will play a very important role in the discussion and analysis of the statistical models considered in Part IV; a sound understanding of this method will be of paramount importance. After the discussion of the concepts of the likelihood function, maximum likelihood estimator (MLE) and score function we go on to discuss the properties of MLE's.