Estimation theory is the most unsatisfactory branch of every school on the foundations of statistics. This is partly due to the unfinished state of our science, but there are general reasons for expecting the unhappy condition to continue. The very concept of estimation is ill adapted to statistics.

The theory of statistical estimation includes estimating particular characteristics of distributions and also covers guessing which possible distribution is the true one. It combines point estimation, in which the estimate of a magnitude is a particular point or value, and interval estimation, in which the estimate is a range of values which, it is hoped, will include the true one. Many of the questions about interval estimation are implicitly treated in the preceding chapter. A suitable interval for an interval estimate is one such that the hypothesis, that the interval includes the true values, is itself well supported. So problems about interval estimation fall under questions about measuring support for composite hypotheses. Thus the best known theories on interval estimation have been discussed already. The present chapter will treat some general questions about estimation, but will chiefly be devoted to defining one problem about point estimation. It is not contended that it is the only problem, but it does seem an important one, and one which has been little studied. When the problem has been set, the succeeding chapter will discuss how it might be solved. But neither chapter aims at a comprehensive survey of estimation theory; both aim only at clarifying one central problem about estimation.

Guesses, estimates, and estimators

An estimate, or at least a point estimate, is a more or less soundly based guess at the true value of a magnitude. The word ‘guess’ has a wider application than ‘estimate’, but we shall not rely on the verbal idiosyncracies of either. It must be noted, however, that in the very nature of an estimate, an estimate is supposed to be close to the true value of what it estimates. Since it is an estimate of a magnitude, the magnitude is usually measured along some scale, and this scale can be expected to provide some idea of how close the estimate is.