Introduction

Statistical models are called parametric models if they are described by a Euclidean parameter (in a nice way). For instance, the binomial model is described by a single parameter p, and the normal model is given through two unknowns: the mean and the variance of the observations. In many situations there is insufficient motivation for using a particular parametric model, such as a normal model. An alternative at the other end of the scale is a non parametric model, which leaves the underlying distribution of the observations essentially free. In this chapter we discuss one example of a problem of nonparametric estimation: estimating the density of a sample of observations if nothing is known a priori. From the many methods for this problem, we present two: kernel estimation and monotone estimation. Notwithstanding its simplicity, this method can be fully asymptotically efficient.

Kernel Estimators

The most popular nonparametric estimator of a distribution based on a sample of observations is the empirical distribution, whose properties are discussed at length in Chapter 19. This is a discrete probability distribution and possesses no density. The most popular method of nonparametric density estimation, the kernel method, can be viewed as a recipe to “smooth out” the pointmasses of sizes in order to tum the empirical distribution into a continuous distribution.

Let be a random sample from a density f on the real line. If we would know that f belongs to the normal family of densities, then the natural estimate of f would be the normal density with mean and variance or the function

In this section we suppose that we have no prior knowledge of the form of f and want to “let the data speak as much as possible for themselves.“

Let K be a probability density with mean 0 and variance 1, for instance the standard normal density.