Introduction

We consider a set of independent and identically distributed (i.i.d.) random variables X1, …, Xn following a probability law Pθ, the form of which is known up to some associated parameter θ, appearing as a constant. Both the Xi and θ can be vector-valued. The set X of possible values of the sample X1, …, Xn is called the sample space. We assume that θ ∈ Θ, the parameter space, so that a parametric family of probability laws may be represented as Pθ = {Pθ; θ ∈ Θ}. One of the objectives of statistical estimation theory is to develop methods of choosing appropriate statistics Tn = T(X1, …, Xn), that is, functions of the sample observations, to estimate θ (i.e., to guess the true value of θ) in a reproducible way. In this context, Tn is an estimator of θ.

In an alternative (nonparametric) setup, we may allow the probability law P to be a member of a general class P, not necessarily indexed by some parameter, and then our interest lies in estimating the probability law P itself or some functional θ(P) thereof, without specifying the form of P.

There may be many variations of this setup wherein the Xi may not be independent nor identically distributed as in multisample models, linear models, time-series models, stochastic processes, or unequal probability sampling models.