Statistical inference is a mathematical activity, or process, intent on extracting meaningful information from measurements in order to characterize systems, establish best model parameters, or determine the plausibility (truthfulness) of hypotheses and models. Indeed, once experimental data have been acquired, one wishes either to characterize the data by some ad hoc functional form, or compare the measured data distribution to some model predictions. In either case, the functional form may involve “free” parameters not constrained by theory and to be determined by the data. There are also cases where no specific functional form is known a priori and as simple a characterization of the data as possible is sufficient or required. It may be sufficient, for instance, to evaluate the mean, variance, and selected higher moments of the data. Inference also includes the notion of statistical test, that is, the use of statistical techniques to assess the adequacy of scientific models and hypotheses.

Traditionally, statistical inference is a subfield of probability and statistics concerned with the determination of the properties of the underlying distribution of datasets based on statistical analysis techniques largely based on the frequentist interpretation of probability. However, the field of inference has greatly expanded in recent decades, and inference may now be approached from a plurality of paradigms, or schools of thought, corresponding to different interpretations of the notion of probability and approaches to the inference process. We will restrict our discussion to the frequentist and Bayesian paradigms, the former still most commonly used in the physical sciences, and the latter receiving a fast growing level of interest in physical and life sciences, engineering, as well as economics and financial fields. Other commonly less adopted paradigms, including the likelihood paradigm, the total survey error paradigm, and the survey sampling paradigm, are discussed in the statistics literature [15, 96, 167].

Similarities and distinctions between the goals and methods of the frequentist and Bayesian inference paradigms are briefly outlined in § 4.1. The remainder of this chapter as well as Chapters 5 and 6 present detailed discussions of classical inference concepts and techniques while Bayesian inference methods are explored in Chapter 7.