Central limit theorems

The Gaussian (or “Normal”) distribution correctly describes an amazing variety of phenomena, not at the “microscopic” level of single events but rather at the “macroscopic” or statistical level (see Figure 1.3). These bell-shaped curves appear in nature ubiquitously due to the wide applicability of the central limit theorem, which states that the distribution for the sum of a large number of statistically independent and identically distributed random variables that have a finite variance converges to a Gaussian.

Khinchin [181], in his renowned book on the foundation of equilibrium statistical mechanics, based his arguments on (1) ergodic theory and (2) the central limit theorem. The necessary and sufficient conditions for the theorem to hold are sweeping, which explains the ubiquitous, but not universal (e.g., see [412]), finding of Gaussian distributions. Even the Maxwell-Boltzmann distribution of velocities of gas particles corresponds to a special case of the Gaussian distribution (for the velocity vectors of the particles).

We do not include here a proof of the central limit theorem due to its wide availability elsewhere. Given the fundamental importance of the theorem, however, we briefly outline the main ideas involved. Consider the sum S of N independent and identically distributed random variables with zero mean and unit variance. Recall that the probability density function of S equals the Fourier convolution of the N individual probability density functions for the N random variables.