Elementary rules of probability theory

The aim of this chapter is to prepare the reader to understand the stochastic analysis used to model finance and other macroeconomic data, and to prepare the reader for the analysis of nonstationary time series via statistical ensemble analysis in Chapter 7. This chapter provides the mathematics needed to follow the rest of the text.

In the absence of the laws of physics, which were themselves extracted from nature, the extraction from empirical data is the only scientific basis for a model. We therefore adopt from the start the frequency definition of probability based upon the law of large numbers, Tschebychev's Theorem, which is presented below. The frequency definition of probability is also called the empirical definition, or the statistical definition. Given an event with possible outcomes A1, A2, …, An, the probability for Ak is pk ≈ nk/N where N is the number of repeated identical experiments or observations and nk is the number of times that the event Ak is observed to occur. The statistical definition of probability agrees with the formal measure theoretic definition. For equally probable events p = 1/N.

For mutually exclusive events (Gnedenko and Khinchin, 1962; Gnedenko, 1967) A and B, probabilities add, P(A or B) = P(A) + P(B). For example, the probability that a coin lands heads plus the probability that it does not land heads add to unity (total probability is normalized to unity in this text).