In Chapter 4 we saw that the second-order moments of a random process – the mean and covariance or, equivalently, the autocorrelation – play a fundamental role in describing the relation of limiting sample averages and expectations. We also saw, for example in Section 4.6.1 and Problem 4.26, that these moments also play a key role in signal processing applications of random processes, especially in linear least squares estimation. Because of the fundamental importance of these particular moments, this chapter considers their properties in greater depth and their evaluation for several important examples. A primary focus is on a second-order moment analog of a derived distribution problem. Suppose we are given the second-order moments of one random process and this process is then used as an input to a linear system. What are the resulting second-order moments of the output random process? These results are collectively known as second-order moment input/output or I/O relations for linear systems.

Linear systems may seem to be a very special case. As we will see, their most obvious attribute is that they are easier to handle analytically, which leads to more complete, useful, and stronger results than can be obtained for the class of all systems. This special case, however, plays a central role and is by far the most important class of systems.