Part I of the book deals with the filtering problem.

Chapter 2 describes the partially observed Markov process model (stochastic state space model) and the Chapman–Kolmogorov equation for prediction.

Chapter 3 discusses the filtering problem and filtering algorithms – particularly the Kalman filter and HMM filter and their properties, and also a short discussion on suboptimal filtering algorithms such as the particle filter.

In Chapter 4, the filters are used to compute the maximum likelihood parameter estimate for HMMs via general purpose optimization algorithms and also the Expectation Maximization (EM) algorithm.

Chapter 5 discusses social learning and filtering on graphs. It deals with the structure of the filtering recursion rather than computations and is indicative of the rest of the book. We return to such structural results in Part III of the book.

Appendix B discusses filters for continuous-time HMMs and Markov modulated Poisson processes, and their numerical implementation.

The material of Part I forms the background to the rest of the book where controlled partially observed Markov models are considered. Please note that the term “Bayesian” here is consistent with the usage in signal processing. It means that Bayes’ formula is used in computation.

Part I is an introductory treatment of Bayesian filtering and is not meant to be a comprehensive account.We mainly discuss filters for finite state Markov chains in noise (including social learning filters) and, to a lesser extent, filters for linear Gaussian state space processes and jump Markov linear systems.

In Parts II to IV of the book, we will deal with controlling the dynamics of the HMM filter. The reader who wishes to proceed rapidly to Part II only needs to read sections of Chapters 2 and 3 pertaining to finite state Markov chains and HMMs, and parts of Chapter 5 in social learning.