The purpose of this volume is to present the theory of Markov and semi-Markov processes in a discrete-time, finite-state framework. Given this background, hidden versions of these processes are introduced and related estimation and filtering results developed. The approach is similar to the earlier book, Elliott et al. (1995). That is, a central tool is the Radon–Nikodym theorem and related changes of probability measure. In the discrete-time, finite-state framework that we employ these have simple interpretations following from Bayes’ theorem.

Markov chains and hidden Markov chains have found many applications in fields from finance, where the chains model different economic regimes, to genomics, where gene and protein structure is modelled as a hidden Markov model. Semi-Markov chains and hidden semi-Markov chains will have similar, possibly more realistic, applications. The genomics applications are modelled by discrete observations of these hidden chains.

Recent books in the area include in particular Koski (2001) and Barbu and Limnios (2008). Koski includes many examples, not much theory and little on semi-Markov Models. Barbu and Limnios say that the estimation of discrete-time semi-Markov systems is almost absent from the literature. They present an alternative specification from the one adopted in this book and so we give alternative methods in a rigorous framework. They provide limited applications in genomics.

This book carefully constructs relevant processes and proves required results. The filters and related parameter estimation methods we obtain for semi-Markov chains include new results. The occupation times in any state of a Markov chain are geometrically distributed; semi-Markov chains can have occupation times which are quite general and not necessarily geometrically distributed.

Works on semi-Markov processes include Barbu and Limnios (2008), C， inlar (1975), Harlamov (2008), Howard (1971), Janssen and Manca (2010), and Koski (2001) from Chapter 11 onwards. C， inlar (1975) considers a countable state space.

Hidden Markov models have found extensive applications in speech processing and genomics. References for these applications include Ferguson (1980), who considers more general occupation times. This problem was also investigated by Levinson (1986a,b), Ramesh and Wilpon (1992), and in the papers Guédon (1992) and Guédon and Cocozza-Thivent (1990). Genomic applications are treated in the thesis of Burge (1997) and the book Burge and Karlin (1997).