Introduction

Continuous-time filtering involves advanced concepts such as Itô calculus and martingale theory which are not covered in this book. This appendix presents the continuous-time HMM filter for three reasons: first, it will be shown that the continuous-time HMM filter can be time-discretized to obtain exactly the discrete-time HMM filter. From a structural point of view, this means that the monotone results for filtering developed in Part III of the book apply to the continuous-time HMM filter. From an algorithm point of view, the practitioner who wishes to numerically implement a continuous-time HMM filter can use the discrete-time HMM filter as an excellent approximation. Second, this discretization procedure reveals an interesting link with adaptive filtering: for sufficiently small discretization intervals, theHMMfilter behaves as a stochastic approximation algorithm. Finally, optimal filters for Markov modulated Poisson processes can be obtained from the continuous-time HMM filter.

Continuous time Markov processes

Let t ∈ [0, T] denotes continuous time. Consider a continuous-time random process {xt} where at each time t, xt is a random variable in the finite state space χ = {e1, e2, …, eX}. Here ei denotes the unit X-vector with 1 in the i-th position. Then the process {xt} is a continuous-time Markov process if the following property holds: given any n time instants, 0 ≤ t1< t2< … < tn < tn+1, the following Markov property holds: where S ⊂χ.

A continuous-time finite-state Markov chain is characterized by the transition rate matrix (infinitesimal generator) denoted as Q: here Q is a X×X matrix and is interpreted in terms of transition probabilities as follows:

Optimal predictor: Fokker–Planck equation: Recall from Chapter 2.2 that in discrete time, the Chapman–Kolmogorov equation characterizes the optimal state predictor. In continuous time, the Fokker–Planck equation (Kolmogorov forward equation) characterizes the optimal predictor. For an X-state homogeneous Markov chain with generator Q and initial distribution π0, define the X-dimensional vector state probability vector πt = [πt(1), …, πt(X)] ′ where πt(i) = ℙ (xt = i).