Hostname: page-component-76fb5796d-dfsvx Total loading time: 0 Render date: 2024-04-27T16:59:06.421Z Has data issue: false hasContentIssue false
  • English
  • Français

Recursive filters for partially observable finite Markov chains

Part of: Stochastic systems and control Markov processes

Published online by Cambridge University Press:  14 July 2016

James Ledoux
James Ledoux*
Affiliation:
Centre de Mathématiques INSA and IRMAR, Rennes
*
∗Postal address: Centre de Mathématiques INSA, 20 avenue des Buttes de Coësmes, CS 14315, 35043 Rennes cedex, France. Email address: james.ledoux@insa-rennes.fr
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

In this note, we consider discrete-time finite Markov chains and assume that they are only partly observed. We obtain finite-dimensional normalized filters for basic statistics associated with such processes. Recursive equations for these filters are derived by means of simple computations involving conditional expectations. An application to the estimation of parameters of the so-called discrete-time batch Markovian arrival process is outlined.

Keywords

Time-inhomogeneous Markov chainhidden Markov chainD-BMAPEM-algorithm

MSC classification

Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces) 93E11: Filtering
Type
Research Papers
Information
Journal of Applied Probability , Volume 42 , Issue 3 , September 2005 , pp. 684 - 697
Copyright
© Applied Probability Trust 2005 

References

Alfa, A. S. and Neuts, M. (1995). Modelling vehicular traffic using the discrete time Markovian arrival process. Transportation Sci. 29, 109–117.Google Scholar
Asmussen, S. (2000). Matrix-analytic models and their analysis. Scand. J. Statist. 27, 193–226.Google Scholar
Aström, K. J. (1965). Optimal control of Markov processes with incomplete state information. J. Math. Anal. Appl. 10, 174–205.Google Scholar
Blondia, C. (1993). A discrete-time batch Markovian arrival process as B-ISDN traffic model. Belgian J. Operat. Res. Statist. Comput. Sci. 32, 3–23.Google Scholar
Boucheron, S. and Gassiat, E. (2005). An information theoretic perspective on order estimation. In Inference in Hidden Markov Models, eds Cappé, O., Moulines, E. and Rydén, T. Springer, New York.Google Scholar
Breuer, L. (2002). An EM algorithm for batch arrival processes and its comparison to a simpler estimation procedure. Ann. Operat. Res. 112, 123–138.Google Scholar
Elliott, R. J. (1995). Recursive estimation for hidden Markov models: a dependent case. Stoch. Anal. Appl. 13, 437–460.Google Scholar
Elliott, R. J. and Huang, H. (1994). How to count and guess well: discrete adaptative filters. Appl. Math. Optimization 30, 51–78.Google Scholar
Elliott, R. J., Aggoun, L. and Moore, J. B. (1995). Hidden Markov Models (Appl. Math. (New York) 29). Springer, New York.Google Scholar
Ephraim, Y. and Merhav, N. (2002). Hidden Markov processes. IEEE Trans. Inf. Theory 48, 1518–1569.Google Scholar
Gassiat, E. and Boucheron, S. (2003). Optimal error exponents for hidden Markov model order estimation. IEEE Trans. Inf. Theory 49, 864–880.Google Scholar
Klemm, A., Lindemann, C. and Lohmann, M. (2003). Modeling IP traffic using the batch Markovian arrival process. Performance Evaluation 54, 149–173.Google Scholar
Lin, D. and Makis, V. (2003). Filters for a partially observable system subject to random failure. Adv. Appl. Prob. 35, 207–227.Google Scholar
Neuts, M. F. (1989). Structured Stochastic Matrices of M/G/1 Type and Their Applications. Marcel Dekker, New York.Google Scholar
Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77, 257–286.Google Scholar
Rydén, T. (1996). An EM algorithm for estimation in Markov-modulated Poisson processes. Comput. Statist. Data Anal. 21, 431–447.Google Scholar
Rydén, T. (1997). Estimating the order of continuous phase-type distributions and Markov-modulated Poisson processes. Stoch. Models 13, 417–433.Google Scholar
Segall, A. (1976). Recursive estimation from discrete-time point processes. IEEE Trans. Inf. Theory 22, 422–431.Google Scholar