Skip to main content Accessibility help
×
Home

Error bounds for augmented truncation approximations of Markov chains via the perturbation method

  • Yuanyuan Liu (a1) and Wendi Li (a1)

Abstract

Let P be the transition matrix of a positive recurrent Markov chain on the integers with invariant probability vector πT, and let (n)P̃ be a stochastic matrix, formed by augmenting the entries of the (n + 1) x (n + 1) northwest corner truncation of P arbitrarily, with invariant probability vector (n)πT. We derive computable V-norm bounds on the error between πT and (n)πT in terms of the perturbation method from three different aspects: the Poisson equation, the residual matrix, and the norm ergodicity coefficient, which we prove to be effective by showing that they converge to 0 as n tends to ∞ under suitable conditions. We illustrate our results through several examples. Comparing our error bounds with the ones of Tweedie (1998), we see that our bounds are more applicable and accurate. Moreover, we also consider possible extensions of our results to continuous-time Markov chains.

Copyright

Corresponding author

* Postal address: School of Mathematics and Statistics, Central South University, Changsha, Hunan, 410075, China.
** Email address: liuyy@csu.edu.cn
*** Email address: liwendi@csu.edu.cn

References

Hide All
[1]Abbas, K., Berkhout, J. and Heidergott, B. (2016). A critical account of perturbation analysis of Markov chains. Markov Process. Relat. Fields 22, 227265.
[2]Altman, E., Avrachenkov, K. E. and Núñez-Queija, R. (2004). Perturbation analysis for denumerable Markov chains with application to queueing models. Adv. Appl. Prob. 36, 839853.
[3]Asmussen, S. (2003). Applied Probability and Queues, 2nd edn. Springer, New York.
[4]Cao, X.-R. (2007). Stochastic Learning and Optimization: A Sensitivity-Based Approach. Springer, New York.
[5]Chen, M. (1999). Single birth processes. Chinese Ann. Math. B 20, 7782.
[6]Cho, G. E. and Meyer, C. D. (2001). Comparison of perturbation bounds for the stationary distribution of a Markov chain. Linear Algebra Appl. 335, 137150.
[7]Gibson, D. and Seneta, E. (1987). Augmented truncations of infinite stochastic matrices. J. Appl. Prob. 24, 600608.
[8]Glynn, P. W. and Meyn, S. P. (1996). A Liapounov bound for solutions of the Poisson equation. Ann. Prob. 24, 916931.
[9]Heidergott, B., Hordijk, A. and Leder, N. (2010). Series expansions for continuous-time Markov processes. Operat. Res. 58, 756767.
[10]Ho, Y.-C. and Cao, X.-R. (1991). Perturbation Analysis of Discrete Event Dynamic Systems. Kluwer, Boston, MA.
[11]Hunter, J. J. (2005). Stationary distributions and mean first passage times of perturbed Markov chains. Linear Algebra Appl. 410, 217243.
[12]Jiang, S., Liu, Y. and Tang, Y. (2017). A unified perturbation analysis framework for countable Markov chains. Linear Algebra Appl. 529, 413440.
[13]Kartashov, N. V. (1985). Criteria for uniform ergodicity and strong stability of Markov chains with a common phase space. Theory Prob. Math. Statist. 30, 7189.
[14]Kartashov, N. V. (1986). Strongly stable Markov chains. J. Soviet Math. 34, 14931498.
[15]Kartashov, N. V. (1996). Strong Stable Markov Chains. VSP, Utrecht.
[16]Kirkland, S. J., Neumann, M. and Sze, N.-S. (2008). On optimal condition numbers for Markov chains. Numer. Math. 110, 521537.
[17]Li, H. and Zhao, Y. (2000). Stochastic block-monotonicity in the approximation of the stationary distribution of infinite Markov chains. Commun. Statist. Stoch. Models 16, 313333.
[18]Liu, Y. (2010). Augmented truncation approximations of discrete-time Markov chains. Operat. Res. Lett. 38, 218222.
[19]Liu, Y. (2012). Perturbation bounds for the stationary distributions of Markov chains. SIAM J. Matrix Anal. Appl. 33, 10571074.
[20]Liu, Y. (2015). Perturbation analysis for continuous-time Markov chains. Sci. China Math. 58, 26332642.
[21]Liu, Y. and Zhang, Y. (2015). Central limit theorems for ergodic continuous-time Markov chains with applications to single birth processes. Front. Math. China 10, 933947.
[22]Liu, Y., Tang, Y. and Zhao, Y. (2015). Censoring technique and numerical computations of invariant distribution for continuous-time Markov chains. Scientia Sinica Math. 45, 671682 (in Chinese).
[23]Mao, Y., Zhang, M. and Zhang, Y. (2013). A generalization of Dobrushin coefficient. Chinese J. Appl. Prob. Statist. 29, 489494.
[24]Masuyama, H. (2015). Error bounds for augmented truncations of discrete-time block-monotone Markov chains under geometric drift conditions. Adv. Appl. Prob. 47, 83105.
[25]Masuyama, H. (2016). Error bounds for augmented truncations of discrete-time block-monotone Markov chains under subgeometric drift conditions. SIAM J. Matrix Anal. Appl. 37, 877910.
[26]Masuyama, H. (2017). Continuous-time block-monotone Markov chains and their block-augmented truncations. Linear Algebra Appl. 514, 105150.
[27]Masuyama, H. (2017). Error bounds for last-column-block-augmented truncations of block-structured Markov chains. J. Operat. Res. Soc. Japan 60, 271320.
[28]Meyer, C. D. (1994). Sensitivity of the stationary distribution of a Markov chain. SIAM J. Matrix Anal. Appl. 15, 715728.
[29]Meyn, S. and Tweedie, R. L. (2009). Markov Chains and Stochastic Stability, 2nd edn. Cambridge University Press.
[30]Mitrophanov, A. Y. (2005). Sensitivity and convergence of uniformly ergodic Markov chains. J. Appl. Prob. 42, 10031014.
[31]Mouhoubi, Z. and Aïssani, D. (2010). New perturbation bounds for denumerable Markov chains. Linear Algebra Appl. 432, 16271649.
[32]Neuts, M. F. (1989). Structured Stochastic Matrices of M/G/1 Type and Their Aapplications. Marcel Dekker, New York.
[33]Seneta, E. (1979). Coefficients of ergodicity: structure and applications. Adv. Appl. Prob. 11, 576590.
[34]Seneta, E. (1980). Computing the stationary distribution for infinite Markov chains. Linear Algebra Appl. 34, 259267.
[35]Tweedie, R. L. (1998). Truncation approximations of invariant measures for Markov chains. J. Appl. Prob. 35, 517536.
[36]Van Dijk, N. M. (2011). Error bounds and comparison results: the Markov reward approach for queueing networks. In Queueing Networks (Internat. Ser. Operat. Res. Manag. Sci. 154), Springer, New York, pp. 397459.

Keywords

MSC classification

Error bounds for augmented truncation approximations of Markov chains via the perturbation method

  • Yuanyuan Liu (a1) and Wendi Li (a1)

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed