Hostname: page-component-77c89778f8-m8s7h Total loading time: 0 Render date: 2024-07-21T05:09:01.570Z Has data issue: false hasContentIssue false
  • English
  • Français

Stochastic and substochastic solutions for infinite-state Markov chains with applications to matrix-analytic methods

Part of: Markov processes Special processes

Published online by Cambridge University Press:  01 July 2016

Winfried K. Grassmann  and
Javad Tavakoli
Winfried K. Grassmann*
Affiliation:
University of Saskatchewan
Javad Tavakoli*
Affiliation:
University of British Columbia Okanagan
*
∗ Postal address: Department of Computer Science, University of Saskatchewan, 101 Science Court, Saskatoon SK S7N 5C9, Canada. Email address: grassman@cs.usask.ca
∗∗ Postal address: Department of Mathematics, Statistics and Physics, University of British Columbia Okanagan, 3333 University Way, Kelowna, Canada V1V 1V7. Email address: javad.tavakoli@ubc.ca
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

This paper deals with censoring of infinite-state banded Markov chains. Censoring involves reducing the time spent in states outside a certain set of states to 0 without affecting the number of visits within this set. We show that, if all states are transient, there is, besides the standard censored Markov chain, a nonstandard censored Markov chain which is stochastic. Both the stochastic and the substochastic solutions are found by censoring a sequence of finite transition matrices. If all matrices in the sequence are stochastic, the stochastic solution arises in the limit, whereas the substochastic solution arises if the matrices in the sequence are substochastic. We also show that, if the Markov chain is recurrent, the only solution is the stochastic solution. Censoring is particularly fruitful when applied to quasi-birth-and-death (QBD) processes. It turns out that key matrices in such processes are not unique, a fact that has been observed by several authors. We note that the stochastic solution is important for the analysis of finite queues.

Keywords

Infinite-state Markov chainmatrix-analytic methodnonrecurrent Markov chainqueueing

MSC classification

Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces) 60K25: Queueing theory
Type
General Applied Probability
Information
Advances in Applied Probability , Volume 40 , Issue 4 , December 2008 , pp. 1157 - 1173
Copyright
Copyright © Applied Probability Trust 2008 

References

Gail, H. R., Hantler, S. L. and Taylor, B. A. (1994). Solution of the basic matrix equation for M/G/1 and G/M/1 type Markov chains. Stoch. Models 10, 1–43.CrossRefGoogle Scholar
Gail, H.R., Hantler, S. L. and Taylor, B. A. (1996). Spectral analysis of M/G/1 and G/M/1 type Markov chains. Adv. Appl. Prob. 28, 114–165.CrossRefGoogle Scholar
Gail, H. R., Hantler, S. L. and Taylor, B. A. (2000). Use of characteristic roots for solving infinite state Markov chains. In Computational Probability, ed. Grassmann, W. K., Kluwer, Boston, MA, pp. 205–254.CrossRefGoogle Scholar
Gohberg, I. Lancaster, P. and Rodman, L. (1982). Matrix Polynomials. Academic Press, New York.Google Scholar
Grassmann, W. K. (1993). Rounding errors in certain algorithms involving Markov chains. ACM Trans. Math. Software 19, 496–508.CrossRefGoogle Scholar
Grassmann, W. K. and Heyman, D. P. (1990). Equilibrium distribution of block-structured Markov chains with repeating rows. J. Appl. Prob. 27, 557–576.Google Scholar
Grassmann, W. K. and Heyman, D. P. (1993). Computation of steady-state probabilities for infinite-state Markov chains with repeating rows. ORSA J. Comput. 5, 292–303.CrossRefGoogle Scholar
Grassmann, W. K. and Stanford, D. A. (2000). Matrix analytic methods. In Computational Probability, ed. Grassmann, W. K., Kluwer, Boston, MA, pp. 153–202.Google Scholar
He, Qi-Ming and Neuts, M. F. (2001). On the convergence and limits of certain matrix sequences arising in quasi-birth-and-death Markov chains. J. Appl. Prob. 38, 519–541.CrossRefGoogle Scholar
Horn, R. A. and Johnson, C. R. (1985). Matrix Analysis. Cambridge University Press.CrossRefGoogle Scholar
Kemeny, J. G., Snell, J. L. and Knapp, A. W. (1966). Denumerable Markov Chains. Van Nostrand, Princeton, NJ.Google Scholar
Latouche, G. (1992). Algorithms for infinite Markov chains with repeating columns. In Linear Algebra, Markov Chains, and Queueing Models (IMA Vols Math. Appl. 48), Springer, Heidelberg, pp. 231–265.Google Scholar
Latouche, G. and Ramaswami, V. (1999). Introduction to Matrix Analytic Methods in Stochastic Modelling. SIAM, Philadelphia, PA.Google Scholar
Latouche, G. and Taylor, P. (2002). Truncation and augmentation of level-independent QBD processes. Stoch. Process. Appl. 99, 53–80.CrossRefGoogle Scholar
Naoumov, V. (1996). Matrix-multiplicative approach to quasi-birth-and-death processes analysis. In Matrix-Analytic Methods in Stoch. Models, Marcel Dekker, New York, pp. 87–106.Google Scholar
Neuts, M. F. (1981). Matrix-Geometric Solutions in Stochastic Models. Johns Hopkins University Press, Baltimore, MD.Google Scholar
Neuts, M. F. (1989). Structured Stochastic Matrices of M/G/1 Type and Their Applications. Marcel Dekker, New York.Google Scholar
O'Cinneide, C. A. (1993). Entrywise perturbabtion theory and error analysis for Markov chains. Numerische Mathematik 65, 109–120.CrossRefGoogle Scholar
O'Cinneide, C. A. (1996). Relative error bounds of the LU decomposition via the GTH algorithm. Numerische Mathematik 73, 507–519.Google Scholar
Stewart, W. (1994). An Introduction to the Numerical Solution of Markov Chains. Princeton University Press.Google Scholar
Wallace, V. (1969). The solution of quasi birth and death processes arising from multiple access computer systems. , University of Michigan.Google Scholar
Wilkinson, J. S. (1965). The Algebraic Eigenvalue Problem. Clarendon Press, Oxford.Google Scholar
Zhao, Y. Q. (2000). Censoring technique in studying block-structured Markov chains. In Advances in Agorithmic Methods for Stochastic Models, eds Latouch, G. and Taylor, P. G., Notable Publications, Neshanic Station, NJ, pp. 417–433.Google Scholar
Zhao, Y. Q., Braun, W. J. and Li, W. (1999). Northwest corner and banded matrix approximation to a countable Markov chain. Naval Res. Logistics 46, 187–197.3.0.CO;2-V>CrossRefGoogle Scholar