Hostname: page-component-76fb5796d-wq484 Total loading time: 0 Render date: 2024-04-28T14:41:17.453Z Has data issue: false hasContentIssue false
  • English
  • Français

Perturbation theory and finite Markov chains

Published online by Cambridge University Press:  14 July 2016

Paul J. Schweitzer
Paul J. Schweitzer*
Affiliation:
Institute for Defense Analyses, Arlington, Virginia
Get access Rights & Permissions [Opens in a new window]

Abstract

A perturbation formalism is presented which shows how the stationary distribution and fundamental matrix of a Markov chain containing a single irreducible set of states change as the transition probabilities vary. Expressions are given for the partial derivatives of the stationary distribution and fundamental matrix with respect to the transition probabilities. Semi-group properties of the generators of transformations from one Markov chain to another are investigated. It is shown that a perturbation formalism exists in the multiple subchain case if and only if the change in the transition probabilities does not alter the number of, or intermix the various subchains. The formalism is presented when this condition is satisfied.

Type
Research Papers
Information
Journal of Applied Probability , Volume 5 , Issue 2 , August 1968 , pp. 401 - 413
Copyright
Copyright © Applied Probability Trust 1968 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Bharucha-Reid, A. T. (1960) Elements of the Theory of Markov Processes and their Applications. McGraw-Hill, New York, 32–35.Google Scholar
[2] Kemeny, J. G. and Snell, J. L. (1960) Finite Markov Chains. Van Nostrand, New York.Google Scholar
[3] Blackwell, D. (1962) Discrete dynamic programming. Ann. Math. Statist. 33, 719–726.CrossRefGoogle Scholar
[4] Doob, J. L. (1953) Stochastic Processes. Wiley, New York, page 181.Google Scholar