Hostname: page-component-5c6d5d7d68-wp2c8 Total loading time: 0 Render date: 2024-08-15T22:51:32.842Z Has data issue: false hasContentIssue false
  • English
  • Français

Criteria for classifying general Markov chains

Published online by Cambridge University Press:  01 July 2016

R. L. Tweedie
R. L. Tweedie*
Affiliation:
C.S.I.R.O. Division of Mathematics and Statistics, Canberra
Get access Rights & Permissions [Opens in a new window]

Abstract

The aim of this paper is to present a comprehensive set of criteria for classifying as recurrent, transient, null or positive the sets visited by a general state space Markov chain. When the chain is irreducible in some sense, these then provide criteria for classifying the chain itself, provided the sets considered actually reflect the status of the chain as a whole. The first part of the paper is concerned with the connections between various definitions of recurrence, transience, nullity and positivity for sets and for irreducible chains; here we also elaborate the idea of status sets for irreducible chains. In the second part we give our criteria for classifying sets. When the state space is countable, our results for recurrence, transience and positivity reduce to the classical work of Foster (1953); for continuous-valued chains they extend results of Lamperti (1960), (1963); for general spaces the positivity and recurrence criteria strengthen those of Tweedie (1975b).

Keywords

MARKOV CHAINRECURRENCETRANSIENCEPOSITIVE RECURRENCEERGODICITYNULL RECURRENCESTATIONARY MEASUREINVARIANT MEASUREIRREDUCIBILITYSTATUS SETMEAN DRIFT
Type
Research Article
Information
Advances in Applied Probability , Volume 8 , Issue 4 , December 1976 , pp. 737 - 771
Copyright
Copyright © Applied Probability Trust 1976 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

References added in proof

Nummelin, E. and Arjas, E. (1976) A direct construction of the R-invariant measure for a Markov chain on a general state space. Ann. Prob. 4, 674–679.Google Scholar
Revuz, D. (1975) Markov Chains. North-Holland, Amsterdam.Google Scholar
Tuominen, P. and Tweedie, R. L. (1977) Markov chains with continuous components. (Submitted for publication).Google Scholar

References

Blackwell, D. (1945) The existence of anormal chains. Bull. Amer. Math. Soc. 51, 465–468.Google Scholar
Calton, W. G. and Rogers, G. S. (1976) On classifying discrete time Markov processes. (Submitted for publication).Google Scholar
Chow, Y. S. and Robbins, H. (1963) A renewal theorem for random variables which are dependent or non-identically distributed. Ann. Math. Statist. 34, 390–395.Google Scholar
Chung, K. L. (1964) The general theory of Markov processes according to Doeblin. Z. Wahrscheinlichkeitsth. 2, 230–254.Google Scholar
Chung, K. L. (1967) Markov Chains with Stationary Transition Probabilities, 2nd edn. Springer-Verlag, Berlin.Google Scholar
Cogburn, R. (1975) A uniform theory for sums of Markov chain transition probabilities. Ann. Prob. 3, 191–214.Google Scholar
Doob, J. L. (1953) Stochastic Processes. Wiley, New York.Google Scholar
Delbrouck, L. E. N. (1971) On stochastic boundedness and stationary measures for Markov processes. J. Math. Anal. Appl. 33, 149–162.CrossRefGoogle Scholar
Foster, F. G. (1953) On the stochastic processes associated with certain queuing processes. Ann. Math. Statist. 24, 355–360.Google Scholar
Harris, C. M. and Marlin, P. G. (1972) A note on feedback queues with bulk service. J. Assoc. Comp. Mach. 19, 727–733.CrossRefGoogle Scholar
Harris, T. E. (1956) Transient Markov chains with stationary measures. Proc. Amer. Math. Soc. 8, 937–942.Google Scholar
Hordijk, A. and Van Goethem, P. (1973) A criterion for the existence of invariant probability measures in Markov processes. Mathematisch Centrum Prepublication SW 22/73.Google Scholar
Jain, N. C. and Jamison, B. (1967) Contributions to Doeblin's theory of Markov processes. Z. Wahrscheinlichkeitsth. 8, 19–40.CrossRefGoogle Scholar
Kingman, J. F. C. (1961) The ergodic behaviour of random walks. Biometrika 48, 391–396.Google Scholar
Lamperti, J. (1960) Criteria for the recurrence or transience of stochastic processes I. J. Math. Anal. Appl. 1, 314–330.Google Scholar
Lamperti, J. (1963) Criteria for stochastic processes II: passage-time moments. J. Math. Anal. Appl. 7, 127–145.Google Scholar
Laslett, G. M., Pollard, D. B. and Tweedie, R. L. (1976) Techniques for establishing ergodic properties of continuous-valued Markov chains. (Submitted for publication).Google Scholar
Marlin, P. G. (1973) On the ergodic theory of Markov chains. Operat. Res. 21, 617–622.Google Scholar
Mauldon, J. G. (1958) On non-dissipative Markov chains. Proc. Camb. Phil. Soc. 53, 825–835.CrossRefGoogle Scholar
Miller, H. D. (1966) Geometric ergodicity in a class of denumerable Markov chains. Z. Wahrscheinlichkeitsth. 4, 354–373.CrossRefGoogle Scholar
Nelson, E. (1958) The adjoint Markoff process. Duke Math. J. 25, 671–690.Google Scholar
Orey, S. (1971) Lecture Notes on Limit Theorems for Markov Chains on a General State Space. Van Nostrand Reinhold, London.Google Scholar
Pakes, A. G. (1969) Some conditions for ergodicity and recurrence of Markov chains. Operat. Res. 17, 1058–1061.Google Scholar
Pollard, D. B. and Tweedie, R. L. (1975) R-theory for Markov chains on a topological space I. J. Lond. Math. Soc. 10, 389–400.Google Scholar
Pollard, D. B. and Tweedie, R. L. (1976) R-theory for Markov chains on a topological space II. Z. Wahrscheinlichkeitsth. 34, 269–278.Google Scholar
Šidák, Z. (1967) Classification of Markov chains with a general state space. Trans. 4th Prague Conf. Inf. Theory Stat. Dec. Functions, Random Procs. 1965, Academia Prague, 547–571.Google Scholar
Tomášek, L. (1971) On Superregular Functions in Markov Chains and Related Inequalities (Czech.). Unpublished Ph.D. thesis, Charles University, Prague.Google Scholar
Tuominen, P. (1976) Notes on 1-recurrent Markov chains. Z. Wahrscheinlichkeitsth. 36, 111–118.CrossRefGoogle Scholar
Tweedie, R. L. (1974a) R-theory for Markov chains on a general state space I: solidarity properties and R-recurrent chains. Ann. Prob. 2, 840–864.Google Scholar
Tweedie, R. L. (1974b) R-theory for Markov chains on a general state space II: r-subinvariant measures for r-transient chains. Ann. Prob. 2, 865–878.Google Scholar
Tweedie, R. L. (1975a) Sufficient conditions for regularity, recurrence and ergodicity of Markov processes. Math. Proc. Camb. Phil. Soc. 78, 125–136.Google Scholar
Tweedie, R. L. (1975b) Sufficient conditions for ergodicity and recurrence of Markov chains on a general state space. Stoch. Proc. Appl. 3, 385–403.CrossRefGoogle Scholar
Tweedie, R. L. (1975c) The robustness of positive recurrence and recurrence of Markov chains on a general state space. J. Appl. Prob. 12, 744–752.Google Scholar
Tweedie, R. L. (1975d) A relation between positive recurrence and mean drift for Markov chains. Austral. J. Statist. 17, 96–102.Google Scholar
Tweedie, R. L. (1976) Modes of convergence of Markov chain transition probabilities. J. Math. Anal. Appl. (To appear) Google Scholar