Hostname: page-component-8448b6f56d-c47g7 Total loading time: 0 Render date: 2024-04-18T10:25:19.626Z Has data issue: false hasContentIssue false
  • English
  • Français

Ladder epochs and ladder chain of a Markov random walk with discrete driving chain

Part of: Special processes Markov processes

Published online by Cambridge University Press:  01 February 2019

Gerold Alsmeyer
Gerold Alsmeyer*
Affiliation:
University of Münster
*
Institute of Mathematical Stochastics, Department of Mathematics and Computer Science, University of Münster, Orléans-Ring 10, D-48149 Münster, Germany. Email address: gerolda@math.uni-muenster.de
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Let (Mn,Sn)n≥0 be a Markov random walk with positive recurrent driving chain (Mn)n≥0 on the countable state space 𝒮 with stationary distribution π. Suppose also that lim supn→∞Sn=∞ almost surely, so that the walk has almost-sure finite strictly ascending ladder epochs σn>. Recurrence properties of the ladder chain (Mσn>)n≥0 and a closely related excursion chain are studied. We give a necessary and sufficient condition for the recurrence of (Mσn>)n≥0 and further show that this chain is positive recurrent with stationary distribution π> and 𝔼π>σ1><∞ if and only if an associated Markov random walk (𝑀̂n,𝑆̂n)n≥0, obtained by time reversal and called the dual of (Mn,Sn)n≥0, is positive divergent, i.e. 𝑆̂n→∞ almost surely. Simple expressions for π> are also provided. Our arguments make use of coupling, Palm duality theory, and Wiener‒Hopf factorization for Markov random walks with discrete driving chain.

Keywords

Markov‐modulated sequenceMarkov random walkdiscrete Markov chainladder variableladder chainexcursion chainstationary distributioncouplingPalm‐dualityWiener‒Hopf factorization

MSC classification

Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 60K15: Markov renewal processes, semi-Markov processes
Type
Original Article
Information
Advances in Applied Probability , Volume 50 , Issue A: Branching and Applied Probability , December 2018 , pp. 31 - 46
Copyright
Copyright © Applied Probability Trust 2018 

References

[1]Alsmeyer, G. (2000).The ladder variables of a Markov random walk.Prob. Math. Statist. 20,151–168.Google Scholar
[2]Alsmeyer, G. (2001).Recurrence theorems for Markov random walks.Prob. Math. Statist. 21,123–134.Google Scholar
[3]Alsmeyer, G. (2014).Quasistochastic matrices and Markov renewal theory. In Celebrating 50 Years of The Applied Probability Trust (J. Appl. Prob. 51A), eds S. Asmussen, P. Jagers, I. Molchanov and L. C. G. Rogers,Applied Probability Trust,Sheffield, pp. 359–376.Google Scholar
[4]Alsmeyer, G. and Buckmann, F. (2018).Fluctuation theory for Markov random walks.J. Theoret. Prob. 31,2266–2342.Google Scholar
[5]Arjas, E. and Speed, T. P. (1973).Symmetric Wiener-Hopf factorisations in Markov additive processes.Z. Wahrscheinlichkeitsth. 26,105–118.Google Scholar
[6]Arjas, E. and Speed, T. P. (1973).Topics in Markov additive processes.Math. Scand. 33,171–192.Google Scholar
[7]Asmussen, S. (1989).Aspects of matrix Wiener-Hopf factorisation in applied probability.Math. Sci. 14,101–116.Google Scholar
[8]Asmussen, S. (2003).Applied Probability and Queues,2nd edn.Springer,New York.Google Scholar
[9]Çinlar, E. (1969).Markov renewal theory.Adv. Appl. Prob. 1,123–187.Google Scholar
[10]Çinlar, E. (1974/75).Markov renewal theory: a survey.Manag. Sci. 21,727–752.Google Scholar
[11]Erickson, K. B. (1971).A renewal theorem for distributions on R 1 without expectation.Bull. Amer. Math. Soc. 77,406–410.Google Scholar
[12]Fuh, C. D. and Lai, T. L. (1998).Wald's equations, first passage times and moments of ladder variables in Markov random walks.J. Appl. Prob. 35,566–580.Google Scholar
[13]Korolyuk, V. S. and Turbin, A. F. (1976).Semi-Markov Processes and Their Applications.Izdat. `Naukova Dumka',Kiev (in Russian).Google Scholar
[14]Lalley, S. P. (1986).Renewal theorem for a class of stationary sequences.Prob. Theory Relat. Fields 72,195–213.Google Scholar
[15]Meyn, S. and Tweedie, R. L. (2009).Markov Chains and Stochastic Stability,2nd edn.Cambridge University Press.Google Scholar
[16]Prabhu, N. U.,Tang, L. C. and Zhu, Y. (1993).Some new results for the Markov random walk.J. Math. Phys. Sci. 25,635–663.Google Scholar
[17]Presman, È. L. (1969).Factorization methods, and a boundary value problem for sums of random variables given on a Markov chain.Izv. Akad. Nauk SSSR Ser. Mat. 33,861–900.Google Scholar
[18]Sigman, K. (1995).Stationary Marked Point Processes: An Intuitive Approach.Chapman & Hall,New York.Google Scholar
[19]Thorisson, H. (2000).Coupling, Stationarity, and Regeneration.Springer,New York.Google Scholar