Skip to main content Accessibility help
×
Home

Discrete-Time Semi-Markov Random Evolutions and their Applications

  • Nikolaos Limnios (a1) and Anatoliy Swishchuk (a2)

Abstract

In this paper we introduce discrete-time semi-Markov random evolutions (DTSMREs) and study asymptotic properties, namely, averaging, diffusion approximation, and diffusion approximation with equilibrium by the martingale weak convergence method. The controlled DTSMREs are introduced and Hamilton–Jacobi–Bellman equations are derived for them. The applications here concern the additive functionals (AFs), geometric Markov renewal chains (GMRCs), and dynamical systems (DSs) in discrete time. The rates of convergence in the limit theorems for DTSMREs and AFs, GMRCs, and DSs are also presented.

    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Discrete-Time Semi-Markov Random Evolutions and their Applications
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Discrete-Time Semi-Markov Random Evolutions and their Applications
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Discrete-Time Semi-Markov Random Evolutions and their Applications
      Available formats
      ×

Copyright

Corresponding author

Postal address: Laboratoire de Mathématiques Appliquées, Université de Technologie de Compiègne, BP 20529, 60205 Compiègne Cedex, France. Email address: nikolaos.limnios@utc.fr
∗∗ Postal address: Department of Mathematics and Statistics, University of Calgary, 2500 University Drive NW, Calgary, Alberta T2N 1N4, Canada.

References

Hide All
[1] Aase, K. K. (1988). Contingent claims valuation when the security price is a combination of an Itô process and a random point process. Stoch. Process. Appl. 28, 185220.
[2] Adams, R. (1979). Sobolev Spaces. Academic Press, New York.
[3] Altman, E. and Shwartz, A. (1991). Markov decision problems and state-action frequencies. SIAM J. Control Optimization 29, 786809.
[4] Anisimov, V. V. (2008). Switching Processes in Queueing Models. John Wiley, Hoboken, NJ.
[5] Barbu, V. S. and Limnios, N. (2008). Semi-Markov Chains and Hidden Semi-Markov Models. Toward Applications (Lecture Notes Statist. 191). Springer, New York.
[6] Bertsekas, D. P. and Shreve, S. E. (1996). Stochastic Optimal Control. Athena Scientific, Belmont, MA.
[7] Beutler, F. J. and Ross, K. W. (1985). Optimal policies for controlled Markov chains with a constraint. J. Math. Anal. Appl. 112, 236252.
[8] Borkar, V. S. (2003). Dynamic programming for ergodic control with partial observations. Stoch. Process. Appl. 103, 293310.
[9] Boussement, M. and Limnios, N. (2004). Markov decision processes with asymptotic average failure rate constraint. Commun. Statist. Theory Meth. 33, 16891714.
[10] Chiquet, J., Limnios, N. and Eid, M. (2009). Piecewise deterministic Markov processes applied to fatigue crack growth modelling. J. Statist. Planning Infer. 139, 16571667.
[11] Cox, J. C., Ross, S. A. and Rubinstein, M. (1979). Option pricing: a simplified approach. J. Financial Econom. 7, 229263.
[12] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes. John Wiley, New York.
[13] Fleming, W. H. and Rishel, R. W. (1975). Deterministic and Stochastic Optimal Control. Springer, Berlin.
[14] Hersh, R. (1974). Random evolutions: a survey of results and problems. Rocky Mountain J. Math. 4, 443477.
[15] Jacod, J. and Shiryaev, A. N. (1987). Limit Theorems for Stochastic Processes. Springer, Berlin.
[16] Jaśkiewicz, A. and Nowak, A. S. (2007). Average optimality for semi-Markov control processes. Morfismos 11, 1536.
[17] Keepler, M. (1998). Random evolutions processes induced by discrete time Markov chains. Portugal. Math. 55, 391400.
[18] Koroliuk, V. S. and Limnios, N. (2005). Stochastic Systems in Merging Phase Space. World Scientific, Hackensack, NJ.
[19] Korolyuk, V. S. and Swishchuk, A. (1995). Evolution of Systems in Random Media. CRC Press, Boca Raton, FL.
[20] Krylov, N. and Bogolyubov, N. (1947). Introduction to Non-Linear Mechanics. Princeton University Press, Princeton.
[21] Kushner, H. (1971). Introduction to Stochastic Control. Holt, Rinehart and Winston, New York.
[22] Ledoux, M. and Talangrand, M. (1991). Probability in Banach Spaces. Springer, Berlin.
[23] Limnios, N. (2011). Discrete-time semi-Markov random evolutions—average and diffusion approximation of difference equations and additive functionals. Commun. Statist. Theory Meth. 40, 33963406.
[24] Limnios, N. and Oprişan, G. (2001). Semi-Markov Processes and Reliability. Birkhäuser, Boston, MA.
[25] Maxwell, M. and Woodroofe, M. (2000). Central limit theorems for additive functionals of Markov chains. Ann. Prob. 28, 713724.
[26] Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, London.
[27] Nummelin, E. (1984). General Irreducible Markov Chains and Nonnegative Operators. Cambridge University Press.
[28] Pinsky, M. A. (1991). Lectures on Random Evolution. World Scientific, River Edge, NJ.
[29] Pyke, R. (1961). Markov renewal processes: definitions and preliminary properties. Ann. Math. Statist. 32, 12311242.
[30] Pyke, R. (1961). Markov renewal processes with finitely many states. Ann. Math. Statist. 32, 12431259.
[31] Pyke, R. and Schaufele, R. (1964). Limit theorems for Markov renewal processes. Ann. Math. Statist. 35, 17461764.
[32] Revuz, D. (1975). Markov Chains. North-Holland, Amsterdam.
[33] Rudin, W. (1991). Functional Analysis. McGraw-Hill, New York.
[34] Shurenkov, V. M. (1984). On the theory of Markov renewal. Theory Prob. Appl. 19, 247265.
[35] Silvestrov, D. S. (1991). The invariance principle for accumulation processes with semi-Markov switchings in a scheme of arrays. Theory Prob. Appl. 36, 519535.
[36] Silvestrov, D. S. (2004). Limit Theorems for Randomly Stopped Stochastic Processes. Springer, London.
[37] Skorokhod, A. V. (1989). Asymptotic Methods in the Theory of Stochastic Differential Equations. American Mathematical Society, Providence, RI.
[38] Skorokhod, A. V., Hoppensteadt, F. C. and Salehi, H. (2002). Random Perturbation Methods with Applications in Science and Engineering. Springer, New York.
[39] Sobolev, S. L. (1991). Some Applications of Functional Analysis in Mathematical Physics. American Mathematical Society, Providence, RI.
[40] Stroock, D. W. and Varadhan, S. R. S. (1979). Multidimensional Diffusion Processes. Springer, Berlin.
[41] Sviridenko, M. N. (1989). Martingale approach to limit theorems for semi-Markov processes. Theory Prob. Appl. 34, 540545.
[42] Swishchuk, A. V. (1995). Random Evolutions and Their Applications. Kluwer, Dordrecht.
[43] Swishchuk, A. and Islam, M. S. (2010). The geometric Markov renewal processes with application to finance. Stoch. Anal. Appl. 29, 684705.
[44] Swishchuk, A. and Islam, M. S. (2010). Diffusion approximations of the geometric Markov renewal processes and option price formulas. Internat. J. Stoch. Anal. 2010, 347105, 21 pp.
[45] Swishchuk, A. V. and Islam, M. S. (2013). Normal deviation and Poisson approximation of GMRP. To appear in Commun. Statist. Theory Meth.
[46] Swishchuk, A. V. and Limnios, N. (2011). Optimal stopping of GMRP and pricing of European and American options. In Proc. 15th Internat. Congress on Insurance: Mathematics and Economics (Trieste, Italy, June 2011).
[47] Swishchuk, A. and Wu, J. (2003). Evolution of Biological Systems in Random Media: Limit Theorems and Stability. Kluwer, Dordrecht.
[48] Vega-Amaya, O. and Luque-Vásquez, F. (2000). Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times. Appl. Math. 27, 343367.
[49] Yin, G. G. and Zhang, Q. (2005). Discrete-Time Markov Chains. Springer, New York.

Keywords

MSC classification

Discrete-Time Semi-Markov Random Evolutions and their Applications

  • Nikolaos Limnios (a1) and Anatoliy Swishchuk (a2)

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed