Skip to main content Accessibility help
×
Home

Markov Processes with Restart

  • Konstantin Avrachenkov (a1), Alexey Piunovskiy (a2) and Yi Zhang (a2)

Abstract

We consider a general homogeneous continuous-time Markov process with restarts. The process is forced to restart from a given distribution at time moments generated by an independent Poisson process. The motivation to study such processes comes from modeling human and animal mobility patterns, restart processes in communication protocols, and from application of restarting random walks in information retrieval. We provide a connection between the transition probability functions of the original Markov process and the modified process with restarts. We give closed-form expressions for the invariant probability measure of the modified process. When the process evolves on the Euclidean space, there is also a closed-form expression for the moments of the modified process. We show that the modified process is always positive Harris recurrent and exponentially ergodic with the index equal to (or greater than) the rate of restarts. Finally, we illustrate the general results by the standard and geometric Brownian motions.

    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Markov Processes with Restart
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Markov Processes with Restart
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Markov Processes with Restart
      Available formats
      ×

Copyright

Corresponding author

Postal address: Inria Sophia Antipolis, 2004 Route des Lucioles, Sophia Antipolis, 06902, France. Email address: k.avrachenkov@sophia.inria.fr
∗∗ Postal address: Department of Mathematical Sciences, University of Liverpool, M&O Building, L69 7ZL, UK.
∗∗∗ Email address: piunov@liverpool.ac.uk
∗∗∗∗ Email address: yi.zhang@liverpool.ac.uk

References

Hide All
[1] Alt, H., et al. (1996). A method for obtaining randomized algorithms with small tail probabilities. Algorithmica 16, 543547.
[2] Avrachenkov, K. E., Filar, J. and Haviv, M. (2002). Singular perturbations of Markov chains and decision processes. In Handbook of Markov Decision Processes, eds Feinberg, E. A. and Shwartz, A., Kluwer, Boston, MA, pp. 113150.
[3] Brin, S. and Page, L. (1998). The anatomy of a large-scale hypertextual Web search engine. Comput. Networks ISDN Systems 30, 107117.
[4] Chung, F. (2007). The heat kernel as the pagerank of a graph. Proc. Nat. Acad. Sci. 104, 1973519740.
[5] Down, D., Meyn, S. P. and Tweedie, R. L. (1995). Exponential and uniform ergodicity of Markov processes. Ann. Prob. 23, 16711691.
[6] Feller, W. (1971). An Introduction to Probability Theory and Its Applications, Vol. II, 2nd edn. John Wiley, New York.
[7] Glynn, P. W. (1994). Some topics in regenerative steady-state simulation. Acta Appl. Math. 34, 225236.
[8] González, M. C., Hidalgo, C. A. and Barabási, A.-L. (2008). Understanding individual human mobility patterns. Nature 453, 779782.
[9] Hernández-Lerma, O. and Lasserre, J.-B. (1996). Discrete-time Markov Control Processes, Springer, New York.
[10] Krishnamurthy, B. and Rexford, J. (2001). Web Protocols and Practice: HTTP/1.1, Networking Protocols, Caching, and Traffic Measurement. Addison Wesley.
[11] Kuznetsov, S. E. (1980). Any Markov process in a Borel space has a transition function. Theory Prob. Appl. 25, 384388.
[12] Luby, M., Sinclair, A. and Zuckerman, D. (1993). Optimal speedup of Las Vegas algorithms. Inf. Process. Lett. 47, 173180.
[13] Maurer, S. M. and Huberman, B. A. (2001). Restart strategies and Internet congestion. J. Econom. Dynamics Control 25, 641654.
[14] Meyn, S. P. and Tweedie, R. L. (1993). Stability of Markov processes III. Forster-Lyapunov criteria for continuous-time processes. Adv. Appl. Prob. 25, 518548.
[15] Ross, S. M. (1996). Stochastic Processes, 2nd edn. John Wiley, New York.
[16] Stevens, W. R. (1994). TCP/IP Illustrated, Volume 1: The Protocols. Addison Wesley.
[17] Walsh, P. D., Boyer, D. and Crofoot, M. C. (2010). Monkey and cell-phone-user mobilities scale similarly. Nature Phys. 6, 929930.

Keywords

MSC classification

Markov Processes with Restart

  • Konstantin Avrachenkov (a1), Alexey Piunovskiy (a2) and Yi Zhang (a2)

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.