Skip to main content Accessibility help
×
Home

Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces

  • Quanxin Zhu (a1) and Tomás Prieto-Rumeau (a2)

Abstract

In this paper we study the bias and the overtaking optimality criteria for continuous-time jump Markov decision processes in general state and action spaces. The corresponding transition rates are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. Under appropriate hypotheses, we prove the existence of solutions to the bias optimality equations, the existence of bias optimal policies, and an equivalence relation between bias and overtaking optimality.

    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces
      Available formats
      ×

Copyright

Corresponding author

. Research partially supported by the Natural Science Foundation of China (10626021), the Natural Science Foundation of Guangdong Province (06300957), and CONACYT grant 45693-F.
∗∗ Postal address: Department of Mathematics, South China Normal University, Guangzhou 510631, P. R. China. Email address: zqx22@126.com
∗∗∗ Postal address: Departamento de Estadística, Facultad de Ciencias, Universidad Nacional de Educación a Distancia, Senda del Rey 9, Madrid 28040, Spain. Email address: tprieto@ccia.uned.es

References

Hide All
[1] Arapostathis, A. et al. (1993). Discrete-time controlled Markov processes with average cost criterion: a survey. SIAM J. Control Optimization 31, 282344.
[2] Cao, X. R. (1998). The relations among potentials, perturbation analysis and Markov decision processes. Discrete Event Dyn. Syst. 8, 7187.
[3] Cao, X. R. and Chen, H. F. (1997). Potentials, perturbation realization and sensitivity analysis of Markov processes. IEEE Trans. Automatic Control 42, 13821397.
[4] Guo, X. P. (2007). Continuous-time Markov decision processes with discounted rewards: the case of Polish spaces. Math. Operat. Res. 32, 7387.
[5] Guo, X. P. and Liu, K. (2001). A note on optimality conditions for continuous-time Markov decision processes with average cost criterion. IEEE Trans. Automatic Control 46, 19841984.
[6] Guo, X. P. and Rieder, U. (2006). Average optimality for continuous-time Markov decision processes in Polish spaces. Ann. Appl. Prob. 16, 730756.
[7] Guo, X. P., Hernández-Lerma, O. and Prieto-Rumeau, T. (2006). A survey of recent results on continuous-time Markov decision processes. Top 14, 177261.
[8] Haviv, M. and Puterman, M. L. (1998). Bias optimality in controlled queueing systems. J. Appl. Prob. 35, 136150.
[9] Hernández-Lerma, O. and Lasserre, J. B. (1996). Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer, New York.
[10] Hernández-Lerma, O. and Lasserre, J. B. (1999). Further Topics on Discrete-Time Markov Control Processes. Springer, New York.
[11] Hernández-Lerma, O., Vega-Amaya, O. and Carrasco, G. (1999). Sample-path optimality and variance-minimization of average cost Markov control processes. SIAM J. Control Optimization 38, 7993.
[12] Jasso-Fuentes, H. and Hernández-Lerma, O. (2008). Characterizations of overtaking optimality for controlled diffusion processes. Appl. Math. Optimization 57, 349369.
[13] Jasso-Fuentes, H. and Hernández-Lerma, O. (2008). Ergodic control, bias, and sensitive discount optimality for Markov diffusion processes. To appear in Stoch. Ann. Appl.
[14] Lund, R. B., Meyn, S. P. and Tweedie, R. L. (1996). Computable exponential convergence rates for stochastically ordered Markov processes. Ann. Appl. Prob. 6, 218237.
[15] Prieto-Rumeau, T. and Hernández-Lerma, O. (2005). The Laurent series, sensitive discount and Blackwell optimality for continuous-time controlled Markov chains. Math. Meth. Operat. Res. 61, 123145.
[16] Prieto-Rumeau, T. and Hernández-Lerma, O. (2006). Bias optimality for continuous-time controlled Markov chains. SIAM J. Control Optimization 45, 5173.
[17] Prieto-Rumeau, T. and Hernández-Lerma, O. (2006). Variance minimization and the overtaking optimality approach to continuous-time controlled Markov chains. Submitted.
[18] Puterman, M. L. (1974). Sensitive discount optimality in controlled one-dimensional diffusions. Ann. Prob. 2, 408419.
[19] Puterman, M. L. (1994). Markov Decision Process. John Wiley, New York.
[20] Zhu, Q. X. (2007). Average optimality inequality for continuous-time Markov decision processes in Polish spaces. Math. Meth. Operat. Res. 66, 299313.
[21] Zhu, Q. X. (2008). Average optimality for continuous-time Markov decision processes with a policy iteration approach. J. Math. Analysis Appl. 339, 691704.
[22] Zhu, Q. X. and Guo, X. P. (2005). Another set of conditions for strong n (n=-1,0) discount optimality in Markov decision processes. Stoch. Anal. Appl. 23, 953974.
[23] Zhu, Q. X. and Guo, X. P. (2007). Markov decision processes with variance minimization: a new condition and approach. Stoch. Anal. Appl. 25, 577592.

Keywords

MSC classification

Related content

Powered by UNSILO

Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces

  • Quanxin Zhu (a1) and Tomás Prieto-Rumeau (a2)

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.