Hostname: page-component-848d4c4894-nmvwc Total loading time: 0 Render date: 2024-06-28T13:07:22.134Z Has data issue: false hasContentIssue false
  • English
  • Français

Zero-sum games for continuous-time jump Markov processes in Polish spaces: discounted payoffs

Part of: Game theory Markov processes

Published online by Cambridge University Press:  01 July 2016

Xianping Guo  and
Onésimo Hernández-Lerma
Xianping Guo*
Affiliation:
Zhongshan University
Onésimo Hernández-Lerma*
Affiliation:
CINVESTAV-IPN
*
∗ Postal address: School of Mathematics and Computational Science, Zhongshan University, Guangzhou, 510275, P. R. China. Email address: mcsgxp@mail.sysu.edu.cn
∗∗ Postal address: Department of Mathematics, CINVESTAV-IPN, A. Postal 14-740, México D.F. 07000, Mexico. Email address: ohernand@math.cinvestav.mx
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

This paper is devoted to the study of two-person zero-sum games for continuous-time jump Markov processes with a discounted payoff criterion. The state and action spaces are all Polish spaces, the transition rates are allowed to be unbounded, and the payoff rates may have neither upper nor lower bounds. We give conditions on the game's primitive data under which the existence of a solution to the Shapley equation is ensured. Then, from the Shapley equation, we obtain the existence of the value of the game and of a pair of optimal stationary strategies using the extended infinitesimal operator associated with the transition function of a possibly nonhomogeneous continuous-time jump Markov process. We also provide a recursive way of computing (or at least approximating) the value of the game. Moreover, we present a ‘martingale characterization’ of a pair of optimal stationary strategies. Finally, we apply our results to a controlled birth and death system and a Schlögl first model, and then we use controlled Potlach processes to illustrate our conditions.

Keywords

Zero-sum gamejump Markov processpair of optimal stationary strategiesmartingale characterization

MSC classification

Primary: 91A15: Stochastic games 91A25: Dynamic games 60J27: Continuous-time Markov processes on discrete state spaces
Type
General Applied Probability
Information
Advances in Applied Probability , Volume 39 , Issue 3 , September 2007 , pp. 645 - 668
Copyright
Copyright © Applied Probability Trust 2007 

Footnotes

Research supported by the NSFC and RFDP.

Research partially supported by CONACyT Grant 45693-F.

References

Anderson, W. J. (1991). Continuous-Time Markov Chains. Springer, New York.CrossRefGoogle Scholar
Borkar, V. S. and Ghosh, M. K. (1992). Stochastic differential games: an occupation measure based approach. J. Optimization Theory Appl. 73, 359–385. (Correction: 88 (1996), 251–252.)CrossRefGoogle Scholar
Chen, M. F. (2004). From Markov Chains to Non-Equilibrium Particle Systems, 2nd edn. World Scientific, Singapore.CrossRefGoogle Scholar
Fan, K. (1953). Minimax theorems. Proc. Nat. Acad. Sci. USA 39, 42–47.Google Scholar
Feller, W. (1940). On the integro-differential equations of purely discontinuous Markoff processes. Trans. Amer. Math. Soc. 48, 488–515.Google Scholar
Filar, J. A. and Vrieze, K. (1997). Competitive Markov Decision Processes. Springer, New York.Google Scholar
Ghosh, M. K. and Kumar, K. S. (2002). A stochastic differential game in the orthant. J. Math. Appl. Appl. 265, 12–37.Google Scholar
Gihman, I. I. and Skorohod, A. V. (1979). Controlled Stochastic Processes. Springer, New York.CrossRefGoogle Scholar
Guo, X. P. (2007). Continuous-time Markov decision processes with discounted rewards: the case of Polish spaces. Math. Operat. Res. 32, 73–87.Google Scholar
Guo, X. P. and Hernández-Lerma, O. (2003). Continuous-time controlled Markov chains. Ann. Appl. Prob. 13, 363–388.Google Scholar
Guo, X. P. and Hernández-Lerma, O. (2003). Drift and monotonicity conditions for continuous-time controlled Markov chains an average criterion. IEEE Trans. Automatic Control 48, 236–245.Google Scholar
Guo, X. P. and Hernández-Lerma, O. (2003). Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates. J. Appl. Prob. 40, 327–345.Google Scholar
Guo, X. P. and Hernández-Lerma, O. (2005). Zero-sum continuous-time Markov games with unbounded transition and discounted payoff rates. Bernoulli 11, 1009–1029.Google Scholar
Hernández-Lerma, O. and Lasserre, J. B. (1999). Further Topics on Discrete-Time Markov Control Processes. Springer, New York.Google Scholar
Hernández-Lerma, O. and Lasserre, J. B. (2001). Zero-sum stochastic games in Borel spaces: average payoff criterion. SIAM J. Control Optimization 39, 1520–1539.CrossRefGoogle Scholar
Jaśkiewicz, A. (2002). Zero-sum semi-Markov games. SIAM J. Control Optimization 41, 723–739.CrossRefGoogle Scholar
Kushner, H. J. (2003). Numerical approximations for stochastic differential games: the ergodic case. SIAM J. Control Optimization 42, 1911–1911.Google Scholar
Lai, H.-C. and Tanaka, K. (1984). On an N-person noncooperative Markov game with a metric state space. J. Math. Anal. Appl. 101, 78–96.Google Scholar
Lal, A. K. and Sinha, S. (1992). Zero-sum two-person semi-Markov games. J. Appl. Prob. 29, 56–72.Google Scholar
Lund, R. B., Meyn, S. P. and Tweedie, R. L. (1996). Computable exponential convergence rates for stochastically ordered Markov processes. Ann. Appl. Prob. 6, 218–237.Google Scholar
Nowak, A. S. (1985). Measurable selection theorems for minimax stochastic optimization problems. SIAM J. Control Optimization 23, 466–476.Google Scholar
Nowak, A. S. (2000). Some remarks on equilibria in semi-Markov games. Appl. Math. (Warsaw) 27, 385–394.Google Scholar
Parthasarathy, T. and Sinha, S. (1989). Existence of equilibrium stationary strategies in nonzero-sum discounted stochastic games with uncountable state space, and state independent transitions. Internat. J. Game Theory 18, 189–194.Google Scholar
Puterman, M. L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley, New York.Google Scholar
Rao, M. M. (1995). Stochastic Processes: General Theory. Kluwer, Dordrecht.CrossRefGoogle Scholar
Rieder, U. (1978). On semi-continuous dynamic games. Preprint, Abt. für Mathematik VII, Universität Ulm.Google Scholar
Schlögl, F. (1972). Chemical reaction models for phase transition. Z. Phys. 253, 147–161.CrossRefGoogle Scholar
Sennott, L. I. (1994). Zero-sum stochastic games with unbounded cost: discounted and average cost cases. Z. Operat. Res. 39, 209–225.Google Scholar
Vega-Amaya, O. (2003). Zero-sum average semi-Markov games: fixed-point solutions of the Shapley equation. SIAM J. Control Optimization 42, 1876–1984.Google Scholar