Hostname: page-component-77c89778f8-5wvtr Total loading time: 0 Render date: 2024-07-17T04:49:21.501Z Has data issue: false hasContentIssue false
  • English
  • Français

Discretizations for the average impulse control of piecewise deterministic processes

Part of: Markov processes Stochastic systems and control

Published online by Cambridge University Press:  14 July 2016

O. L. V. Costa
O. L. V. Costa*
Affiliation:
Universidade de São Paulo
*
∗ Postal address: Departamento de Engenharia Eletrônica, Escola Politécnica da Universidade de São Paulo, 05508 900 São Paulo SP Brazil.
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper presents a state space and time discretization for the general average impulse control of piecewise deterministic Markov processes (PDPs). By combining several previous results we show that under some continuity, boundedness and compactness conditions on the parameters of the process, boundedness of the discretizations, and compactness of the state space, the discretized problem will converge uniformly to the original one. An application to optimal capacity expansion under uncertainty is given.

Keywords

LONG-RUN COSTUNIFORM CONVERGENCE APPROXIMATION METHODSOPTIMIZATION PROBLEMSMARKOV DECISION MODELS'CAPACITY EXPANSION

MSC classification

Primary: 60J25: Continuous-time Markov processes on general state spaces
Secondary: 90C40: Markov and semi-Markov decision processes 93E20: Optimal stochastic control
Type
Research Papers
Information
Journal of Applied Probability , Volume 30 , Issue 2 , June 1993 , pp. 405 - 420
Copyright
Copyright © Applied Probability Trust 1993 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This research was partially supported by the National Research Council — CNPq, Brazil, under grant 305173/88–0/EE/FV, and FAPESP (Research Council of the State of São Paulo), under grant 91/0508–3.

References

[1] Bertsekas, D. and Shreve, S.E. (1978) Stochastic Optimal Control: The Discrete-Time Case. Academic Press, New York.Google Scholar
[2] Costa, O. L. V. (1989) Average impulse control of piecewise-deterministic processes. IMA J. Math. Control Inf. 6, 375–397.CrossRefGoogle Scholar
[3] Costa, O. L. V. (1991) Asymptotic convergence for the general average impulse control of piecewise deterministic processes. IMA J. Math. Control Inf. 8, 1–27.Google Scholar
[4] Costa, O. L. V. (1991) Impulse control of piecewise-deterministic processes via linear programming. IEEE Trans. Autom. Control. 36, 371–375.CrossRefGoogle Scholar
[5] Costa, O. L. V. and Davis, M. H. A. (1988) Approximations for optimal stopping of a piecewise-deterministic process. Math. Control. Sig. Syst. 1, 123–146.CrossRefGoogle Scholar
[6] Costa, O. L. V. and Davis, M. H. A. (1989) Impulse control of piecewise-deterministic processes. Math. Control. Sig. Syst. 2, 187–206.CrossRefGoogle Scholar
[7] Davis, M. H. A. (1984) Piecewise-deterministic Markov processes: a general class of non-diffusion stochastic models. J. R. Statist. Soc. B46, 353–388.Google Scholar
[8] Davis, M. H. A. (1985) Control of piecewise-deterministic processes via discrete-time dynamic programming, Proc. 3rd Bad Honnef Conf. pp. 140–150, Lecture Notes in Control and Information Sciences, Springer-Verlag, Berlin.Google Scholar
[9] Davis, M. H. A., Dempster, M. A. H., Sethi, S. P. and Vermes, D. (1987) Optimal capacity expansion under uncertainty. Adv. Appl. Prob. 19, 156–176.Google Scholar
[10] De Leve, G., Federgruen, A. and Tijms, H. C. (1977) A general Markov decision method I: Model and Techniques; II: Applications. Adv. Appl. Prob. 9, 296–335.Google Scholar
[11] Gatarek, D. (1990) On the value functions for impulsive control of piecewise-deterministic processes. Stochastics 32, 27–52.Google Scholar
[12] Gatarek, D. and Stettner, L. (1992) On the compactness method in general ergodic impulsive control of Markov processes. Stochastics. To appear.Google Scholar
[13] Gugerli, U. S. (1986) Optimal stopping of a piecewise-deterministic Markov process. Stochastics 19, 221–236.Google Scholar
[14] Hordijk, A. and van der Duyn Schouten, F. A. (1984) Discretization and weak convergence in Markov decision drift processes. Math. Operat. Res. 9, 112–141.CrossRefGoogle Scholar
[15] Hordijk, A. and van der Duyn Schouten, F. A. (1983) Average optimal policies in Markov decision drift processes with applications to a queueing and a replacement model. Adv. Appl. Prob. 15, 274–303.Google Scholar
[16] Lenhart, S. M. and Liao, Y. C. (1985) Integro-differential equations associated with optimal stopping of a piecewise deterministic process. Stochastics 15, 183–208.CrossRefGoogle Scholar
[17] Lepeltier, J. P. and Marchal, B. (1984) Théorie générale du contrôle impulsionnel Markovien. SIAM J. Control Optim. 22, 645–665.CrossRefGoogle Scholar
[18] Munroe, M. ?. (1964) Introductory Real Analysis. Addison-Wesley, Reading, MA.Google Scholar
[19] Robin, M. (1981) On some impulsive control problems with long-run average cost. SIAM J. Control Optim. 19, 333–358.Google Scholar
[20] Ross, S. (1970) Applied Probability Models with Optimization Applications. Holden-Day, San Francisco.Google Scholar
[21] Soner, M. (1986) Optimal control with state space constraint. SIAM J. Control Optim. 24, 1110–1122.CrossRefGoogle Scholar
[22] Stettner, L. (1983) On impulsive control with long run average cost criterion. Studia Math. 76, 279–298.CrossRefGoogle Scholar
[23] Stettner, L. (1986) On the Poisson equation and optimal stopping of ergodic Markov processes. Stochastics 18, 25–48.Google Scholar
[24] Stettner, L. (1986) On ergodic impulsive control problems. Stochastics 18, 49–72.Google Scholar
[25] Vermes, D. (1985) Optimal control of piecewise-deterministic Markov processes. Stochastics 14, 165–208.Google Scholar
[26] Yushkevich, A. A. (1983) Continuous-time Markov decision processes with intervention. Stochastics 9, 235–274.Google Scholar
[27] Yushkevich, A. A. (1987) Bellman inequalities in Markov decision deterministic drift processes. Stochastics. CrossRefGoogle Scholar
[28] Zabczyk, J. (1984) Lectures in stochastic control. Control Theory Centre Report No. 125, University of Warwick.Google Scholar