Hostname: page-component-8448b6f56d-wq2xx Total loading time: 0 Render date: 2024-04-18T20:20:29.968Z Has data issue: false hasContentIssue false
  • English
  • Français

A class of singular stochastic control problems

Published online by Cambridge University Press:  01 July 2016

Ioannis Karatzas
Ioannis Karatzas*
Affiliation:
Columbia University
*
∗Postal address: Department of Mathematical Statistics, Columbia University, New York, NY 10027, U.S.A.
Get access Rights & Permissions [Opens in a new window]

Abstract

We consider the problem of tracking a Brownian motion by a process of bounded variation, in such a way as to minimize total expected cost of both ‘action' and ‘deviation from a target state 0'. The former is proportional to the amount of control exerted to date, while the latter is being measured by a function which can be viewed, for simplicity, as quadratic. We discuss the discounted, stationary and finite-horizon variants of the problem. The answer to all three questions takes the form of exerting control in a singular manner, in order not to exit from a certain region. Explicit solutions are found for the first and second questions, while the third is reduced to an appropriate optimal stopping problem. This reduction yields properties, as well as global upper and lower bounds, for the associated moving boundary. The pertinent Abelian and ergodic relationships for the corresponding value functions are also derived.

Keywords

STOCHASTIC CONTROLOPTIMAL STOPPINGREFLECTED BROWNIAN MOTIONFREE-BOUNDARY PROBLEMS
Type
Research Article
Information
Advances in Applied Probability , Volume 15 , Issue 2 , June 1983 , pp. 225 - 254
Copyright
Copyright © Applied Probability Trust 1983 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This paper was presented at the second Bad Honnef Workshop on Stochastic Differential Systems, University of Bonn, West Germany, 28 June–2 July 1982.

Research partly supported by NSF Grant MCS 81–03435.

References

[1] Bather, J. A. (1962) Bayes procedures for deciding the sign of a normal mean. Proc. Camb. Phil. Soc. 58, 599–620.Google Scholar
[2] Bather, J. A. (1970) Optimal stopping problems for Brownian motion. Adv. Appl. Prob. 2, 259–286.Google Scholar
[3] Bather, J. A. and Chernoff, H. (1966) Sequential decisions in the control of a spaceship. Proc. 5th Berkeley Symp. Math. Statist. Prob. 3, 181–207.Google Scholar
[4] Bather, J. A. and Chernoff, H. (1967) Sequential decisions in the control of a spaceship (finite fuel). J. Appl. Prob. 4, 584–604.Google Scholar
[5] Beneš, V. E., Shepp, L. A. and Witsenhausen, H. S. (1980) Some solvable stochastic control problems. Stochastics 4, 39–83.Google Scholar
[6] Bensoussan, A. and Lions, J. L. (1978) Applications des inéquations variationelles en contrôle stochastique. Dunod, Paris.Google Scholar
[7] Birkhoff, G. (1963) Lattice Theory. American Mathematical Society, New York.Google Scholar
[8] Chernoff, H. (1968) Optimal stochastic control. Sankhya A 30, 221–252.Google Scholar
[9] Chernoff, H. and Petkau, A. J. (1979) A satellite control problem. In Optimizing Methods in Statistics, Academic Press, New York, 89–124.Google Scholar
[10] Doleans-Dade, C. and Meyer, P. A. (1970) Intégrales stochastiques par rapport aux martingales locales. Séminaire de Probabilités TV, Lecture Notes in Mathematics 124, Springer-Verlag, Berlin, 77–107.Google Scholar
[11] Ferebee, B. (1981) Testing a new drug against a standard–the diffusion approximation. Preprint No. 111, Sonderforschungsbereich 123, Universität Heidelberg.Google Scholar
[12] Fleming, W. H. and Rishel, R. W. (1975) Deterministic and Stochastic Optimal Control. Springer-Verlag, Berlin.Google Scholar
[13] Friedman, A. (1959) Free boundary problems for parabolic equations I: Melting of solids. J. Math. Mech. 8, 499–518.Google Scholar
[14] Friedman, A. (1973) Stochastic games and variational inequalities. Arch. Rat. Mech. Anal. 51, 321–346.Google Scholar
[15] Friedman, A. (1973) Regularity theorems for variational inequalities in unbounded domains and applications to stopping time problems. Arch. Rat. Mech. Anal. 52, 134–160.Google Scholar
[16] Friedman, A. (1976). Stochastic Differential Equations and Applications, Vol. 2. Academic Press, New York.Google Scholar
[17] Gihman, I. I. and Skorohod, A. V. (1972) Stochastic Differential Equations (English translation). Springer-Verlag, Berlin.Google Scholar
[18] Ikeda, N. and Watanabe, S. (1981) Stochastic Differential Equations and Diffusion Processes. North-Holland, Amsterdam.Google Scholar
[19] Karatzas, I. (1981) The monotone follower problem in stochastic decision theory. Appl. Math. Optim. 7, 175–189.Google Scholar
[20] Karlin, S. and Taylor, H. M. (1975) A First Course in Stochastic Processes, 2nd edn. Academic Press, New York.Google Scholar
[21] Krylov, N. V. (1980) Controlled Diffusion Processes (English translation). Springer-Verlag, Berlin.Google Scholar
[22] Lehoczky, J. P. (1977) Formulas for stopped diffusions with stopping times based on the maximum. Ann. Prob. 5, 601–607.Google Scholar
[23] McKean, H. P. Jr. (1969) Stochastic Integrals. Academic Press, New York.Google Scholar
[24] Ross, S. M. (1970) Applied Probability Models with Optimization Applications. Holden-Day, San Francisco.Google Scholar
[25] Shreve, S. E., Lehoczky, J. P. and Gaver, D. P. (1982) Optimal consumption for general diffusions with absorbing and reflecting barriers. SIAM J. Control and Optimization. To appear.Google Scholar
[26] Taylor, H. M. (1975) A stopped Brownian motion formula. Ann. Prob. 3, 234–246.Google Scholar
[27] Van Moerbeke, P. (1976) Optimal stopping and free boundary problems. Arch. Rat. Mech. Anal. 60, 101–148.CrossRefGoogle Scholar
[28] Widder, D. V. (1941) The Laplace Transform. Princeton University Press, Princeton.Google Scholar
[29] Williams, D. (1976) On a stopped Brownian motion formula of H. M. Taylor. Séminaire de Probabilités X, Lecture Notes in Mathematics 511, Springer-Verlag, Berlin, 235–239.Google Scholar