Hostname: page-component-76fb5796d-vfjqv Total loading time: 0 Render date: 2024-04-25T21:22:16.325Z Has data issue: false hasContentIssue false
  • English
  • Français

Optimal stopping for measure-valued piecewise deterministic Markov processes

Part of: Stochastic processes Markov processes Stochastic systems and control

Published online by Cambridge University Press:  16 July 2020

Bertrand Cloez ,
Benoîte de Saporta  and
Maud Joubaud
Bertrand Cloez*
Affiliation:
INRA
Benoîte de Saporta*
Affiliation:
University of Montpellier
Maud Joubaud*
Affiliation:
University of Montpellier
*
*Postal address: MISTEA, INRA, Montpellier SupAgro, University of Montpellier, Montpellier, France.
**Postal address: IMAG, Univ. Montpellier, CNRS, Montpellier, France. Email: benoite.de-saporta@umontpellier.fr
**Postal address: IMAG, Univ. Montpellier, CNRS, Montpellier, France. Email: benoite.de-saporta@umontpellier.fr
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper investigates the random horizon optimal stopping problem for measure-valued piecewise deterministic Markov processes (PDMPs). This is motivated by population dynamics applications, when one wants to monitor some characteristics of the individuals in a small population. The population and its individual characteristics can be represented by a point measure. We first define a PDMP on a space of locally finite measures. Then we define a sequence of random horizon optimal stopping problems for such processes. We prove that the value function of the problems can be obtained by iterating some dynamic programming operator. Finally we prove via a simple counter-example that controlling the whole population is not equivalent to controlling a random lineage.

Keywords

Markov processesmeasure spaceoptimal stoppingdynamic programmingpopulation dynamics

MSC classification

Primary: 60J25: Continuous-time Markov processes on general state spaces
Secondary: 60G40: Stopping times; optimal stopping problems; gambling theory 92D25: Population dynamics (general) 93E20: Optimal stochastic control
Type
Research Papers
Information
Journal of Applied Probability , Volume 57 , Issue 2 , June 2020 , pp. 497 - 512
Copyright
© Applied Probability Trust 2020

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bäuerle, N. and Rieder, U. (2011). Markov Decision Processes with Applications to Finance (Universitext). Springer, Heidelberg.10.1007/978-3-642-18324-9CrossRefGoogle Scholar
Benaïm, M. and Lobry, C. (2016). Lotka–Volterra with randomly fluctuating environments or ‘how switching between beneficial environments can make survival harder’. Ann. Appl. Prob. 26, 3754–3785.10.1214/16-AAP1192CrossRefGoogle Scholar
Bertoin, J. (2006). Random Fragmentation and Coagulation Processes (Cambridge Studies in Advanced Mathematics 102). Cambridge University Press, Cambridge.Google Scholar
Buckwar, E. and Riedler, M. G. (2011). An exact stochastic hybrid model of excitable membranes including spatio-temporal evolution. J. Math. Biol. 63, 1051–1093.10.1007/s00285-010-0395-zCrossRefGoogle ScholarPubMed
Campillo, F., Champagnat, N. and Fritsch, C. (2016). Links between deterministic and stochastic approaches for invasion in growth–fragmentation–death models. J. Math. Biol. 73, 1781–1821.10.1007/s00285-016-1012-6CrossRefGoogle ScholarPubMed
Cloez, B. (2017). Limit theorems for some branching measure-valued processes. Adv. Appl. Prob. 49, 549–580.10.1017/apr.2017.12CrossRefGoogle Scholar
Cloez, B., Dessalles, R., Genadot, A., Malrieu, F., Marguet, A. and Yvinec, R. (2017). Probabilistic and piecewise deterministic models in biology. ESAIM: Procs 60, 225–245.10.1051/proc/201760225CrossRefGoogle Scholar
Costa, M. (2016). A piecewise deterministic model for a prey–predator community. Ann. Appl. Prob. 26, 3491–3530.10.1214/16-AAP1182CrossRefGoogle Scholar
Davis, M. H. A. (1984). Piecewise-deterministic Markov processes: a general class of nondiffusion stochastic models. J. R. Statist. Soc. Ser. B 46, 353–388.Google Scholar
Davis, M. H. A. (1993). Markov Models and Optimization (Monographs on Statistics and Applied Probability 49). Chapman & Hall, London.Google Scholar
de Saporta, B., Dufour, F. and Zhang, H. (2015). Numerical Methods for Simulation and Optimization of Piecewise Deterministic Markov Processes: Application to Reliability (Mathematics and StatisticsSeries). Wiley-ISTE.Google Scholar
Doumic, M., Hoffmann, M., Krell, N. and Robert, L. (2015). Statistical estimation of a growth-fragmentation model observed on a genealogical tree. Bernoulli 21, 1760–1799.10.3150/14-BEJ623CrossRefGoogle Scholar
Fournier, N. and Méléard, S. (2004). A microscopic probabilistic description of a locally regulated population and macroscopic approximations. Ann. Appl. Prob. 14, 1880–1919.10.1214/105051604000000882CrossRefGoogle Scholar
Genadot, A. and Thieullen, M. (2014). Multiscale piecewise deterministic Markov process in infinite dimension: central limit theorem and Langevin approximation. ESAIM Probab. Statist. 18, 541–569.10.1051/ps/2013051CrossRefGoogle Scholar
Gugerli, U. S. (1986). Optimal stopping of a piecewise-deterministic Markov process. Stochastics 19, 221–236.10.1080/17442508608833426CrossRefGoogle Scholar
Guyon, J. (2007). Limit theorems for bifurcating Markov chains: application to the detection of cellular aging. Ann. Appl. Prob. 17, 1538–1569.10.1214/105051607000000195CrossRefGoogle Scholar
Kallenberg, O. (2017). Random Measures, Theory and Applications (Probability Theory and Stochastic Modelling 77). Springer, Cham.Google Scholar
Kolokoltsov, V. N. (2011). Markov Processes, Semigroups and Generators (Gruyter Studies in Mathematics 38). Walter de Gruyter, Berlin.Google Scholar
Riedler, M. G., Thieullen, M. and Wainrib, G. (2012). Limit theorems for infinite-dimensional piecewise deterministic Markov processes: applications to stochastic excitable membrane models. Electron. J. Prob. 17 (55), 48.10.1214/EJP.v17-1946CrossRefGoogle Scholar
Robert, L., Hoffmann, M., Krell, N., Aymerich, S., Robert, J. and Doumic, M. (2014). Division in Escherichia coli is triggered by a size-sensing rather than a timing mechanism. BMC Biology 12, 17.10.1186/1741-7007-12-17CrossRefGoogle ScholarPubMed