Book contents
- Frontmatter
- Contents
- Preface
- 1 Introduction
- Part I Stochastic Models and Bayesian Filtering
- Part II Partially Observed Markov Decision Processes: Models and Applications
- Part III Partially Observed Markov Decision Processes: Structural Results
- Part IV Stochastic Approximation and Reinforcement Learning
- Appendix A Short primer on stochastic simulation
- Appendix B Continuous-time HMM filters
- Appendix C Markov processes
- Appendix D Some limit theorems
- References
- Index
Part II - Partially Observed Markov Decision Processes: Models and Applications
Published online by Cambridge University Press: 05 April 2016
- Frontmatter
- Contents
- Preface
- 1 Introduction
- Part I Stochastic Models and Bayesian Filtering
- Part II Partially Observed Markov Decision Processes: Models and Applications
- Part III Partially Observed Markov Decision Processes: Structural Results
- Part IV Stochastic Approximation and Reinforcement Learning
- Appendix A Short primer on stochastic simulation
- Appendix B Continuous-time HMM filters
- Appendix C Markov processes
- Appendix D Some limit theorems
- References
- Index
Summary
Part II of the book describes the Markov decision process (MDP) problem and the partially observed Markov decision process (POMDP) problem.
Chapter 6 gives a concise description of finite state MDPs, Bellman's stochastic dynamic programming equation for solving MDPs, and solution methods for infinite horizon Markov decision processes. Also constrained Markov decision processes are discussed briefly.
Chapter 7 describes the POMDP problem, Bellman's dynamic programming equation and associated algorithms, and discusses optimal search theory (which is a useful POMDP example).
Chapter 8 discusses several examples of POMDPs in controlled sensing including radar scheduling, social learning and risk-averse formulations.
To give some perspective, from a control theory point of view, one can consider three frameworks:
(i) Centralized control comprised of a single controller;
(ii) Team theory where there are several controllers but a single objective;
(iii) Game theory where multiple selfish decision-makers have their own individual utilities.
In this book we focus only on centralized control, apart from social learning examples where the interaction of multiple agents is discussed.
- Type
- Chapter
- Information
- Partially Observed Markov Decision ProcessesFrom Filtering to Controlled Sensing, pp. 119 - 120Publisher: Cambridge University PressPrint publication year: 2016