Introduction

In this chapter we briefly review some results on discounted dynamic programming under uncertainty, and indicate how Markov processes and random dynamical systems are generated by optimal policies. In Section 6.2, following a precise description of the dynamic programming framework, we turn to the special case where the set S of states is countable, and the set A of actions is finite. Here the link between optimal policies and the celebrated functional equation can be established with no measure theoretic complications. In Section 6.3 we study the maximum theorem, which is of independent interest in optimization theory and is a key to the understanding of the basic result in the next section. In Section 6.4 we explore the more general model where S is a Borel subset of a Polish space, and A is a compact action space, and spell out the conditions under which there is a stationary optimal policy.

The dynamic programming technique reviewed here has been particularly powerful in attacking a variety of problems in intertemporal economics. In Section 6.5 we discuss in detail the aggregative model of optimal economic growth under uncertainty.