The previous two chapters of the book have focused on the basic issue: perception of the environment from two different viewpoints. Specifically, Chapter 3 focused on perception viewed as power-spectrum estimation, which is applicable to applications such as cognitive radio, where spectrum sensing of the radio environment is of paramount importance. In contrast, Chapter 4 focused on Bayesian filtering for state estimation, which is applicable to another class of applications exemplified by cognitive radar, where estimating the parameters of a target in a radar environment is the issue of interest. Although those two chapters address entirely different topics, they also have a common perspective: they both pertain to the perceptor of a cognitive dynamic system, the basic function of which is to account for perception in the perception–action cycle. In this chapter, we move on to the action part of this cycle, which is naturally the responsibility of the actuator of the cognitive dynamic system.

Just as the Bayesian framework provides the general formalism for the perceptor to perceive the environment by estimating the hidden state of the environment, Bellman's dynamic programming provides the general formalism for the actuator to act in the environment through control or decision-making.

Dynamic programming is a technique that deals with situations where decisions are made in stages (i.e. different time steps), with the outcome of each decision being predictable to some extent before the next decision is made.