We can now begin to deal with the main topic of this book: the analysis of differential games. In this chapter we shall see how the concept of Nash equilibrium introduced in chapter 2 can be applied in a dynamic setting. Each of the N players seeks to maximize his objective functional – the present value of utility derived over a finite or infinite time horizon – by designing a strategy for those variables which are under his control. His choice influences the evolution of the state of the game via a differential equation (the system dynamics) as well as the objective functionals of his opponents. Under the assumptions of the present chapter, we shall see that each player faces an optimal control problem of the form discussed in chapter 3. An important feature of each of these player-specific control problems is that the actions of the opponents become part of the definition of the problem. The most important assumptions of the present chapter are (i) that players make their choices simultaneously and (ii) that they represent the solutions to their control problems by Markovian strategies. We state conditions which can be used to verify that a given N-tuple of Markovian strategies constitutes a Nash equilibrium. We also discuss the important concepts of time consistency and subgame perfectness.

The Nash equilibrium

Consider a differential game which extends over the bounded time interval [0, T] or the unbounded time interval [0, ∞).