In this chapter, we turn to the analysis of non-linear difference equations. Since most applications in economics assume a stationary environment, that is, an environment that does not change over time, the focus of our study will be on autonomous equations. Such equations typically admit constant solutions – so-called fixed points – or periodic solutions, and the first step of the analysis of an autonomous difference equation is often the identification of these simple types of solutions. Therefore, we collect in section 3.1 a number of results about fixed points and periodic points.

As a next step, we turn to the investigation of the dynamics locally around the fixed points and the periodic points, respectively. This is greatly facilitated by local linearization techniques and the Hartman–Grobman theorem, which we present in section 3.2. In section 3.3, we introduce the important notion of the stability of a fixed point or a periodic point and we derive stability criteria. Some of these criteria are based on local linearization techniques, whereas others involve Lyapunov functions.

As we shall see in section 3.4, the appropriate definition of stability for an economic problem depends on how many of the system variables are pre-determined and how many are jump variables. This will lead us to the concept of saddle point stability. Finally, in section 3.5 we demonstrate that in the case of ‘too much stability’, a notion that we will formally define, purely deterministic economic models can admit stochastic solutions if we properly take into account the influence that expectations have on the behaviour of economic agents.

Invariant sets, fixed points, and periodic points

In this section, we consider autonomous difference equations of the form

xt+1= f(xt), (3.1)

where the law of motion f : X → X is a given function and the system domain X ⊆ ℝn is a non-empty set.

The present textbook grew out of lecture notes for courses on the methods and applications of dynamic economic analysis that I have been teaching to graduate students over the years. The book is not meant to cover the whole state of the art in this area but to provide a compact presentation of the most essential concepts and results and to illustrate them by selected applications from various fields of economic research. The target readership consists of students and researchers who have little or no experience with solution techniques for dynamic economic models but who have a decent background knowledge in economics, calculus, and linear algebra. I hope that the book helps its readers to get acquainted with the basic issues and the most popular modelling frameworks of dynamic economic analysis and that it raises the appetite of its audience for a more complete and detailed study of this area.

Dynamic economic analysis is a vast area and the relevant literature is extensive. In order to achieve my goal of a compact presentation, I deliberately make two important restrictions that are also reflected in the title of the book. First, I only consider models, that are formulated in discrete time and, second, I do not deal with stochastic dynamics. The main justification of the first restriction is that I want to provide an overview of the most important concepts and methods of dynamic economic analysis without getting lost in technicalities. For some dynamic economic models, the choice between a discrete-time formulation and a continuous-time formulation is simply a matter of taste, whereas for others this choice is driven by the quest for analytical tractability. Moreover, there exist dynamic economic models that generate quite different predictions depending on which of the two formulations of time is applied. In my opinion, however, the basic issues arising in dynamic economic analysis can be illustrated with less technical effort in a discrete-time setting than in a continuous-time setting, which is why I restrict the presentation to the former case. By no means do I consider the discrete-time formulation as more relevant or more realistic than its continuous-time counterpart.

Non-cooperative games (or strategic games) are mainly studied through two models: normal form or extensive form. The latter will be presented in Chapter II. The former describes the choice spaces of each player and the result of their common choices. This is evaluated in terms of the players' von Neumann–Morgenstern utilities (i.e., the utility of a random variable is its expected utility (von Neumann and Morgenstern, 1944, Chapter I, 3.5)), and hence the following definition:

A normal form game is defined by a set of I, strategy spaces Si, i ∈ I, and real payoff functionsFi, i ∈ I, on S = Πi Si.

It is finite (or a bi-matrix game if #I = 2) if I and all Si are finite.

Under suitable measurability conditions one defines the mixed extension of a game G = (I, (Si, Fi)i∈I) as the game Γ = (I, (Σi, φi)i∈I), where Σi is the set of probabilities on Si and φi (σ) = ∫SFi(S) Πi∈I σi(dsi). An element of Si will be called a pure strategy, while an element of Σi will be called a mixed strategy (of player i). Unless explicitly specified (or self-evident), the following definitions are always used on the mixed extension of the game.

An (ε-)equilibrium is an I-tuple s such that for every i, si is an (ε-)best reply to s−i.

THE MODEL

In this chapter we introduce formally the general model of repeated games.

We start with a non-cooperative game G and define a new game Г∞, a play of which is an infinite sequence of plays of G.

In fact, it appears in many applications that current moves not only influence the current payoff but also the future play, hence some state variable of the model. This is the reason why stochastic games appear in a natural way.

Moreover we have to describe the information available to the players. There may be some differences between their initial knowledge of the characteristics: initial state, preferences, even transition law. This is taken into account in the framework of games with incomplete information.

Finally it is necessary for a full description of the game to specify what additional information is transmitted to the players after each stage of the play. It is easy to see that assuming the knowledge of the other players' strategies is unrealistic. A more plausible weaker assumption may be that only the actual moves are observed. An even weaker assumption is that only the individual player's payoff is known to him. Even more generally, we may consider a model without full monitoring of previous moves, or of the outcomes, or even of the players' own payoffs. This leads to the notion of signals that may depend in a random way on the actual moves and state.

To integrate all such effects it is sufficient to define a state- and move-dependent lottery that selects at every stage the signals for the players, their payoffs, and the next state of nature. If one wants in addition to incorporate the effect of information lags, this transition may also depend on the past events. In fact, we will see that this quite huge construction can be reduced to a simple and convenient form (cf. Proposition IV.2.3, p. 183).

It follows from the above presentation that this model is an adequate description of a stationary multistage game in the sense that its formulation is time-shift invariant (which may require adding new states and payoffs, if necessary) and needs only some counter to let the stage of the game be known to the players.

In this chapter we give a formal treatment to the basic concepts of models with incomplete information, namely, beliefs, types, consistency of beliefs, common knowledge, etc. Given a parameter space K, we first construct the Universal Belief Space Ω. A point ω in Ω, which we call a state of the world, consists of (in addition to the value of the parameters) the specification of the “state of mind” of each of the players (i.e., the probabilities generated by beliefs on K, beliefs on these beliefs, and so on). We study the mathematical structure and properties of Ω, introducing the notion of belief subspace, which is closely related to the concept of common knowledge. We next define the concept of consistent beliefs as beliefs that can be derived as conditional probabilities from some common prior distribution, given each player's private information. We characterize the consistent states of the world (i.e., states of the world in which the players' beliefs are consistent). An approximation theorem is then presented stating roughly that any incomplete information situation can be approximated by one in which there are finitely many potential types of each player. In the last section we discuss models and strategic equilibria of games with incomplete information, based on the structure of the beliefs space developed in this chapter.

THE UNIVERSAL BELIEF SPACE

When considering a situation involving a finite player set I facing some uncertainty about a space K (which we refer to as the space of states of nature), one is naturally led to handle infinite hierarchies of beliefs: adopting a Bayesian approach, each player will base his decision on some subjective beliefs (i.e., probability measure) on K. Since the outcome is determined not only by the player's own actions but the other players' actions as well, and those are influenced by their beliefs on K, each player must also have beliefs on other players' beliefs on K. By the same argument he must have beliefs on other players' beliefs on his own beliefs on K, beliefs on other players' beliefs on his beliefs on their beliefs on K, etc.

CHAPTER I

Section I.1 The original proof of Sion's theorem (1958) (Theorem I.1.1 in this volume) uses the KKM lemma (I.4, Ex. 18) in ℝn (which is equivalent to the fixed point theorem). However, Sion wrote “the difficulty lies in the fact that we cannot use a fixed point theorem (due to lack of continuity) nor the separation of disjoint convex sets by a hyperplane (due to lack of convexity).”

Also in Sion's paper is the proof that his theorem implies Fan's theorem (1953) (cf. the remark after Proposition I.2.7).

The proof of Sion's theorem using Lemma I.1.2, as well as the lemma itself and its proof, appears in Berge (1966, p. 220, resp. p. 172). In fact he wrote: “We note that Sion's proof led us to the statement of the intersection theorem.” On the other hand, the original proof of the lemma (Berge, 1959) also uses KKM.

A direct proof of Proposition I.1.9 was obtained by Kneser (1952) (all these results are in the Hausdorff case).

A survey and specific results for games on the square can be found in Yanovskaya (1974).

Section I.2 The proof of I.2, Ex. 17 is due to Karamata (cf., e.g., Titchmarsh, 1939, p. 227).

Section I.3 The original finite minmax theorem (I.3, Ex. 1) can be proved by “elementary tools,” namely, the theorem of the alternative (von Neumann and Morgenstern, 1944, pp. 138, 154–155) (also Ville, 1938): the iterated elimination of variables implies the existence of optimal strategies (and a fortiori the value) in the ordered field of coefficients. The first analysis in this framework and Theorem I.3.6 are due to Weyl (1950); cf. I.3, Ex. 13. This elementary aspect was used later by Bewley and Kohlberg (1976a) in analyzing stochastic games.

Another elementary proof by induction on the size of the matrix is due to Loomis (1946).

We now start to study repeated games with incomplete information. In the present chapter we consider the simplest class of those games, namely, two-person zero-sum games in which one player, say, player I, is fully informed about the state of nature, while the other player, player II, knows only the prior distribution according to which the state is chosen.

GENERAL PROPERTIES

In this section we prove some general properties of a one-shot game with incomplete information, which later will be applied to various versions of the game: finitely or infinitely repeated games or discounted games. The game considered here is a two-person zero-sum game of the following form: chance chooses a state k from a finite set K of states (games) according to some probability p ∈ Π = Δ(K). Player I (the maximizer) is informed which k was chosen but player II is not. Players I and II then choose simultaneously σk ∈ Σ and τ ∈ J, respectively, and finally Gk(σk, τ) is paid to player I by player II. The sets Σ and J are some convex sets of strategies, and the payoff functions Gk(σk, τ) are bi-linear and uniformly bounded on Σ × J.

In normal form this is a game in which the strategies are σ ∈ ΣK and τ ∈ J, respectively, and the payoff function is Gp(σ, τ) = ∑kpkGk (σk, τ). Denote this game by Г(p).

Theorem V.1.1.w(p) = infτsupσGp(σ, τ) and w(p) = supσinfτGp(σ, τ) are concave.

Proof. The proof is the same for both functions. We write it for w(p). Let (pe)e∈E be finitely many points in Δ(K), and let α = (αe)e∈E be a point in Δ(E) such that ∑e∈E αepe = p; we claim that w(p) ≥ ∑e∈E αew(pe).