This lecture segues into the third part of the course, where we ask: Do we expect strategic agents to reach an equilibrium? If so, which learning algorithms quickly converge to an equilibrium? Reasoning about these questions requires specifying dynamics, which describe how agents act when not at equilibrium. We consider dynamics where each agent's behavior is governed by an algorithm that attempts to, in some sense, learn the best response to how the other agents are acting. Ideally, we seek results that hold for multiple simple and natural learning algorithms. Then, even though agents may not literally follow such an algorithm, we can still have some confidence that our conclusions are robust and not an artifact of the particular choice of dynamics. This lecture focuses on variations of “best-response dynamics,” while the next two lectures study dynamics based on regret-minimization.

Section 16.1 defines best-response dynamics and proves convergence in potential games. Sections 16.2 and 16.3 introduce e-best-response dynamics and prove that several variants of it converge quickly in atomic selfish routing games where all agents have a common origin and destination. Section 16.4 proves that, in the (λ, μ)-smooth games defined in Lecture 14, several variants of best-response dynamics quickly reach outcomes with objective function value almost as good as at an equilibrium.

Best-Response Dynamics in Potential Games

Best-response dynamics is a straightforward procedure by which agents search for a pure Nash equilibrium (PNE) of a game (Definition 13.2), using successive unilateral deviations.