Hostname: page-component-76fb5796d-skm99 Total loading time: 0 Render date: 2024-04-26T22:07:25.082Z Has data issue: false hasContentIssue false
  • English
  • Français

Repeated games for multiagent systems: a survey

Published online by Cambridge University Press:  18 March 2013

Andriy Burkov  and
Brahim Chaib-Draa
Andriy Burkov
Affiliation:
Department of Computer Science and Software Engineering, Université Laval, Québec, QC G1V OA 6, Canada; e-mail: burkov@damas.ift.ulaval.ca, chaib@ift.ulaval.ca
Brahim Chaib-Draa
Affiliation:
Department of Computer Science and Software Engineering, Université Laval, Québec, QC G1V OA 6, Canada; e-mail: burkov@damas.ift.ulaval.ca, chaib@ift.ulaval.ca
Get access Rights & Permissions [Opens in a new window]

Abstract

Repeated games are an important mathematical formalism to model and study long-term economic interactions between multiple self-interested parties (individuals or groups of individuals). They open attractive perspectives in modeling long-term multiagent interactions. This overview paper discusses the most important results that actually exist for repeated games. These results arise from both economics and computer science. Contrary to a number of existing surveys of repeated games, most of which originated from the economic research community, we are first to pay a special attention to a number of important distinctive features proper to artificial agents. More precisely, artificial agents, as opposed to the human agents mainly aimed by the economic research, are usually bounded whether in terms of memory or performance. Therefore, their decisions have to be based on the strategies defined using finite representations. Furthermore, these strategies have to be efficiently computed or approximated using a limited computational resource usually available to artificial agents.

Type
Articles
Information
The Knowledge Engineering Review , Volume 29 , Issue 1 , January 2014 , pp. 1 - 30
Copyright
Copyright © Cambridge University Press 2013 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abreu, D. 1986. Extremal equilibria of oligopolistic supergames. Journal of Economic Theory 39(1), 191–225.CrossRefGoogle Scholar
Abreu, D. 1988. On the theory of infinitely repeated games with discounting. Econometrica 56, 383–396.CrossRefGoogle Scholar
Abreu, D., Pearce, D., Stacchetti, E. 1990. Toward a theory of discounted repeated games with imperfect monitoring. Econometrica 58(5), 1041–1063.CrossRefGoogle Scholar
Abreu, D., Rubinstein, A. 1988. The structure of Nash equilibrium in repeated games with finite automata. Econometrica 56(6), 1259–1281.CrossRefGoogle Scholar
Aumann, R. 1981. Survey of repeated games. Essays in Game Theory and Mathematical Economics in Honor of Oskar Morgenstern, 11–42.Google Scholar
Aumann, R., Maschler, M., Stearns, R. 1995. Repeated Games With Incomplete Information. The MIT press.Google Scholar
Aumann, R., Shapley, L. 1994. Long term competition: a game theoretic analysis. Essays in Game Theory in Honor of Michael Maschler, 1–15.Google Scholar
Banerjee, B., Peng, J. 2003. Adaptive policy gradient in multiagent learning. In Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS'03). ACM Press, 686–692.Google Scholar
Banks, J., Sundaram, R. 1990. Repeated games, finite automata, and complexity. Games and Economic Behavior 2(2), 97–117.CrossRefGoogle Scholar
Ben-Porath, E. 1990. The complexity of computing a best response automaton in repeated games with mixed strategies. Games and Economic Behavior 2(1), 1–12.CrossRefGoogle Scholar
Ben-Porath, E. 1993. Repeated games with finite automata. Journal of Economic Theory 59, 17–39.CrossRefGoogle Scholar
Ben-Porath, E., Peleg, B. 1987. On the Folk Theorem and Finite Automata. Mimeo, Hebrew University of Jerusalim.Google Scholar
Ben-Sasson, E., Kalai, A. T., Kalai, E. 2007. An approach to bounded rationality. In Advances in Neural Information Processing Systems 19, Schisölkopf, B., Platt J. & Hoffman, T. (eds). MIT Press, 145–152.CrossRefGoogle Scholar
Benoit, J.-P., Krishna, V. 1985. Finitely repeated games. Econometrica 53(4), 905–922.CrossRefGoogle Scholar
Benoit, J.-P., Krishna, V. 1999. The Folk Theorems for Repeated Games: A Synthesis. Mimeo, Pennsylvania State University.Google Scholar
Bernstein, D., Givan, R., Immerman, N., Zilberstein, S. 2003. The complexity of decentralized control of Markov decision processes. Mathematics of Operations Research 27(4), 819–840.CrossRefGoogle Scholar
Berry, D., Fristedt, B. 1985. Bandit Problems. Chapman and Hall London.CrossRefGoogle Scholar
Bhaskar, V., Obara, I. 2002. Belief-based equilibria in the repeated Prisoners’ Dilemma with private monitoring. Journal of Economic Theory 102(1), 40–69.CrossRefGoogle Scholar
Borgs, C., Chayes, J., Immorlica, N., Kalai, A., Mirrokni, V., Papadimitriou, C. 2008. The myth of the folk theorem. In Proceedings of the 40th Annual ACM Symposium on Theory of Computing (STOC'08). ACM Press, 365–372.Google Scholar
Bowling, M., Veloso, M. 2002. Multiagent learning using a variable learning rate. Artificial Intelligence 136(2), 215–250.CrossRefGoogle Scholar
Burkov, A., Chaib-draa, B. 2009. Effective learning in the presence of adaptive counterparts. Journal of Algorithms 64(4), 127–138.CrossRefGoogle Scholar
Burkov, A., Chaib-draa, B. 2010. An approximate subgame-perfect equilibrium computation technique for repeated games. In Proceedings of Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI'10). AAAI Press, 729–736.Google Scholar
Chen, X., Deng, X. 2006. Settling the complexity of two-player Nash equilibrium. In Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06). IEEE Computer Society, 261–272.Google Scholar
Cheng, S., Reeves, D., Vorobeychik, Y., Wellman, M. 2004. Notes on equilibria in symmetric games. In AAMAS-04 Workshop on Game-Theoretic and Decision-Theoretic Agents.Google Scholar
Compte, O. 1998. Communication in repeated games with imperfect private monitoring. Econometrica 66(3), 597–626.CrossRefGoogle Scholar
Conitzer, V., Sandholm, T. 2007. AWESOME: a general multiagent learning algorithm that converges in self-play and learns a best response against stationary opponents. Machine Learning 67(1), 23–43.CrossRefGoogle Scholar
Cronshaw, M. 1997. Algorithms for finding repeated game equilibria. Computational Economics 10(2), 139–168.CrossRefGoogle Scholar
Cronshaw, M., Luenberger, D. 1994. Strongly symmetric subgame perfect equilibria in infinitely repeated games with perfect monitoring and discounting. Games and Economic Behavior 6(2), 220–237.CrossRefGoogle Scholar
Daskalakis, C., Goldberg, P., Papadimitriou, C. 2006. The complexity of computing a Nash equilibrium. In Proceedings of the Thirty-Eighth Annual ACM Symposium on Theory of Computing (STOC'06). ACM Press, 71–78.Google Scholar
Ely, J., Hörner, J., Olszewski, W. 2005. Belief-free equilibria in repeated games. Econometrica 73(2), 377–415.Google Scholar
Ely, J., Valimaki, J. 2002. A robust folk theorem for the prisoner's dilemma. Journal of Economic Theory 102(1), 84–105.Google Scholar
Friedman, J. 1971. A non-cooperative equilibrium for supergames. The Review of Economic Studies, 1–12.CrossRefGoogle Scholar
Fudenberg, D., Kreps, D., Maskin, E. 1990. Repeated games with long-run and short-run players. The Review of Economic Studies 57(4), 555–573.CrossRefGoogle Scholar
Fudenberg, D., Levine, D. 1991. An approximate folk theorem with imperfect private information. Journal of Economic Theory 54(1), 26–47.Google Scholar
Fudenberg, D., Levine, D., Maskin, E. 1994. The folk theorem with imperfect public information. Econometrica 62(5), 997–1039.Google Scholar
Fudenberg, D., Levine, D., Takahashi, S. 2007. Perfect public equilibrium when players are patient. Games and Economic Behavior 61(1), 27–49.Google Scholar
Fudenberg, D., Tirole, J. 1991. Game Theory. MIT Press.Google Scholar
Gilboa, I. 1988. The complexity of computing best-response automata in repeated games. Journal of economic theory 45(2), 342–352.Google Scholar
Gossner, O., Tomala, T. 2009. Repeated games. Encyclopedia of Complexity and Systems Science, forthcoming.CrossRefGoogle Scholar
Hart, S., Mansour, Y. 2007. The communication complexity of uncoupled Nash equilibrium procedures. In Proceedings of the Thirty-Ninth Annual ACM Symposium on Theory of Computing (STOC'07). ACM Press, 345–353.Google Scholar
Hörner, J., Lovo, S. 2009. Belief-free equilibria in games with incomplete information. Econometrica 77(2), 453–487.Google Scholar
Hörner, J., Olszewski, W. 2006. The folk theorem for games with private almost-perfect monitoring. Econometrica 74(6), 1499–1544.CrossRefGoogle Scholar
Hörner, J., Olszewski, W. 2007. How robust is the folk theorem with imperfect public monitoring. Northwestern University.Google Scholar
Jong, S., Tuyls, K., Verbeeck, K. 2008. Fairness in multi-agent systems. The Knowledge Engineering Review 23(2), 153–180.CrossRefGoogle Scholar
Judd, K., Yeltekin, S., Conklin, J. 2003. Computing supergame equilibria. Econometrica 71(4), 1239–1254.CrossRefGoogle Scholar
Kaelbling, L., Littman, M., Cassandra, A. 1998. Planning and acting in partially observable stochastic domains. Artificial Intelligence 101(1–2), 99–134.CrossRefGoogle Scholar
Kalai, E., Stanford, W. 1988. Finite rationality and interpersonal complexity in repeated games. Econometrica 56(2), 397–410.CrossRefGoogle Scholar
Kandori, M., Obara, I. 2006. Efficiency in repeated games revisited: the role of private strategies. Econometrica 74(2), 499–519.CrossRefGoogle Scholar
Kreps, D., Wilson, R. 1982. Sequential equilibria. Econometrica: Journal of the Econometric Society, 863–894.Google Scholar
Kushilevitz, E., Nisan, N. 1997. Communication Complexity. Cambridge University Press.Google Scholar
Laraki, R. 2002. Repeated games with lack of information on one side: the dual differential approach. Mathematics of Operations Research 27(2), 419–440.CrossRefGoogle Scholar
Lehrer, E., Pauzner, A. 1999. Repeated games with differential time preferences. Econmetrica 67(2), 393–412.CrossRefGoogle Scholar
Lehrer, E., Yariv, L. 1999. Repeated games with incomplete information on one side: the case of different discount factors. Mathematics of Operations Research 24(1), 204–218.CrossRefGoogle Scholar
Lipman, B., Wang, R. 2000. Switching costs in frequently repeated games. Journal of Economic Theory 93(2), 149–190.CrossRefGoogle Scholar
Lipman, B., Wang, R. 2009. Switching costs in infinitely repeated games. Games and Economic Behavior 66(1), 292–314.CrossRefGoogle Scholar
Littman, M., Stone, P. 2005. A polynomial-time Nash equilibrium algorithm for repeated games. Decision Support Systems 39(1), 55–66.CrossRefGoogle Scholar
Mailath, G., Matthews, S., Sekiguchi, T. 2002. Private strategies in finitely repeated games with imperfect public monitoring. Contributions to Theoretical Economics 2(1), 1046.Google Scholar
Mailath, G., Morris, S. 2002. Repeated games with almost-public monitoring. Journal of Economic Theory 102(1), 189–228.CrossRefGoogle Scholar
Mailath, G., Samuelson, L. 2006. Repeated Games and Reputations: Long-run Relationships. Oxford University Press.CrossRefGoogle Scholar
Matsushima, H. 2004. Repeated games with private monitoring: two players. Econometrica 72(3), 823–852.CrossRefGoogle Scholar
Mertens, J., Sorin, S., Zamir, S. 1994. Repeated games, Part A: background material. CORE Discussion Papers, 9420.Google Scholar
Myerson, R. 1991. Game Theory: Analysis of Conflict. Harvard University Press.Google Scholar
Nash, J. 1950. Equilibrium points in n-person games. Proceedings of the National Academy of Sciences of the United States of America 36(1), 48–49.CrossRefGoogle ScholarPubMed
Neme, A., Quintas, L. 1995. Subgame perfect equilibrium of repeated games with implementation costs. Journal of Economic Theory 66(2), 599–608.CrossRefGoogle Scholar
Neyman, A. 1985. Bounded complexity justifies cooperation in the finitely repeated Prisoner's Dilemma. Economics Letters 19(3), 227–229.CrossRefGoogle Scholar
Neyman, A. 1995. Cooperation, repetition, and automata. In Cooperation: Game Theoretic Approaches, volume 155 of NATO ASI Series F. Springer-Verlag, 233–255.Google Scholar
Neyman, A. 1998. Finitely repeated games with finite automata. Mathematics of Operations Research 23(3), 513–552.CrossRefGoogle Scholar
Obara, I. 2009. Folk theorem with communication. Journal of Economic Theory 144(1), 120–134.Google Scholar
Osborne, M., Rubinstein, A. 1994. A Course in Game Theory. MIT Press.Google Scholar
Papadimitriou, C. 1992. On players with a bounded number of states. Games and Economic Behavior 4(1), 122–131.Google Scholar
Papadimitriou, C., Yannakakis, M. 1988. Optimization, approximation, and complexity classes. In Proceedings of the Twentieth Annual ACM Symposium on Theory of Computing. ACM Press, 229–234.Google Scholar
Papadimitriou, C., Yannakakis, M. 1994. On complexity as bounded rationality (extended abstract). In Proceedings of the Twenty-Sixth Annual ACM Symposium on Theory of Computing. ACM Press, 726–733.Google Scholar
Pearce, D. 1992. Repeated games: cooperation and rationality. In Advances in Economic Theory: Sixth World Congress, vol. 1. Cambridge University Press, 132–174.Google Scholar
Piccione, M. 2002. The repeated prisoner's dilemma with imperfect private monitoring. Journal of Economic Theory 102(1), 70–83.CrossRefGoogle Scholar
Radner, R. 1986. Repeated partnership games with imperfect monitoring and no discounting. The Review of Economic Studies 53(1), 43–57.CrossRefGoogle Scholar
Ramchurn, S., Huynh, D., Jennings, N. 2004. Trust in multi-agent systems. The Knowledge Engineering Review 19(1), 1–25.CrossRefGoogle Scholar
Rasmusen, E. 1994. Games and Information. Blackwell Cambridge.Google Scholar
Renault, J., Scarlatti, S., Scarsini, M. 2005. A folk theorem for minority games. Games and Economic Behavior 53(2), 208–230.CrossRefGoogle Scholar
Renault, J., Scarlatti, S., Scarsini, M. 2008. Discounted and finitely repeated minority games with public signals. Mathematical Social Sciences 56(1), 44–74.CrossRefGoogle Scholar
Rubinstein, A. 1979. Equilibrium in supergames with the overtaking criterion. Journal of Economic Theory 21(1), 1–9.CrossRefGoogle Scholar
Russell, S., Norvig, P. 2009. Artificial Intelligence: A Modern Approach, 3rd edn.Prentice Hall.Google Scholar
Sekiguchi, T. 1997. Efficiency in repeated Prisoner's Dilemma with private monitoring. Journal of Economic Theory 76(2), 345–361.CrossRefGoogle Scholar
Sorin, S. 1986. On repeated games with complete information. Mathematics of Operations Research 11(1), 147–160.CrossRefGoogle Scholar
Sutton, R. S., Barto, A. G. 1998. Reinforcement Learning: An Introduction. The MIT Press.Google Scholar
Zemel, E. 1989. Small talk and cooperation: a note on bounded rationality. Journal of Economic Theory 49(1), 1–9.CrossRefGoogle Scholar