# Polynomial-Time Computation of Optimal Correlated Equilibria in Two-Player Extensive-Form Games with Public Chance Moves and Beyond

NIPS 2020, 2020.

EI

微博一下：

摘要：

Unlike normal-form games, where correlated equilibria have been studied for more than 45 years, extensive-form correlation is still generally not well understood. Part of the reason for this gap is that the sequential nature of extensive-form games allows for a richness of behaviors and incentives that are not possible in normal-form se...更多

代码：

数据：

简介

- A vast body of literature in computational game theory has focused on computing Nash equilibria (NEs) in two-player zero-sum imperfect-information extensive-form games.
- The authors show that the set of correlation plans Ξ of a triangle-free game coincides with the von Stengel-Forges polytope V of the game—a polytope that only requires a polynomial number of linear “probability-mass-conserving” constraints.
- They show that in two-player perfect-recall games without chance moves, Ξ coincides with a particular polytope V—which the authors call the von Stengel-Forges polytope—whose description only uses a polynomial number of linear constraints, which are “probability-mass-conserving” constraints:

重点内容

- A vast body of literature in computational game theory has focused on computing Nash equilibria (NEs) in two-player zero-sum imperfect-information extensive-form games
- In this paper we significantly refine this complexity threshold by showing that, in two-player games, an optimal correlated equilibrium can be computed in polynomial time, provided that a certain triangle-freeness condition—which can be checked in polynomial time—is satisfied
- In the third example, where our decomposition fails, all information sets have ∅-rank 2. We prove that such situations cannot occur, provided the game satisfies the following condition, which can be verified in polynomial time in the size of the Extensive-form games (EFGs)
- We showed that an optimal extensive-form correlated equilibrium, extensive-form coarse correlated equilibrium, and normal-form coarse correlated equilibrium can be computed in polynomial time in two-player perfect-recall games that satisfy a certain triangle-freeness condition that we introduced and that can be checked in polynomial time
- To show that such equilibria can be found in polynomial time, we gave and combined several results that may be of independent interest: (1) the existence of a scaled-extension-based structural decomposition for the von Stengel-Forges polytope of the game, (2) a characterization of when the von Stengel-Forges polytope coincides with the polytope of correlation plans, and (3) a result about the integrality of the vertices of the von Stengel-Forges polytope in triangle-free games
- In this paper we give a positive complexity result, showing that optimal equilibrium according to three important extensive-form imperfect-information game correlated solution concepts can be computed efficiently in settings—two-player games with public chance moves—where it was generally believed to be impossible

结果

- Farina et al [12] recently showed that in two-player games without chance moves, a particular structural decomposition theorem holds for the von Stengel-Forges polytope V.
- A two-player extensive-form game with public chance moves is triangle-free.
- The von Stengel-Forges polytope V of a two-player perfect-recall triangle-free EFG can be expressed via a sequence of scaled extensions with simplexes and singleton sets: V = {1} h1 X1 h2 X2 h3 · · · hn Xn, where, for i = 1, .
- Let Γ be a two-player perfect-recall extensive-form game, let V be its von Stengel-Forges polytope, and let Ξ be its polytope of correlation plans.
- Let V be the von Stengel-Forges polytope of a two-player triangle-free game (Definition 3).
- In a two-player perfect-recall extensive-form game that satisfies the triangle-freeness condition (Definition 3), the polytope of correlation plans coincides with the von Stengel-Forges polytope.
- An optimal EFCE, EFCCE, or NFCCE can be computed in polynomial time in two-player triangle-free games.
- The authors showed that an optimal extensive-form correlated equilibrium, extensive-form coarse correlated equilibrium, and normal-form coarse correlated equilibrium can be computed in polynomial time in two-player perfect-recall games that satisfy a certain triangle-freeness condition that the authors introduced and that can be checked in polynomial time.

结论

- To show that such equilibria can be found in polynomial time, the authors gave and combined several results that may be of independent interest: (1) the existence of a scaled-extension-based structural decomposition for the von Stengel-Forges polytope of the game, (2) a characterization of when the von Stengel-Forges polytope coincides with the polytope of correlation plans, and (3) a result about the integrality of the vertices of the von Stengel-Forges polytope in triangle-free games.
- In this paper the authors give a positive complexity result, showing that optimal equilibrium according to three important extensive-form imperfect-information game correlated solution concepts can be computed efficiently in settings—two-player games with public chance moves—where it was generally believed to be impossible.
- The ability to select particular correlated equilibria could be used to minimize social welfare, maximize only one of the agent’s utility, or minimize all others’ utilities—thereby furthering existing inequality or creating new inequality

总结

- A vast body of literature in computational game theory has focused on computing Nash equilibria (NEs) in two-player zero-sum imperfect-information extensive-form games.
- The authors show that the set of correlation plans Ξ of a triangle-free game coincides with the von Stengel-Forges polytope V of the game—a polytope that only requires a polynomial number of linear “probability-mass-conserving” constraints.
- They show that in two-player perfect-recall games without chance moves, Ξ coincides with a particular polytope V—which the authors call the von Stengel-Forges polytope—whose description only uses a polynomial number of linear constraints, which are “probability-mass-conserving” constraints:
- Farina et al [12] recently showed that in two-player games without chance moves, a particular structural decomposition theorem holds for the von Stengel-Forges polytope V.
- A two-player extensive-form game with public chance moves is triangle-free.
- The von Stengel-Forges polytope V of a two-player perfect-recall triangle-free EFG can be expressed via a sequence of scaled extensions with simplexes and singleton sets: V = {1} h1 X1 h2 X2 h3 · · · hn Xn, where, for i = 1, .
- Let Γ be a two-player perfect-recall extensive-form game, let V be its von Stengel-Forges polytope, and let Ξ be its polytope of correlation plans.
- Let V be the von Stengel-Forges polytope of a two-player triangle-free game (Definition 3).
- In a two-player perfect-recall extensive-form game that satisfies the triangle-freeness condition (Definition 3), the polytope of correlation plans coincides with the von Stengel-Forges polytope.
- An optimal EFCE, EFCCE, or NFCCE can be computed in polynomial time in two-player triangle-free games.
- The authors showed that an optimal extensive-form correlated equilibrium, extensive-form coarse correlated equilibrium, and normal-form coarse correlated equilibrium can be computed in polynomial time in two-player perfect-recall games that satisfy a certain triangle-freeness condition that the authors introduced and that can be checked in polynomial time.
- To show that such equilibria can be found in polynomial time, the authors gave and combined several results that may be of independent interest: (1) the existence of a scaled-extension-based structural decomposition for the von Stengel-Forges polytope of the game, (2) a characterization of when the von Stengel-Forges polytope coincides with the polytope of correlation plans, and (3) a result about the integrality of the vertices of the von Stengel-Forges polytope in triangle-free games.
- In this paper the authors give a positive complexity result, showing that optimal equilibrium according to three important extensive-form imperfect-information game correlated solution concepts can be computed efficiently in settings—two-player games with public chance moves—where it was generally believed to be impossible.
- The ability to select particular correlated equilibria could be used to minimize social welfare, maximize only one of the agent’s utility, or minimize all others’ utilities—thereby furthering existing inequality or creating new inequality

研究对象与分析

relevant sequence pairs: 107

The runtime was averaged over 100 independent runs. Our decomposition algorithm performs well, and is able to scale to the largest game (Goofspiel with k = 5 ranks, which has 3.6 × 107 relevant sequence pairs). In Figure 3(right) we used the characterization Ξ = V to compute the set of all payoffs that can be reached by an EFCE, EFCCE, or NFCCE in 3-rank Goofspiel

引用论文

- Itai Ashlagi, Dov Monderer, and Moshe Tennenholtz. On the value of correlation. Journal of Artificial Intelligence Research, 33:575–613, 2008.
- Benjamin Assarf, Ewgenij Gawrilow, Katrin Herr, Michael Joswig, Benjamin Lorenz, Andreas Paffenholz, and Thomas Rehn. Computing convex hulls and counting integer points with polymake. Mathematical Programming Computation, 9(1):1–38, Mar 2017. ISSN 1867-2957. doi: 10.1007/s12532-016-0104-z.
- Robert Aumann. Subjectivity and correlation in randomized strategies. Journal of Mathematical Economics, 1:67–96, 1974.
- Michael Bowling, Neil Burch, Michael Johanson, and Oskari Tammelin. Heads-up limit hold’em poker is solved. Science, 347(6218), January 2015.
- Noam Brown and Tuomas Sandholm. Safe and nested subgame solving for imperfectinformation games. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS), 2017.
- Noam Brown and Tuomas Sandholm. Superhuman AI for heads-up no-limit poker: Libratus beats top professionals. Science, page eaao1733, Dec. 2017.
- Noam Brown and Tuomas Sandholm. Superhuman AI for multiplayer poker. Science, 365 (6456):885–890, 2019.
- Andrea Celli, Stefano Coniglio, and Nicola Gatti. Computing optimal ex ante correlated equilibria in two-player sequential games. In International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2019.
- Andrea Celli, Alberto Marchesi, Tommaso Bianchi, and Nicola Gatti. Learning to correlate in multi-player general-sum sequential games. 2019.
- Gabriele Farina, Christian Kroer, and Tuomas Sandholm. Regret circuits: Composability of regret minimizers. In International Conference on Machine Learning, pages 1863–1872, 2019.
- Gabriele Farina, Chun Kai Ling, Fei Fang, and Tuomas Sandholm. Correlation in extensiveform games: Saddle-point formulation and benchmarks. In Conference on Neural Information Processing Systems (NeurIPS), 2019.
- Gabriele Farina, Chun Kai Ling, Fei Fang, and Tuomas Sandholm. Efficient regret minimization algorithm for extensive-form correlated equilibrium. In Conference on Neural Information Processing Systems (NeurIPS), 2019.
- Gabriele Farina, Tommaso Bianchi, and Tuomas Sandholm. Coarse correlation in extensiveform games. In AAAI Conference on Artificial Intelligence, 2020.
- Ewgenij Gawrilow and Michael Joswig. Polymake: a Framework for Analyzing Convex Polytopes, pages 43–73. Birkhäuser Basel, Basel, 2000. ISBN 978-3-0348-8438-9. doi: 10.1007/978-3-0348-8438-9_2.
- Itzhak Gilboa and Eitan Zemel. Nash and correlated equilibria: Some complexity considerations. Games and Economic Behavior, 1:80–93, 1989.
- Wan Huang and Bernhard von Stengel. Computing an extensive-form correlated equilibrium in polynomial time. In International Workshop on Internet and Network Economics, pages 506–513.
- Albert Jiang and Kevin Leyton-Brown. Polynomial-time computation of exact correlated equilibrium in compact games. In Proceedings of the ACM Conference on Electronic Commerce (EC), 2011.
- Marc Lanctot. Monte Carlo sampling and regret minimization for equilibrium computation and decision-making in large extensive form games. PhD thesis, 2013.
- Marc Lanctot, Kevin Waugh, Martin Zinkevich, and Michael Bowling. Monte Carlo sampling for regret minimization in extensive games. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS), 2009.
- Matej Moravcík, Martin Schmid, Neil Burch, Viliam Lisý, Dustin Morrill, Nolan Bard, Trevor Davis, Kevin Waugh, Michael Johanson, and Michael Bowling. Deepstack: Expert-level artificial intelligence in heads-up no-limit poker. Science, May 2017.
- H. Moulin and J.-P. Vial. Strategically zero-sum games: The class of games whose completely mixed equilibria cannot be improved upon. International Journal of Game Theory, 7(3-4): 201–221, 1978.
- Christos Papadimitriou and Tim Roughgarden. Computing equilibria in multi-player games. In Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 82–91, Vancouver, BC, Canada, 2005. SIAM.
- Sheldon M Ross. Goofspiel—the game of pure strategy. Journal of Applied Probability, 8(3): 621–625, 1971.
- Bernhard von Stengel and Françoise Forges. Extensive-form correlated equilibrium: Definition and computational complexity. Mathematics of Operations Research, 33(4):1002–1022, 2008.
- 1. Hence, all a ∈

下载 PDF 全文

标签

评论