Learning to Cooperate and Compete in Diplomacy
Host
Jacob Andreas
CSAIL
This is a hybrid talk: for in-person, come to 32-262; for zoom, come to https://mit.zoom.us/j/93541413653?pwd=U1FERXFrVHdBTWh1cmNYUENxZEVQUT09.
AI has made incredible progress in purely adversarial games such as chess, go, and poker. However, the real world involves a complex mixture of cooperation and competition, sometimes with irrational or suboptimal participants, and in these settings past AI techniques fall apart. For this reason, Diplomacy, a popular game focused on negotiation and alliance-building, has recently emerged as a grand challenge for AI that is far more difficult than any prior game and has far greater implications if AI algorithms eventually succeed. In this talk I will describe Diplomacy and cover recent research results from FAIR, DeepMind, and MILA that make progress on the no-communication version of this game. In particular, I will present results strongly suggesting that self play alone is incapable of reaching superhuman performance in this game, unlike in purely adversarial and purely cooperative games. I will also present results showing that a combination of supervised learning on human data, equilibrium-finding search, and reinforcement learning can nevertheless achieve a strong human level of play in no-communication Diplomacy. Finally, I will conclude with thoughts on the challenges that await as research shifts from the no-communication version of Diplomacy to versions that involve private communication between the players.
Noam Brown is a Research Scientist at Facebook AI Research working on multi-agent artificial intelligence and computational game theory, with a particular focus on sequential imperfect-information games. He co-created Libratus and Pluribus, the first AIs to defeat top humans in two-player no-limit poker and multiplayer no-limit poker, respectively. He has received the Marvin Minsky Medal for Outstanding Achievements in AI, was named one of MIT Tech Review's 35 Innovators Under 35, and his work on Pluribus was named by Science to be one of the top 10 scientific breakthroughs of 2019. Noam received his PhD from Carnegie Mellon University, for which he received the AAMAS Victor Lesser Distinguished Dissertation Award and the CMU School of Computer Science Distinguished Dissertation Award.
AI has made incredible progress in purely adversarial games such as chess, go, and poker. However, the real world involves a complex mixture of cooperation and competition, sometimes with irrational or suboptimal participants, and in these settings past AI techniques fall apart. For this reason, Diplomacy, a popular game focused on negotiation and alliance-building, has recently emerged as a grand challenge for AI that is far more difficult than any prior game and has far greater implications if AI algorithms eventually succeed. In this talk I will describe Diplomacy and cover recent research results from FAIR, DeepMind, and MILA that make progress on the no-communication version of this game. In particular, I will present results strongly suggesting that self play alone is incapable of reaching superhuman performance in this game, unlike in purely adversarial and purely cooperative games. I will also present results showing that a combination of supervised learning on human data, equilibrium-finding search, and reinforcement learning can nevertheless achieve a strong human level of play in no-communication Diplomacy. Finally, I will conclude with thoughts on the challenges that await as research shifts from the no-communication version of Diplomacy to versions that involve private communication between the players.
Noam Brown is a Research Scientist at Facebook AI Research working on multi-agent artificial intelligence and computational game theory, with a particular focus on sequential imperfect-information games. He co-created Libratus and Pluribus, the first AIs to defeat top humans in two-player no-limit poker and multiplayer no-limit poker, respectively. He has received the Marvin Minsky Medal for Outstanding Achievements in AI, was named one of MIT Tech Review's 35 Innovators Under 35, and his work on Pluribus was named by Science to be one of the top 10 scientific breakthroughs of 2019. Noam received his PhD from Carnegie Mellon University, for which he received the AAMAS Victor Lesser Distinguished Dissertation Award and the CMU School of Computer Science Distinguished Dissertation Award.