Skip to content
The methodOpen data
Back to the collection

Student of Games: A unified learning algorithm for both perfect and imperfect information games

Martin Schmid, Matej Moravcik, Neil Burch

Show all 13 authors

Rudolf Kadlec, Josh Davidson, Kevin Waugh, Nolan Bard, Finbarr Timbers, Marc Lanctot, G. Zacharias Holland, Elnaz Davoodi, Alden Christianson, Michael Bowling

2021 · arXiv

AlgorithmsMachine learningPsychology
Read the paper
HN score
0
Mentions
3
Readers
3
Months
3
What’s in a score?

From the abstract

Games have a long history as benchmarks for progress in artificial intelligence. Approaches using search and learning produced strong performance across many perfect information games, and approaches using game-theoretic reasoning and learning demonstrated strong performance for specific imperfect information poker variants. We introduce Student of Games, a general-purpose algorithm that unifies previous approaches, combining guided search, self-play learning, and game-theoretic reasoning. Student of Games achieves strong empirical performance in large perfect and imperfect information games -- an important step towards truly general algorithms for arbitrary environments. We prove that Student of Games is sound, converging to perfect play as available computation and approximation capacity increases. Student of Games reaches strong performance in chess and Go, beats the strongest openly available agent in heads-up no-limit Texas hold'em poker, and defeats the state-of-the-art agent in Scotland Yard, an imperfect information game that illustrates the value of guided search, learning, and game-theoretic reasoning.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

3 selected excerpts · 3 total mentions

gwern
Neutral mention · rule
The standard AlphaZero doesn't handle that. For that you'd need to graduate to more complex variants like the aforementioned ReBeL, AlphaZe* https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10213697/ or BetaZero https://arxiv.org/abs/2306.00249 or ExIt-OOS https://arxiv.org/abs/1808.10120 or Player of Games https://arxiv.org/abs/2112.03178#deepmind . (You could also move straight to MuZero variations: https://arxiv.org/abs/2106.04615#deepmind https://openreview.net/forum?id=X6D9bAHhBQ1#deepmind https://openreview.net/forum?id=QnzSSoqmAvB )
Original comment Show HN: Easily train AlphaZero-like agents on any environment you want!
thomasahle
Neutral mention · rule
Bayesian play is not necessarily optimal for imperfect information games. The reason is: You don't only need to play optimally with respect to the information you have observed, you also need to hide your own information and balance those two needs. See the Deep Mind "Player of Games" paper from last year for an agent that takes a more game theoretic approach, which is probably needed for "simpler" games like Poker, that we can play to higher levels of accuracy: https://arxiv.org/pdf/2112.03178.pdf
Original comment Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning
splonk
Neutral mention · rule
Here's a recent paper from DeepMind about their recent efforts towards generalized game solving (caveat: I haven't actually read it yet). https://arxiv.org/pdf/2112.03178.pdf Abstract: "Games have a long history of serving as a benchmark for progress in artificial intelligence. Recently, approaches using search and learning have shown strong performance across a set of perfect information games, and approaches using game-theoretic reasoning and learning have shown strong performance for specific imperfect information poker variants. We introduce Player of Games, a general-purpose algorithm that unifies previous approaches, combining guided search, self-play learning, and…
Read the full comment
game-theoretic reasoning. Player of Games is the first algorithm to achieve strong empirical performance in large perfect and imperfect information games — an important step towards truly general algorithms for arbitrary environments. We prove that Player of Games is sound, converging to perfect play as available computation time and approximation capacity increases. Player of Games reaches strong performance in chess and Go, beats the strongest openly available agent in heads-up no-limit Texas hold’em poker (Slumbot), and defeats the state-of-the-art agent in Scotland Yard, an imperfect information game that illustrates the value of guided search, learning, and game-theoretic reasoning"
Original comment How A.I. Conquered Poker

Keep following the thread.

More in Algorithms

More from the same topics.

2018arXiv

Talent vs Luck: the role of randomness in success and failure

A. Pluchino. A. E. Biondo, A. Rapisarda

The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.

Psychology
16 mentions14 months
2.5HN score