Skip to content
The methodOpen data
Back to the collection

Superhuman AI for multiplayer poker

Noam Brown, Tuomas Sandholm

2019 · Science

Psychology
Read the paper
HN score
0
Mentions
6
Readers
6
Months
5
What’s in a score?

From the abstract

AI now masters six-player poker Computer programs have shown superiority over humans in two-player games such as chess, Go, and heads-up, no-limit Texas hold'em poker. However, poker games usually include six players—a much trickier challenge for artificial intelligence than the two-player variant. Brown and Sandholm developed a program, dubbed Pluribus, that learned how to play six-player no-limit Texas hold'em by playing against five copies of itself (see the Perspective by Blair and Saffidine). When pitted against five elite professional poker players, or with five copies of Pluribus playing against one professional, the computer performed significantly better than humans over the course of 10,000 hands of poker. Science , this issue p. 885 ; see also p. 864

Metadata: Crossref. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

6 selected excerpts · 6 total mentions

aelaguiz
Neutral mention · rule
This is my area of expertise. I love the experiment. In general games of imperfect information such as Poker, Diplomacy, etc are much much harder than perfect information games such as Chess. Multiplayer (3+) poker in particular is interesting because you cannot achieve a nash equilibrium (e.g. it is not zero sum). That is part of the reason they are a fantastic venue for exploration of the capabilities of LLMs. They also mirror the decision making process of real life. Bezos framed it as "making decisions with about 70% of the information you wish you had." As it currently stands having built many poker AIs, including what I believe to be the current best in the world, I…
Read the full comment
don't think LLMs are remotely close to being able to do what specialized algorithms can do in this domain. All of the best poker AI's right now are fundamentally based on counter factual regret minimization. Typically with a layer of real time search on top. Noam Brown (currently director of research at OpenAI) took the existing CFR strategies which were fundamentally just trying to scale at train time and added on a version of search, allowing it to compute better policies at TEST TIME (e.g. when making decisions). This ultimately beat the pros (Pluribus beat the pros at 6 max in 2018 I believe). It stands as the state of the art, although I believe that some of the deep approaches may eventually topple it. Not long after Noam joined OpenAI they released the o1-preview "thinking" models, and I can't help but think that he took some of his ideas for test time compute and applied them on top of the base LLM. It's amazing how much poker AI research is actually influencing the SOTA AI we see today. I would be surprised if any general purpose model can achieve true human level or super human level results, as the purpose built SOTA poker algorithms at this point play substantially perfect poker. Background: - I built my first poker AI when I was in college, made half a million bucks on party poker. It was a pseudo expert system. - Created PokerTableRatings.com and caught cheaters at scale using machine learning on a database of all poker hands in real time - Sold my poker AI company to Zynga in 2011 and was Zynga Poker CTO for 2 years pre/post IPO - Most recently built a tournament version of Pluribus (https://www.science.org/doi/10.1126/science.aay2400). Launching as duolingo for poker at pokerskill.com
Original comment Poker Tournament for LLMs
slymon99
Neutral mention · rule
The other answers here are fairly uninformed. AI has been reliably beating humans in NHLE since 2019 with just deep learning [1], modern solvers can play within a small epsilon of a nash equilibrium (a perfect strategy) and are effectively unbeatable. Referencing a solver while playing (what people call real-time assistance or RTA) is definitely a problem in online poker and is always prohibited. Solvers however play in a fairly predictable way and poker sites can detect if people are using them, though I'd imagine imperfectly. Saying that 99% of online poker is vs AIs is a hilarious overstatement. [1] https://www.science.org/doi/10.1126/science.aay2400
Original comment An introduction to the theory and practice of poker (2020)
cthalupa
Neutral mention · rule
That's sort of a no-true-scotsman argument: They lose because they're not playing true GTO. Well, yes, but the point is that a fully GTO non-exploitable strategy is not solved for Hold'em. At least not outside of limit (and kinda no-limit) heads up play. https://arxiv.org/pdf/1805.08195.pdf (There is also an argument that Libratus actually just approaches Nash equilibrium and isn't fully there, since there are 1e+160 decision points in a heads up no-limit game, but compared to human players the difference is probably meaningless. ) The closest we have seen for larger games is Pluribus - https://www.science.org/doi/10.1126/science.aay2400 - but the researchers there aren't…
Read the full comment
even attempting to say they have solved 6max, just that they could be less exploitable than some of the best in the world. It is not particularly useful for online play because it does not try to counter variance or manage bankroll, and in further results it began to lose quite heavily as the pros adapted to it, losing 700BB over the 10k hands. By no means am I saying GTO-based strategy is ineffective - I'd probably be wasting my DTO subscription if I did think so ;) - but we're only part of the way there and the missing bits mean that you can't play perfectly unexploitable poker.
Original comment How to make decisions like a poker player
jessriedel
Neutral mention · rule
Where is the variance-reduction technique discussed? I looked at this paper https://www.science.org/doi/abs/10.1126/science.aay2400 https://par.nsf.gov/servlets/purl/10077416 and it just says > Finally, we tested Libratus against top humans. In January 2017, Libratus played against a team of four top HUNL specialist professionals in a 120,000 hand Brains vs. AI challenge match over 20 days. The participants were Jason Les, Dong Kim, Daniel McCauley, and Jimmy Chou. A prize pool of $200,000 was allocated to the four humans in aggregate. Each human was guaranteed $20,000 of that pool. The remaining $120,000 was divided among them based on how much better the human did…
Read the full comment
against Libratus than the worstperforming of the four humans. Libratus decisively defeated the humans by a margin of 147 mbb/hand, with 99.98% statistical significance and a p-value of 0.0002 (if the hands are treated as independent and identically distributed), see Fig. 3 (57). It also beat each of the humans individually.
Original comment How A.I. Conquered Poker
noambrown
Neutral mention · rule
Bots are superhuman in no-limit Texas hold'em. Libratus beat top humans in two-player in 2017 and Pluribus beat top humans in six-player in 2019: https://www.science.org/doi/abs/10.1126/science.aao1733 https://www.science.org/doi/abs/10.1126/science.aay2400 It's shocking that the reporter didn't mention these results or anything else more recent than 2015.
Original comment How A.I. Conquered Poker
onekorg
Neutral mention · rule
I read the 2019 paper[0] where they beat 5 pros in a 6-max table. It did not assume fixed stack sizes, it uses a technique they call "action abstraction" where they train on some number of stack sizes. During a game they use these pre-computed values as starting points for real time searching of the game tree. Their papers are very well written. With enough determination someone could build a multi player bot. I was also thinking about trying to build something simpler that works decently well. They published a paper a while back about a bot that only did real time search with no training. It performed better than older poker bots that used to require offline computation. I don't think you'd be able to beat pros with it, but it should do well vs amateurs. [0] https://www.science.org/doi/10.1126/science.aay2400
Original comment A $10k poker win changed how I think

Keep following the thread.

More in Psychology

More from the same topics.

2018arXiv

Talent vs Luck: the role of randomness in success and failure

A. Pluchino. A. E. Biondo, A. Rapisarda

The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.

Psychology
16 mentions14 months
2.5HN score