Skip to content
The methodOpen data
Back to the collection

Giraffe: Using Deep Reinforcement Learning to Play Chess

Matthew Lai

2015 · arXiv

Machine learningPsychology
Read the paper
HN score
0
Mentions
7
Readers
7
Months
5
What’s in a score?

From the abstract

This report presents Giraffe, a chess engine that uses self-play to discover all its domain-specific knowledge, with minimal hand-crafted knowledge given by the programmer. Unlike previous attempts using machine learning only to perform parameter-tuning on hand-crafted evaluation functions, Giraffe's learning system also performs automatic feature extraction and pattern recognition. The trained evaluation function performs comparably to the evaluation functions of state-of-the-art chess engines - all of which containing thousands of lines of carefully hand-crafted pattern recognizers, tuned over many years by both computer chess experts and human chess masters. Giraffe is the most successful attempt thus far at using end-to-end machine learning to play chess.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

7 selected excerpts · 7 total mentions

gwern
Neutral mention · rule
It's certainly not the first NN chess program. You may remember one of OP author's Giraffe NN (https://arxiv.org/abs/1509.01549) which was essentially 'AlphaGo for chess'. But like the original AG, it struggles to learn and Lai had a lot less computation as a student than he does now at DM. What they're doing is applying AlphaGo Zero expert iteration with some simplifications and TPUs. And that pwns previous work like Giraffe the way AlphaGo Zero pwns AlphaGo. Quantity becomes a quality all its own.
Original comment Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
kyberias
Neutral mention · rule
> They're useless to you because you don't seem to understand how neural network or current AI tech works. In the current fashion, generalist AI like a game engine is very much impossible. I have some understanding how that works. You seem to read me incorrectly and extend what I say to something different. Again, and I repeat myself, I don't claim "generalist AI". I don't claim that we can have a machine learning algorithm that learns ANY game mechanics perfectly just by looking at limited amount game play. Obviously that's not true for any neural network including human brain. What I do claim is that we can have a machine learning algorithm that can learn some…
Read the full comment
interesting game mechanics just by looking at game play. That is what the researchers intended and got some interesting results. > Game states are too large to be entirely seen even by a computer. We need pruning and MCTS for simple adversarial searches in games like chess, Go; What makes you think some magical AI can imitate a game engine? Ok, you may have a different definition of "game state" than I do. My definition (which I think is the correct one): The state of the variables in a game program that define the state of the game. The game state may include variables that contain the position of monsters, player's score etc. The game state can be used Game state is NOT all the possible game states of a particular game. That's called game state space. I'm sure you know that, I'm just being pedantic of course. Now let us consider the game of Chess, which is not the best example, but let's do it anyway. The game state in chess is basically the position of the chess pieces on a board. That is about 64 variables. It is conceivable that we could use a machine learning algorithm to learn the _rules_ of the game and how to render the current state of the chess game ("game engine") just by showing the algorithm valid input (player move, image of the game board before move) and output (image of the game board after move) of the game. Can we learn the rules of chess perfectly? Well, that might be hard, I fully understand that. We should carefully select all the interesting chess game positions to make sure the machine learns everything correctly and we would never be sure. But that is not what we're trying to do here! We're trying to use machine learning to get some useful and possibly creative game engines without manually defining the rules (programming). The idea is not to replicate something existing perfectly. That was never my claim. BTW, although this is not exactly what we're discussing here, Matthew Lai has done some interesting research on this topic. [1] "Instead of the conventional method of teaching computers how to play chess by giving them hardcoded rules, this project is an attempt to use machine learning to figure out how to play chess through self-play, and have them derive their own rules from the games." > What makes you think some magical AI can imitate a game engine? Your rhetorical question seems to assume that I've said something I haven't said. If you have an argument, state that in sentences. Let me repeat. A neural network (or some other machine learning model) can learn to do similar things than a game engine does, namely take user input, maintain game state and render game state. Therefore this model can be considered as a game engine for a given game being modeled. That's the extent of my argument. Nothing more. I don't think we need any magic with this. [1] https://arxiv.org/abs/1509.01549
Original comment AI Uses Less Than Two Minutes of Videogame Footage to Recreate Game Engine
naveen99
Neutral mention · rule
There was a project for chess, called giraffe https://bitbucket.org/waterreaction/giraffe whose author shut it down after joining google deep mind: http://www.talkchess.com/forum/viewtopic.php?t=59003 He thinks it's only a matter of time till machine learning beats hand crafted systems like stockfish even in chess. http://arxiv.org/abs/1509.01549
Original comment Alpha Go Zero: How and Why It Works
shmageggy
Neutral mention · rule
I didn't see the talk, but I'm guessing he was referring to the Giraffe engine done by Matthew Lai (https://arxiv.org/abs/1509.01549). The main thing there is that he only learns an evaluation function, not a policy. Giraffe still uses classical alpha-beta search over the full action space. AFAIK nobody has learned a decent policy network for chess, probably because 1) it's super tactical, and 2) nobody cares that much because alpha-beta is so strong
Original comment AlphaGo Zero: Learning from scratch
jhartmann
Neutral mention · rule
Seems very interesting, I really think these approaches will work rather well on a variety of games. Very exciting times. Btw, there is another already Open Source Neural Network Chess engine that might be worth exploring for similar ideas. That codebase already has a minmax search based on the network evaluation. Paper is at: http://arxiv.org/abs/1509.01549 Code is at: https://bitbucket.org/waterreaction/giraffe
Original comment Spawkfish: Neural network chess
ColinWright
Neutral mention · rule
Discussion? https://news.ycombinator.com/item?id=10216710 : 2 comments https://news.ycombinator.com/item?id=10230201 : 1 comment https://news.ycombinator.com/item?id=10216715 : 1 comment https://news.ycombinator.com/item?id=10219759 : 2 comments ======== Full summary: http://arxiv.org/abs/1509.01549 Deep Learning engine teaches itself to play chess to Master level (arxiv.org) https://news.ycombinator.com/item?id=10217320 https://news.ycombinator.com/item?id=10219759 ======== http://fusion.net/story/197214/chess-master-deep-learning/ A computer taught itself to play chess at master level in just three days…
Read the full comment
(fusion.net) https://news.ycombinator.com/item?id=10217931 https://news.ycombinator.com/item?id=10227560 ======== http://trendspot.it/de03 Deep Learning Machine Teaches Itself Chess Plays at Master Level (trendspot.it) https://news.ycombinator.com/item?id=10216894 ======== http://www.technologyreview.com/view/541276/deep-learning-ma... AI Teaches Itself Chess in 72 Hours, Plays at International Master Level (technologyreview.com) https://news.ycombinator.com/item?id=10216715 https://news.ycombinator.com/item?id=10232115 https://news.ycombinator.com/item?id=10221277 ======== http://www.technologyreview.com/view/541276/deep-learning-ma... Deep Learning Machine Teaches Itself Chess in 72 Hours, Plays at Master Level (technologyreview.com) https://news.ycombinator.com/item?id=10224620 https://news.ycombinator.com/item?id=10215597 https://news.ycombinator.com/item?id=10215722 https://news.ycombinator.com/item?id=10216710 https://news.ycombinator.com/item?id=10232115 https://news.ycombinator.com/item?id=10214988 https://news.ycombinator.com/item?id=10215546 https://news.ycombinator.com/item?id=10215675 https://news.ycombinator.com/item?id=10217963 https://news.ycombinator.com/item?id=10218149 https://news.ycombinator.com/item?id=10220272 https://news.ycombinator.com/item?id=10220568 https://news.ycombinator.com/item?id=10230201 https://news.ycombinator.com/item?id=10215927
Original comment In just 72 hours a deep learning engine has learned chess to Master level
cs702
Neutral mention · rule
Interestingly, the neural net used is not really that deep: it has only two hidden layers (see figure 3 in the paper[1]). The innovations are really in how the researcher is representing inputs using different "modalities" (see section 4.1 "Feature Representation"), generating a training corpus (section 4.3), and training the neural net (sections 4.4 and 4.5). [1] http://arxiv.org/pdf/1509.01549v2.pdf
Original comment Deep Learning Machine Teaches Itself International Master Level Chess in 72 HRs

Keep following the thread.

More in Machine learning

More from the same topics.

2018arXiv

Talent vs Luck: the role of randomness in success and failure

A. Pluchino. A. E. Biondo, A. Rapisarda

The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.

Psychology
16 mentions14 months
2.5HN score