Skip to content
The methodOpen data
Back to the collection

Move Evaluation in Go Using Deep Convolutional Neural Networks

Chris J. Maddison, Aja Huang, Ilya Sutskever, David Silver

2014 · arXiv

Computer visionMachine learningDatabases
Read the paper
HN score
0
Mentions
6
Readers
3
Months
4
What’s in a score?

From the abstract

The game of Go is more challenging than other board games, due to the difficulty of constructing a position or move evaluation function. In this paper we investigate whether deep convolutional networks can be used to directly represent and learn this knowledge. We train a large 12-layer convolutional neural network by supervised learning from a database of human professional games. The network correctly predicts the expert move in 55% of positions, equalling the accuracy of a 6 dan human player. When the trained convolutional network was used directly to play games of Go, without any search, it beat the traditional search program GnuGo in 97% of games, and matched the performance of a state-of-the-art Monte-Carlo tree search that simulates a million positions per move.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

6 selected excerpts · 6 total mentions

Jach
Neutral mention · rule
I read http://arxiv.org/abs/1412.6564 earlier this year and tldr they trained a deep net bot that managed to hold about 1-2d on KGS. Since there's no searching going on, that's pretty good, though not as good as CrazyStone or other MCTS bots. (I hadn't heard they went from 6d to 5d, but sure enough http://www.gokgs.com/graphPage.jsp?user=CrazyStone I wonder what happened?) I suspect combining the two methods will yield even stronger programs, I've heard some folks at Facebook and Google are each working on their own Go bots based on deep learning. If some company spent a few years building dedicated hardware like Deep Blue I even bet they could beat 9d pros with current methods. Go is close to falling.
Original comment Why Go Still Foils the Computers
deepnet
Neutral mention · rule
This Facebook Video suggests it has had months of either training or development, which sounds might be it learning by playing, reinforcement learning in this space could be very interesting - more details would be good. The two breakthrough papers using Deep Conv Nets to play Go - trained on Expert Games to predict the Experts Next Move. This is made possible by the huge archives of expert play from online Go servers that have reached Big Data sizes in the last few years. Teaching Deep Convolutional Neural Networks to Play Go - Clark & Storkey Edinburgh Informatics 2015 http://arxiv.org/abs/1412.3409 Move Evaluation in Go Using Deep Convolutional Neural Networks…
Read the full comment
Maddison, Huang, Silver, Sutskever Deepmind 2015 http://arxiv.org/abs/1412.6564 David Silver has work on Reinforcement Learning and Go: Reinforcement Learning of Local Shape in the Game of Go Silver et al 2007. https://scholar.google.co.uk/citations?view_op=view_citation... Go is scored by occupying and surrounding territory and the opponents pieces taken. The current State of the Art programs use Monte Carlo Tree Search with each move being evaluated using random playouts - this is well suited to Go as it reasnoble estimate for comparison of the territory that will result from a move. As the Go search tree branches a lot (~200 branches per play) tree search takes some time and cannot be exhaustive. The Expert Move Predicting Convnets provide very good moves very quickly, and can provide and provide probabilities the expert would move for each square in a single forward pass. Most of the current development now centers around using the Convnets to prune the search trees for the random playout MCTS engines. Go is useful from an AI perspective because it is a huge state space and the patterns are subtle with long term implications. No computer can yet beat the best humans - so it is an area where natural intelligence beats artificial. Yet the convnets are not doing planning, only predicting what an expert human would do next, the planning is implicit in the dataset but not a part of the convnet itself (though the internals of deepnets are mysterious so who knows). The Computer Go mailing list is very lively and full of the best program creators with regular computer tournaments against each other. http://computer-go.org/mailman/listinfo/computer-go The Facebook video shows GnuGo is the opposing player, this is not the best computer player and suggests Facebooks player is not as strong as the Convnets as yet - it also does not detail how deep the playouts of the tree search are, i.e. what level GnuGo is playing at, so this cannot really be compared to recent work. Hopefully Facebook will publish more details soon - if they are learning through play then their results may well be interesting. Remi Coulom's excellent slides detail the Monte Carlo revolution in Go programs. http://www.remi-coulom.fr/JFFoS/JFFoS.pdf AFAIK MCTS with Random playout and hand coded heuristics are still SOTA. When I visited in the summer Edinburgh Uni Informatics Department convnet Go was in active development.
Original comment Watch an AI bot play Go [video]
karmakaze
Neutral mention · rule
What I find shocking isn't only the level of play by machines, but also the number of techniques that can be applied to achieve it. Traditionally they were very domain knowledge based and now they tend to be hybrids. This for me really quantified how deep into the age of AI we're in. I never expected Go to be approched this quickly as the search space is so vast. AyaMC (AyaBot 3d on KGS) was Aya[1] rewritten as a Monte Carlo simulation. Many Faces of Go[2] (ManyFaces 2d on KGS) has been the strongest sold Go program which has transitioned to using Monte Carlo Tree Search. NiceGo19N, was Detlef Schmicker's latest version of oakfoam, using a Convolutional Neural Network.…
Read the full comment
The possibilty of using a CNN to play Go recently been described as promising, so it is good to see one such engine developed already. See this paper[3] for more information. Hiratuka No Igo (aka HiraBot[4] 1d on KGS). I don't know its algorithm but its game invitation message is "15000 playouts per move". [1] http://senseis.xmp.net/?Aya [2] http://senseis.xmp.net/?TheManyFacesOfGo [2] http://arxiv.org/abs/1412.6564 [4] http://senseis.xmp.net/?HiraBot
Original comment 112th KGS Computer Go Tournament
deepnet
Neutral mention · rule
Monte Carlo Tree Search ( Random playout ) is currently the best computer strategy for evaluating a Go position. This is likely due to the way Go works , random playout provides a rough estimate of who controls what territory ( this is how Go is scored ). Recently two deep-learning papers showed very impressive results. http://arxiv.org/abs/1412.3409 http://arxiv.org/abs/1412.6564 The neural networks were tasked with predicting what move an expert would make given a position. The MCTS takes a long time 100,000 playouts are typical - once trained the neural nets are orders of magnitude faster. The neural nets output a probability for each move ( that an expert would make that move ) - all positions are evauluated in a single forward pass. Current work centers around combining the two approaches, MCTS evaluates the best suggestions from the neural net. Expert Human players are still unbeatable by computer Go.
Original comment How Google Translate squeezes deep learning onto a phone
deepnet
Neutral mention · rule
*references Teaching Deep Convolutional Neural Networks to Play Go by Clarke & Storkey http://arxiv.org/abs/1412.3409 Move Evaluation in Go Using Deep Convolutional Neural Networks by Maddison , Huang , Silver & Sutskever http://arxiv.org/abs/1412.6564 Tesauro's TD-Gammon http://webdocs.cs.ualberta.ca/~sutton/book/ebook/node108.htm... Playing Atari with Deep Reinforcement Learning , Mnih et al. https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf Otoro's CNE Bibliography http://blog.otoro.net/2015/01/27/neuroevolution-algorithms/ Neuro Evolution - http://en.wikipedia.org/wiki/Neuroevolution
Original comment Show HN: Recurrent neural net plays 'neural slime volleyball' in JavaScript

Keep following the thread.

More in Computer vision

More from the same topics.

2017arXiv

The Case for Learned Index Structures

Tim Kraska, Alex Beutel, Ed H. Chi +2

Indexes are models: a B-Tree-Index can be seen as a model to map a key to the position of a record within a sorted array, a Hash-Index as a model to map a key to a position of a record within an unsorted array, and a BitMap-Index as a model to indicate if a data record exists or not.

Databases
19 mentions14 months
5HN score