Teaching Deep Convolutional Neural Networks to Play Go

Maddison, Huang, Silver, Sutskever Deepmind 2015 http://arxiv.org/abs/1412.6564 David Silver has work on Reinforcement Learning and Go: Reinforcement Learning of Local Shape in the Game of Go Silver et al 2007. https://scholar.google.co.uk/citations?view_op=view_citation... Go is scored by occupying and surrounding territory and the opponents pieces taken. The current State of the Art programs use Monte Carlo Tree Search with each move being evaluated using random playouts - this is well suited to Go as it reasnoble estimate for comparison of the territory that will result from a move. As the Go search tree branches a lot (~200 branches per play) tree search takes some time and cannot be exhaustive. The Expert Move Predicting Convnets provide very good moves very quickly, and can provide and provide probabilities the expert would move for each square in a single forward pass. Most of the current development now centers around using the Convnets to prune the search trees for the random playout MCTS engines. Go is useful from an AI perspective because it is a huge state space and the patterns are subtle with long term implications. No computer can yet beat the best humans - so it is an area where natural intelligence beats artificial. Yet the convnets are not doing planning, only predicting what an expert human would do next, the planning is implicit in the dataset but not a part of the convnet itself (though the internals of deepnets are mysterious so who knows). The Computer Go mailing list is very lively and full of the best program creators with regular computer tournaments against each other. http://computer-go.org/mailman/listinfo/computer-go The Facebook video shows GnuGo is the opposing player, this is not the best computer player and suggests Facebooks player is not as strong as the Convnets as yet - it also does not detail how deep the playouts of the tree search are, i.e. what level GnuGo is playing at, so this cannot really be compared to recent work. Hopefully Facebook will publish more details soon - if they are learning through play then their results may well be interesting. Remi Coulom's excellent slides detail the Monte Carlo revolution in Go programs. http://www.remi-coulom.fr/JFFoS/JFFoS.pdf AFAIK MCTS with Random playout and hand coded heuristics are still SOTA. When I visited in the summer Edinburgh Uni Informatics Department convnet Go was in active development.

that computers have yet to master." Deep Blue didn't have anything to master, if he can beat the world champion that was it! A rough analogy would be: A bird can fly by flapping its wings 5 times a second while a Cessna 172's propeller has to make 200 revolutions per minute (numbers made up), so we still have some avian tricks to master. They are two different approaches to a problem! At the time, Deep Blue required a 32-node IBM RS/6000 SP high-performance computer (https://www.research.ibm.com/deepblue/meet/html/d.3.shtml) for its power, now a regular MBP can run an instance of Stockfish that would give a GM a good run for its money. Now, if you can design an AI that can learn a comparably simpler board game, say, Settlers of Catan together with a human (not fed millions of games) and can play with reasonable strategy, that would be a teaching itself how to play.

drawing things [16] and many more. [1] http://blog.kaggle.com/2015/07/27/taxi-trajectory-winners-in... [2] http://blog.kaggle.com/2012/11/01/deep-learning-how-i-did-it... [3] http://arxiv.org/abs/1403.6652 [4] http://arxiv.org/abs/1502.03240 [5] http://arxiv.org/abs/1411.4734 [6] http://arxiv.org/abs/1409.0473 [7] http://arxiv.org/abs/1502.03044 [8] http://arxiv.org/abs/1410.5401 [9] http://arxiv.org/abs/1506.02216 [10] http://arxiv.org/abs/1507.04808 [11] http://arxiv.org/abs/1503.08895 [12] http://arxiv.org/abs/1308.0850 [13] http://arxiv.org/abs/1412.3409 [14] https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf [15] http://www.sciencedirect.com/science/article/pii/S0893608014... [16] https://www.youtube.com/watch?v=Zt-7MI9eKEo