![\"\"](\"https:\/\/world.edu\/wp-content\/uploads\/2017\/10\/google-s-go.jpg\")
<\/p>\n<\/p>\n
Just last year Google DeepMind\u2019s\u00a0AlphaGo<\/a>\u00a0took the world of Artificial Intelligence (AI) by storm, showing that a\u00a0computer program could beat the world\u2019s best human Go players<\/a>.<\/p>\n But in a demonstration of the feverish rate of progress in modern AI, details of a new milestone reached by an improved version called\u00a0AlphaGo Zero<\/a>\u00a0were\u00a0published this week in Nature<\/a>.<\/p>\n Using less computing power and only three days of training time, AlphaGo Zero beat the original AlphaGo in a 100-game match by 100 to 0. It wasn\u2019t even worth humans showing up.<\/p>\n Go is a game of strategy between two players who take it in turns to place \u201cstones\u201d on a 19×19 board. The goal is to surround a larger area of the board than your opponent.<\/p>\n The game of Go, simple to learn but a lifetime to master\u2026 for a human. Paragorn Dangsombroon\/Shutterstock<\/p><\/div>\n Go has proved much more challenging than chess for computers to master. There are many more possible moves in each position in Go than chess, and many more possible games.<\/p>\n The\u00a0original AlphaGo first learned<\/a>\u00a0from studying 30 million moves of expert human play. It then improved beyond human expertise by playing many games against itself, taking several months of computer time.<\/p>\n By contrast, AlphaGo Zero never saw humans play. Instead, it began by knowing only the rules of the game. From a relatively modest five million games of self-play, taking only three days on a smaller computer than the original AlphaGo, it then learned super-AlphaGo performance.<\/p>\n Fascinatingly, its learning roughly mimicked some of the stages through which humans progress as they master Go. AlphaGo Zero rapidly learned to reject naively short-term goals and developed more strategic thinking, generating many of the patterns of moves often used by top-level human experts.<\/p>\n But remarkably it then started rejecting some of these patterns in favour of new strategies never seen before in human play.<\/p>\n AlphaGo Zero achieved this feat by approaching the problem differently from the original AlphaGo. Both versions use a combination of two of the most powerful algorithms currently fuelling AI:\u00a0deep learning and reinforcement learning<\/a>.<\/p>\n To play a game like Go, there are two basic things the program needs to learn. The first is a policy: the probability of making each of the possible moves in a given position. The second is a value: the probability of winning from any given position.<\/p>\n In the pure reinforcement learning approach of AlphaGo Zero, the only information available to learn policies and values was for it to predict who might ultimately win. To make this prediction it used its current policy and values, but at the start these were random.<\/p>\n This is clearly a more challenging approach than the original AlphaGo, which used expert human moves to get a head-start on learning. But the earlier version learned policies and values with separate neural networks.<\/p>\n The algorithmic breakthrough in AlphaGo Zero was to figure out how these could be combined in just one network. This allowed the process of training by self-play to be greatly simplified, and made it feasible to start from a clean slate rather than first learning what expert humans would do.<\/p>\nLearning to play Go<\/strong><\/h4>\n
![\"\"](\"https:\/\/world.edu\/wp-content\/uploads\/2017\/10\/gameGo.jpg\")
Beyond human play<\/strong><\/h4>\n