AI Reading Group | Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model

Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart et al.

​With this session we begin a new 3-part mini-track on World Models, led by Leo Pinetzki. Moving away from alignment, we now ask: how can an agent learn an internal model of its environment and use it to plan?

​Our paper is Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model (Schrittwieser et al., 2020) — better known as MuZero.

​Previous systems like AlphaZero achieved superhuman performance at board games, but relied on access to a perfect simulator — they needed to know the rules. MuZero drops that requirement entirely.

​MuZero is a natural starting point for the world models track: What does it mean for an agent to "understand" its environment if its learned model isn't human-interpretable? How far can you get by only modelling what matters for the task at hand? And what are the limits of this approach when we move beyond games to messier, open-ended domains?