The Centre for Artificial Intelligence (CAI) are delighted to be welcoming André Barreto, from Deepmind, to deliver his seminar, "Generalised policy updates: From successor features to behaviour foundation models."

Abstract

Generalised policy evaluation (GPE) and generalised policy improvement (GPI) let an agent evaluate a set of policies across many tasks and combine them into a single, improved policy. In this talk I will present these generalised policy updates as a unifying framework for reinforcement learning involving multiple tasks. I will show how universal value function approximators, successor features and geometric horizon models can all be seen as instances of GPE, and offer a geometric interpretation of GPI that sheds light on how it works. Once the framework is laid out, I will discuss the two fundamental challenges underlying it: how to learn features and how to define a good policy basis. I will describe concrete approaches to both. I will then connect these ideas to behaviour foundation models (BFMs) and show how they address the same two challenges. The talk will close with open problems that I find particularly interesting.