Model-based reinforcement learning under uncertainty: the importance of knowing what you don't know

Abstract

In this guest lecture I present the role of uncertainty quantification in model-based RL. I show how to quantify uncertainty in dynamics models and how to use it to 1) reduce “model bias” and 2) target exploration. I highlight the importance of disentangling the different sources of uncertainty (epistemic/aleatoric). Finally, I present some of the papers that pioneered these ideas.