Reproducibility in Deep Reinforcement Learning and Beyond

Abstract:
In recent years, significant progress has been made in solving challenging problems across various domains using deep reinforcement learning (RL). Unfortunately, reproducing results for state-of-the-art deep RL methods is seldom straightforward. In particular, non-determinism in standard benchmark environments, combined with variance intrinsic to the methods, can make reported results tough to interpret. Without significance metrics and tighter standardization of experimental reporting, it is difficult to determine whether improvements over the prior state-of-the-art are meaningful. In this talk, I will discuss challenges posed by reproducibility, proper experimental techniques, and reporting procedures. I will illustrate the variability in reported metrics and results when comparing against common baselines and suggest guidelines to make future results in deep RL more reproducible. I will also comment on findings from the ICLR 2018 reproducibility challenge.

Bio:
Joelle Pineau is an Associate Professor and William Dawson Scholar at McGill University where she co-directs the Reasoning and Learning Lab. She is also the director of the Facebook Artificial Intelligence Research Lab in Montreal. Dr. Pineau’s research focuses on developing new models and algorithms for planning and learning in complex partially-observable domains. She also works on applying these algorithms to complex problems in robotics, health care, games and conversational agents. She serves on the editorial board of the Journal of Artificial Intelligence Research and the Journal of Machine Learning Research and is currently President of the International Machine Learning Society. She is a AAAI Fellow, a Senior Fellow of CIFAR, and a member of the College of New Scholars, Artists and Scientists by the Royal Society of Canada.