Grounded Transfer for Reinforcement Learning Using Natural Language

In this work, we focus on utilizing textual descriptions of the environment to adapt and transfer control policies across domains.This question is motivated by a long-standing goal of reinforcement learning to acquire universal policies that can apply across a variety of tasks, where current methods have limited transfer capabilities. Natural language plays an important role in human exploration of the world, enabling compact transfer of knowledge and helping people generalize from their own and each other's experiences. We hypothesize that using natural language descriptions of the environment will help guide policy transfer for autonomous agents while providing an anchor for efficient exploration in the new domain. We focus on addressing two specific challenges -- (1) developing a common representation for the model dynamics of several different environments for use in reinforcement learning, and (2) developing a model to infer the dynamics of a new environment from specifications provided in natural language. We use a deep neural network for the general policy approximation, and condition the policy on the representation of the text descriptions of the environment. This framework allows us to bootstrap the agent's policy and guide exploration in new environments.