Robust training of deep neural networks for visual navigation requires a large amount of data.

However, gathering and labeling this data can be expensive and demonstrated data may not cover the distribution of experiences an agent (network) will encounter in application.  We address this by growing a training set throughout an agents life from its own experiences and mistakes.  This allows the agent to efficiently adjust to changing circumstances (motion dynamics, visual features, etc) in complex environments.  We apply our method to the task of navigating 3D mazes in Minecraft with randomly changing block types.