[Thesis Defense] Generalization Under Distribution Shift

Machine learning systems have achieved remarkable performance in tasks where test data closely resembles the training distribution. However, real-world applications often require systems capable of handling more challenging situations—specifically, extrapolating to data points outside the distribution of the training set. In this talk, I will present two frameworks that enable machine learning models to generalize effectively to out-of-distribution scenarios without sacrificing the power of modern overparameterized models.

The first part of the talk focuses on few-shot task learning, which involves agents learning new tasks from minimal data and applying them to new environments. We formulate the problem of few-shot task learning as Few-Shot Task Learning through Inverse Generative Modeling which allows us to leverage the power of neural generative models, pre-trained on a set of base tasks. We adapt a method for efficient concept learning to few-shot task learning based on our formulation and rapidly learn new tasks with only a few examples, enabling task execution from autonomous driving to real-world robotic manipulation tasks in novel settings without the need for extensive retraining.

In the second part of the talk, I will discuss a method that converts an out-of-support generalization problem into an out-of-combination problem via a transductive reparameterization, which is possible under low-rank style conditions. I will then explore how this idea can be applied to domains like robotics, where the environment is changing, and materials and molecular design, where predicting properties of materials and molecules outside of known ranges is crucial to driving more efficient materials discovery.