Thesis Defense: Balancing Exploration and Exploitation: Loss-Targeted Exploration for Scientific Decision-Making


Genevieve Flaspohler


John Fisher and Nicholas Roy


How do we collect observational data that reveal fundamental properties of scientific phenomena? This is a key challenge in modern scientific decision-making. Scientific phenomena are complex---they have high-dimensional and continuous state, exhibit chaotic or complex dynamics, and generate incomplete or noisy sensor observations. This thesis argues that autonomous decision-making in real-world, scientific domains requires loss-targeted exploration---exploration strategies that are tuned to a specific task or loss function. By explicitly quantifying the change in task performance due to exploratory actions, we enable decision-makers that explore parsimoniously and contend with highly uncertain real-world environments. We develop novel algorithms for loss-targeted exploration in partially observable Markov decision processes (POMDPs) and online learning problems. These methods are motivated by and applied to real-world scientific applications, including robotic deep sea hydrothermal plume discovery and climate and weather forecasting. This thesis demonstrates that autonomous decision-making can enhance human scientific discovery, placing sensors in the right place at the right time to validate a hypothesis or collect a critical observation.

Thesis Supervisor(s): John Fisher and Nicholas Roy
Committee: John Fisher (MIT), Nicholas Roy (MIT), Claudia Cenedese (Woods Hole Oceanographic Institution), Munzer Dahleh (MIT), and Lester Mackey (MSR New England)