Rigorously Tested & Reliable Machine Learning for Health

Abstract: How do we make machine learning as rigorously tested and reliable as any medication or diagnostic test? Machine learning (ML) has the potential to improve decision-making in healthcare, from predicting treatment effectiveness to diagnosing disease. However, standard retrospective evaluations can give a misleading sense for how well models will perform in practice. Evaluation of ML-derived treatment policies can be biased when using observational data, and predictive models that perform well in one hospital may perform poorly in another. In this talk, I will introduce new tools to proactively assess and improve the reliability of machine learning in healthcare. A central theme will be the application of external knowledge, including review of patient records, incorporation of limited clinical trial data, and interpretable stress tests. Throughout, I will discuss how evaluation can directly inform model design.

Thesis Committee: David Sontag (MIT), Jonas Peters (ETH Zurich), Tommi Jaakkola (MIT)