Managing Exploratory AI

Abstract: Today’s data science systems, ranging from batch jobs to interactive interfaces, are surprisingly fragile. Data scientists typically use dozens of libraries, but a single bug in any of them can destroy hours or even days of computation, causing significant pain. This issue has been widely discussed in the data science community and academic literature. Yet, no principled mechanisms have been proposed to address the issue. It may be puzzling to database researchers. Existing databases implement checkpointing to periodically save changes for future recovery. Why haven’t data science systems adopted it? Are there any unique properties that challenge the adoption? 

In this talk, I will first identify a core challenge: the lack of mechanisms for detecting data changes, a key premise of checkpointing. Unlike databases with centralized buffer pools, data science systems intentionally omit centralized data spaces, allowing individual libraries to use shared memory, GPUs, and remote machines. Changes across these diverse locations must still be identified. To address this, we are making exciting progress around one central theme: a nonintrusive state manager that behaves like conventional buffer pools without requiring data to be placed in a central location. The key idea is to construct a mathematical map of library-managed data—including data dependencies—using graphs. These graphs enable new algorithms to detect changes, save them incrementally, and restore states correctly. We are actively developing an open-source system, Kishu, to benefit all data practitioners.

Bio: Yongjoo Park is an Assistant Professor in the School of Computing and Data Science at the University of Illinois at Urbana-Champaign. His research focuses on systems for data-intensive AI. Yongjoo is also a Chief Scientist of Keebo, a start-up company he co-founded based on his Ph.D. research. Yongjoo obtained a Ph.D. in Computer Science and Engineering from the University of Michigan, Ann Arbor. He is a recipient of 2018 SIGMOD Jim Gray Dissertation Honorable Mention and ACM SIGMOD 2023 Best Artifact Award Honorable Mention.

-- For the zoom passcode, contact the organizer at markakis@mit.edu