MIT CSAIL alumnus Jonathan Frankle PhD ‘23 has received the 2023 AAAI/ACM SIGAI Doctoral Dissertation Award for his empirical work studying how we can build smaller, more efficient neural networks — essentially, machine-learning systems that use nodes to recognize patterns.
From chatbots to image recognition software and drug discovery, neural networks enable AI systems to complete tasks that require them to make decisions based on common threads in data they’ve seen. But before Frankle’s work, engineers believed we needed to build massive neural networks to ensure performance, then shrink them down post-training.
Frankle found that within large neural networks were smaller, more efficient ones, potentially minimizing computational costs. His dissertation, aptly titled “The lottery ticket hypothesis,” revealed that you could keep networks small for much or all of the training process — thanks to particularly efficient subnetworks he calls “winning tickets.” These components performed comparably to larger networks despite being 10-20% of their size. Frankle’s paper would go on to receive a Best Paper Award at ICLR 2019.
During his CSAIL days, Frankle worked with MIT professor and CSAIL principal investigator Michael Carbin. In 2021, they co-founded MosaicML as a CSAIL spinout to make it possible for everyone to build advanced AI models by making the training process more efficient. Three years later, their company was acquired for $1.3 billion from Databricks, where Frankle now works as Chief AI Scientist.
He continues to study how contemporary neural networks learn in practice and to strive to enable everyone to build with modern AI. “I think we can build better and better AI systems — and do so more efficiently — if we understand how neural networks learn,” says Frankle. “And I think the best way to do so is to study the real artifacts that have made us so excited about AI in the first place.”
