MIT CSAIL PhD students William Moses and Valentin Churavy are among a team of researchers that have won Best Student Paper at SuperComputing 2022, receiving the honor alongside colleagues from Argonne National Lab and Technical University of Munich. The team received this award for their work on Enzyme, a framework that can differentiate numerous parallel models and directly control code generation.
The annual conference focuses on high-performance computing research, welcoming over 11,000 attendees while featuring more than 350 exhibit booths. Hosted in Dallas, organizers aimed to build a “deeper understanding” of the world’s latest computing trends by showcasing emerging research in the field.
Moses and Churavy’s team stood out among the groundbreaking research showcased at the conference, winning Best Student Paper while also being a Best Paper Finalist. Their work, “Scalable Automatic Differentiation of Multiple Parallel Paradigms through Compiler Augmentation," demonstrates how Enzyme optimizes hardware capacity, utilizing automatic differentiation to efficiently compute derivatives in various parallel programming models.
Commonly referred to as “Autodiff” or “AD,” automatic differentiation can improve machine learning models by accurately evaluating derivatives of numeric functions implemented in computer programs. The process is also the core algorithm used within neural network training, uncertainty quantification, and more broadly in machine learning and scientific computing.
Enzyme also has a broad range of applications, including public health analysis and climate simulation. The framework is also the first automatic differentiation tool that can handle multiple models and languages with a single implementation. Enzyme can operate as a compiler plug-in, importing foreign code into TensorFlow and other machine learning platforms to save researchers the extra step of having to rewrite code. Additionally, the studies indicated that the tool can run optimizations before differentiation, allowing it to execute programs up to four times faster than existing tools.
Moses was supported in part by his Department of Energy (DOE) Computational Sciences Graduate Fellowship, while Churavy receives funding from NSF and DARPA. The DOE recently awarded the team with a grant for their work on Enzyme as well. This material is based upon work supported by the DOE, while the research was supported in part by Los Alamos National Laboratories and in part by NSF Cyberinfrastructure for Sustained Scientific Innovation (CSSI).
This project was made possible by Eric and Wendy Schmidt by recommendation of the Schmidt Futures program, as well as by the Paul G. Allen Family Foundation, Charles Trimble, and the Audi Environmental Foundation. The United States Air Force Research Laboratory and the United States Air Force Artificial Intelligence Accelerator sponsored the research, though the views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the United States Air Force or the U.S. Government.
