Don’t Panic! Empowering AI Research At Scale Beyond Big Names and Big Budgets

Abstract: The recent success of large, expensive pretrained models, especially in natural language processing and computer vision, has lead many to believe that cutting edge modeling work is no longer something that academic and non-profit researchers can concern themselves with. Some even go further and assert that researchers should accept that their job is now to study the interesting models that large corporate actors create.

This narrative, despite being common, is deeply misguided. In this talk I will discuss how "GPU poor" actors continue to be essential to the development of large scale AI models, how institutions can get access to the compute required to train large models, and why there's lots of interesting work to do in these fields for people who don't want to train their own large models.

Bio: Stella Biderman is the executive director of EleutherAI, a non-profit research institute specializing in large language models. She builds public infrastructure for training, finetuning, and evaluating large scale AI systems, has trained four models to hold the title "the largest open source language model [at the time] in the world," and studies how we can design technology to be more inclusive, less concentrated, and less harmful.