Efficient Auto-Tuning with Kernel Tuner

From climate simulation to artificial intelligence, modern science and technology increasingly rely on powerful computers, especially GPUs. These machines are extremely fast, but getting the most out of them is surprisingly difficult. Small choices in how a program is written, such as how data is stored or how computation is organized, can make performance vary wildly. Because the number of possible choices is enormous, automatic performance tuning (auto-tuning) is needed: letting the computer automatically test different program configurations until it finds the best. Auto-tuning frameworks like Kernel Tuner make this possible in practice. However, auto-tuning and the expanding complexity bring with it a host of new challenges. In this talk, I will discuss several research directions to address these challenges, including specialized optimization algorithms like Bayesian Optimization, fast resolution of massive search spaces, and whether LLMs can design bespoke optimization algorithms that outperform traditional methods.