Title: Compiler 2.0: Building the Next Generation Compilers with Machine Learning

Abstract: The genius of FORTRAN, introduced in 1957, was not just that it was the first high-level language, but that it made hardware disappear. The same FORTRAN program could run on an IBM System/360, a DEC PDP-8, a CDC 6600, or even a Cray-1 with enough performance that most programmers never had to touch assembly or write architecture-specific code.

Over time, as hardware and software grew more complex, compilers quietly lost the ability to maintain this abstraction. Multicores, complex vector instructions, and specialized accelerators have all pushed more of the performance burden back onto the programmer. Today we have, in many ways, come a full circle. For example, getting peak performance from NVIDIA’s V100, A100, H100, and B100 families often requires different, architecture-specific CUDA kernels and, on the latest parts, even hand-written PTX assembly to fully exploit tensor cores and the Tensor Memory Accelerator. To the best of our knowledge, the Apple/Arm SME accelerator does not even have a high-level compiler, and most high-performance code must be written using intrinsics or assembly.

This raises a fundamental question: can the next generation of compilers restore the original promise of the high-level programming languages—hiding architectural complexity while still delivering near-peak performance? 

In this talk, I will argue that the answer is yes, and I will outline a path forward for us to modernize compilers—both to make them far more effective on today’s architectures and to radically simplify how we build them—by creating proper abstractions and leveraging machine learning and other modern techniques.
 

Bio: Prof. Saman Amarasinghe is the Thomas and Gerd Perkins Professor of EECS, MIT and leads the Commit compiler research group in MIT’s Computer Science & Artificial Intelligence Laboratory (CSAIL), which focuses on programming languages and compilers that maximize application performance on modern computing platforms. He is a world leader in the field of high-performance domain-specific languages. Prof. Amarasinghe's group developed the Halide, TACO, Simit, StreamIt, StreamJIT, PetaBricks, MILK, Cimple, and GraphIt domain-specific languages and compilers, all of which combine language design and sophisticated compilation techniques to deliver unprecedented performance for targeted application domains such as image processing, stream computations, and graph analytics. Dr. Amarasinghe also pioneered the application of machine learning for compiler optimizations, from Meta optimization in 2003 to OpenTuner extendable autotuner today. With professor Anant Agarwal, he co-led the Raw architecture project, which did pioneering work on scalable multicores.

Prof. Amarasinghe's entrepreneurship activities include founding Determina, Inc. (acquired by VMWare) based on computer security research pioneered in his research group at MIT and co-founding Lanka Internet Services, Ltd., the first Internet Service Provider in Sri Lanka. Prof. Amarasinghe is also the faculty director of MIT Global Startup Labs, whose summer programs in 17 countries have helped to create more than 20 thriving startups.

Prof. Amarasinghe developed the popular Performance Engineering of Software Systems (6.172) class with Professor Charles Leiserson. He also created individualized software project classes such as the Open Source Software Project Lab, the Open Source Entrepreneurship Lab, and the Bring Your Own Software Project Lab.

Prof. Amarasinghe received a bachelor’s degree in electrical engineering and computer science from Cornell University in 1988, a master’s degree and PhD in electrical engineering from Stanford University in 1990 and 1997 respectively, and joined the MIT faculty as an assistant professor in 1997. He was elected as an ACM fellow in 2019.