A Single Idea in Compiler Goes a Long Way in ML - Generalized Redundancy Removal for Machine Learning
In a recent series of research, Dr. Shen and his colleagues have shown that generalization of a single idea in compiler, redundancy removal, can yield a whole set of novel techniques that significantly improve the speed and computing efficiency of modern machine learning. This talk will explain the idea behind generalized redundancy removal, the set of techniques it has led to, and how these techniques help halve DNN training and inference times, enable power-efficient concurrent training of thousands of DNN models, and improve the speed in finding a well pruned DNN by a factor of 173 with no quality loss nor addition of extra computing resource. The talk will in addition describe Egeria, a framework for automatic synthesis of HPC advising tools through multi-Layered Natural Language Processing. (The talk is based on Dr. Shen's recent publications at PLDI'2019, ICDE'2019, SC'2018, SC'2017)
Xipeng Shen is a Professor in the Computer Science Department at North Carolina State University. His research has received a number of recognitions, including Early Career Research Award from the US Department of Energy, CAREER Award from US NSF, Google Faculty Research Award, IBM Center for Advanced Studies Faculty Fellow Award, ACM Distinguished Member, ACM Distinguished Speaker, and so on. His primary research work lies in the field of compiler and programming systems, but features an emphasis on inter-disciplinary problems and approaches. His research has produced influential results in multicore memory performance enhancements, GPU program compilations, high-level code optimizations, and other topics in programming systems. A number of the results have been incorporated into commercial compilers (e.g., IBM XL compilers) and other products. Meanwhile, his research has led to a number of progresses in machine learning and artificial intelligence, exemplified with a set of new machine learning algorithms (e.g., Yinyang K-Means, Multi-label Scene Classification) published at major ML or AI venues and adopted by Microsoft and other industry companies. He has chaired ASPLOS, PPOPP, and other major conferences, and served on the technical advisory boards of some leading IT companies. Prior to joining NC State in 2014 as a Chancellor's Faculty Excellence Program cluster hire in Data-Driven Science, Shen was the Adina Allen Term Distinguished Associate Professor in the Computer Science Department at The College of William and Mary. He spent his sabbatical at MIT, Microsoft Research, and Intel Labs between 2012 and 2013. He was an assistant professor at The College of William and Mary from 2006 to 2012. He received his Ph.D. in Computer Science from University of Rochester in 2006. In 2018, he was honored with University Faculty Scholars Award as an emerging academic leader who turns research into solutions to society’s most pressing issues.