Trillion Parameter AI Serving Infrastructure for Scientific Discovery: A Survey and Vision

Deep learning methods are transforming research, enabling new techniques, andultimately leading to new discoveries. As the demand for more capable AI modelscontinues to grow, we are now entering an era of Trillion Parameter Models(TPM), or models with more than a trillion parameters – such as Huawei'sPanGu-Σ. We describe a vision for the ecosystem of TPM users andproviders that caters to the specific needs of the scientific community. Wethen outline the significant technical challenges and open problems in systemdesign for serving TPMs to enable scientific research and discovery.Specifically, we describe the requirements of a comprehensive software stackand interfaces to support the diverse and flexible requirements of researchers.