Designing, Implementing, and Evaluating the Upcoming OpenSHMEM Teams API

For many years, the OpenSHMEM parallel pro- gramming interface has provided a high-performance alternative to MPI that emphasizes one-sided messaging, simplifies com- munication across a global memory space, and bolsters the capabilities of rapidly evolving fabric interconnect technolo- gies. The OpenSHMEM specification standardizes the library interfaces, prioritizing a performant and portable API. The specification continues to mature with vigorous support from several authoritative vendors and researchers. For example, the OpenSHMEM specification committee is actively standardizing a unique teams API that enables user-defined subsets of application processes to efficiently and productively perform communication operations, such as collectives routines, remote memory accesses, and remote atomic operations. This paper describes the OpenSHMEM teams interface and several interesting aspects and challenges in implementing the API, as well as possible extensions that could improve the pro- grammability and/or performance. We evaluate the performance of a preliminary implementation and show that using teams effectively can facilitate impressive improvements of collective operations at scale (2-16x speedup, depending on the algorithm, node count, and buffer size), even while simplifying the under- lying programming model.