Machine learning frameworks rely on vendor libraries or autotuning frameworks for high-performance implementations of key operators like matrix multiplication and convolution. The transform dialect has recently been developed in the MLIR framework to facilitate the composition of transformations to implement optimized schedules for tensor computations. However, its interface to users is complex and requires an understanding of the underlying compiler transformations. In this paper, we describe a higher level scheduling language PEAK that is built on top of the MLIR transform dialect to ease the process of developing optimized schedules for deep learning operators on GPUs. PEAK expresses a simplified scheduling language in the MLIR/IREE compiler; it exploits domain-specific properties about data reuse in tensor contractions to determine thread mapping strategies and stage data through the GPU memory hierarchy. PEAK integrates an autotuner to explore implementations of high-performance code with schedules based on the MLIR Transform Dialect. PEAK enables a significant reduction of effort to construct high-performance GPU code using MLIR. A comparison with the state-of-the-art TVM/Ansor autotuning compiler framework shows higher performance for matrix-vector products, nearly comparable performance for matrix-matrix multiplication, but lower performance for convolutions. The paper presents insights into the limitations of the MLIR/IREE infrastructure that currently impact the performance achievable with PEAK.
General-purpose memory allocators are made to perform well on average for any given program. They thus make decisions which can benefit a broad set of applications and can miss out on possible optimizations. When a given general-purpose allocator does not fit the needs of a program, the developer has a choice of either switching to a different allocator or writing a custom one from scratch. Both options can be quite costly, and can still fail to satisfy the developer's requirements. We propose a different approach to memory allocation: allocators are automatically generated from the ground up for any given program and optimized for the needed metric. We outline metrics of allocator performance, present a taxonomy of single-threaded memory allocators, and a framework for generating custom allocators based on the taxonomy. We show that allocators generated in such way can match or outperform general-purpose allocators and that different applications benefit from different components of our taxonomy.
The tiling transformation is one of the most crucial code optimization techniques to expose data locality and parallelism. The main idea is to split the initial iteration space into blocks and traverse them in a special order. This transformation is parametric and very sensitive to parameter tuning. Poor parameter tuning can lead to much lower performance than the initial code. Existing state-of-the-art solutions consider a restricted list of parameters to handle this issue and guarantee safe solutions. Our work proposes solutions that go beyond current state-of-the-art techniques and gain additional speedup considering a larger set of options for tiling. Our approach is based on Machine Learning methods and automatically derives heuristics to tune tiling parameters. We can predict: 1) the optimal partitioning matrix of the iteration space 2) the tile sizes, 3) the optimal directions for scanning inter-tiles, 4) the optimal directions for scanning intra-tile elements. The optimal selection of these parameters is crucial especially for programs that have data dependencies. We introduce sets of features that feed our models in their predictions. The first set encodes data dependencies, the second one captures the level of parallelism and data locality in a code. The third one aggregates information about the iteration space. Our approach surpasses existing feature spaces for tiling parameters prediction. Moreover, it could be used in conjunction with auto-tuners to iterate through the iterative search.
Heterogeneous and multi-device nodes are widely used in high-performance computing and data centers. However, current programming models do not provide simple, transparent, and portable support for automatically targeting heterogeneous nodes. In this paper, we present SEER, a new C++ library that provides a descriptive programming model to enable applications to benefit from heterogeneous nodes in a transparent and portable way across multiple device types. SEER provides efficient memory management and can select the proper device[s] depending on the computational cost of the applications. All this is completely transparent to the programmer, thereby providing a highly productive programming environment. We evaluate the SEER library on two heterogeneous nodes of Summit (#5 TOP500) and Crusher supercomputers. Notably, the smaller-scale Crusher test-bed machine uses identical hardware and software as ORNL’s Frontier (#1 TOP500). This work also includes a detailed performance study conducted with a set of representative test cases in high-performance computing (e.g., Basic Linear Algebra Subprograms (BLAS), Tridiagonal Solve, and Conjugate Gradient). SEER provides high accelerations of up to 30× for sparse matrix and 8× for batch BLAS applications thanks to automatic and transparent device selection and multi-device exploitation respectively.
Answer-set programming (ASP) is a declarative approach to solving combinatorial problems. It lends itself easily to parallel execution. Several ASP solvers are available; we show sample programs in several dialects. The samples include Sudoku puzzles, logic puzzles, Costas arrays, tiling, and Room arrangements. This paper is based on an interactive demonstration presented at the 36th International Workshop on Languages and Compilers for Parallel Computing, Lexington, KY, October 2023.
