Machine learning frameworks rely on vendor libraries or autotuning frameworks for high-performance implementations of key operators like matrix multiplication and convolution. The transform dialect has recently been developed in the MLIR framework to facilitate the composition of transformations to implement optimized schedules for tensor computations. However, its interface to users is complex and requires an understanding of the underlying compiler transformations. In this paper, we describe a higher level scheduling language PEAK that is built on top of the MLIR transform dialect to ease the process of developing optimized schedules for deep learning operators on GPUs. PEAK expresses a simplified scheduling language in the MLIR/IREE compiler; it exploits domain-specific properties about data reuse in tensor contractions to determine thread mapping strategies and stage data through the GPU memory hierarchy. PEAK integrates an autotuner to explore implementations of high-performance code with schedules based on the MLIR Transform Dialect. PEAK enables a significant reduction of effort to construct high-performance GPU code using MLIR. A comparison with the state-of-the-art TVM/Ansor autotuning compiler framework shows higher performance for matrix-vector products, nearly comparable performance for matrix-matrix multiplication, but lower performance for convolutions. The paper presents insights into the limitations of the MLIR/IREE infrastructure that currently impact the performance achievable with PEAK.
General-purpose memory allocators are made to perform well on average for any given program. They thus make decisions which can benefit a broad set of applications and can miss out on possible optimizations. When a given general-purpose allocator does not fit the needs of a program, the developer has a choice of either switching to a different allocator or writing a custom one from scratch. Both options can be quite costly, and can still fail to satisfy the developer's requirements. We propose a different approach to memory allocation: allocators are automatically generated from the ground up for any given program and optimized for the needed metric. We outline metrics of allocator performance, present a taxonomy of single-threaded memory allocators, and a framework for generating custom allocators based on the taxonomy. We show that allocators generated in such way can match or outperform general-purpose allocators and that different applications benefit from different components of our taxonomy.
The tiling transformation is one of the most crucial code optimization techniques to expose data locality and parallelism. The main idea is to split the initial iteration space into blocks and traverse them in a special order. This transformation is parametric and very sensitive to parameter tuning. Poor parameter tuning can lead to much lower performance than the initial code. Existing state-of-the-art solutions consider a restricted list of parameters to handle this issue and guarantee safe solutions. Our work proposes solutions that go beyond current state-of-the-art techniques and gain additional speedup considering a larger set of options for tiling. Our approach is based on Machine Learning methods and automatically derives heuristics to tune tiling parameters. We can predict: 1) the optimal partitioning matrix of the iteration space 2) the tile sizes, 3) the optimal directions for scanning inter-tiles, 4) the optimal directions for scanning intra-tile elements. The optimal selection of these parameters is crucial especially for programs that have data dependencies. We introduce sets of features that feed our models in their predictions. The first set encodes data dependencies, the second one captures the level of parallelism and data locality in a code. The third one aggregates information about the iteration space. Our approach surpasses existing feature spaces for tiling parameters prediction. Moreover, it could be used in conjunction with auto-tuners to iterate through the iterative search.
Heterogeneous and multi-device nodes are widely used in high-performance computing and data centers. However, current programming models do not provide simple, transparent, and portable support for automatically targeting heterogeneous nodes. In this paper, we present SEER, a new C++ library that provides a descriptive programming model to enable applications to benefit from heterogeneous nodes in a transparent and portable way across multiple device types. SEER provides efficient memory management and can select the proper device[s] depending on the computational cost of the applications. All this is completely transparent to the programmer, thereby providing a highly productive programming environment. We evaluate the SEER library on two heterogeneous nodes of Summit (#5 TOP500) and Crusher supercomputers. Notably, the smaller-scale Crusher test-bed machine uses identical hardware and software as ORNL’s Frontier (#1 TOP500). This work also includes a detailed performance study conducted with a set of representative test cases in high-performance computing (e.g., Basic Linear Algebra Subprograms (BLAS), Tridiagonal Solve, and Conjugate Gradient). SEER provides high accelerations of up to 30× for sparse matrix and 8× for batch BLAS applications thanks to automatic and transparent device selection and multi-device exploitation respectively.
Answer-set programming (ASP) is a declarative approach to solving combinatorial problems. It lends itself easily to parallel execution. Several ASP solvers are available; we show sample programs in several dialects. The samples include Sudoku puzzles, logic puzzles, Costas arrays, tiling, and Room arrangements. This paper is based on an interactive demonstration presented at the 36th International Workshop on Languages and Compilers for Parallel Computing, Lexington, KY, October 2023.
Motivated by the challenges of programming irregular applications for machines with million-fold parallelism, we present a key-based programming model, called key-value map-shuffle-reduce (KVMSR), that enables programmers to optimize fine-grained parallel programs. KVMSR expresses parallelism on a global address space and features modular interfaces to flexibly bind computation to available compute resources. We define the KVMSR model and illustrate it with three programs, convolution filter, PageRank and BFS, to show its ability to separate computation expression from binding to computation location for high performance. On a 8,192-way parallel compute system, KVMSR modular computation location control achieves up to 2,317 × performance with static approaches and an increase of 549 × to 2,715 × speedup with dynamic approaches for computation location binding.
Quantum systems with limited physical qubits cannot execute quantum circuits with more logical qubits than physically-available ones, leading to compile-time errors. As it is unrealistic to expect quantum systems to provide sufficient qubits in the near future, there is a pressing need to explore strategies to execute large circuits on small systems, as current systems are comparatively small in comparison to the needs of the existing and emerging quantum algorithms/circuits. In this work, we analyze quantum programs to identify qubits that can be reused mid-program to execute the circuit with fewer qubits; this process is termed as resizing or serialization. Based on our analysis, we then propose a compiler-driven approach that selects the most beneficial qubits for circuit resizing, and provide proof of work for the algorithm. The results with our proposed circuit resizing indicate that it can i) execute large circuits that cannot originally fit into small number of physical qubits in current quantum systems, ii) significantly improve PST (Probability of Successful Trial) by 2.1X, and iii) and 53
This paper describes COMPASS – a suite of irregular benchmarks comprising special patterns referred to as subscripted subscript patterns. When the values of an array appear at the subscript of another array in a for-loop, e.g. a[b[i]] with cross-iteration accesses to the host array (array a), such a pattern is referred to as a subscripted subscript pattern. These patterns represent an important class of dynamic, irregular memory access patterns observed in scientific applications and pose a challenge for optimizing compilers. The suite is a collection of subscripted subscript benchmarks from various application domains such as Machine Learning, Linear System Solvers, Adaptive Mesh Refinement, Sorting Algorithms and Sparse Matrix computations. The primary purpose of this suite is to promote the development of advanced compiler analysis and transformation techniques that enable parallelization of the subscripted subscript loops as well as techniques for improving locality and thread synchronization. We present the necessary and sufficient conditions for eventual parallelization of the subscripted subscript loops and discuss techniques described in the literature for determining said conditions. Experimental results show that subscripted subscript loops appear in key program sections and parallelizing them leads to a substantial improvement in the performance of the overall applications.
Production compilers have achieved a high level of maturity in terms of generating efficient code. The code generated by any two production compilers can turn out to be very different based on pros and cons of their respective Intermediate Representation (IR), implemented loop transformations and their ordering, cost models used and even instruction selection (such as vector instructions) and scheduling. Hence, the performance of produced code for a program segment by a given compiler may not necessarily be matched by other compilers. This paper proposes a meta-compilation framework, the MCompiler, which allows different segments of a program to be compiled with different compilers/optimizers and combined into a single executable. It turns the differences between compilation processes and performance optimizations in each compiler from a weakness to a strength. Utilizing the highest performing code for each segment can lead to a significant overall improvement in performance. A loop nest is used as a segment in this work, but other choices can be made. The question is, though, which compiler will produce the best code for a segment. This work then presents a technique to accomplish this using Machine Learning. It learns inherent characteristics of loop nests and then predicts during compilation which code optimizer is the most suited for each loop nest in an application. The results show that our framework improves the overall performance for applications over state-of-the-art compilers (compiled at equivalent of-O3) by a geometric mean of 1.97x for auto-vectorized code and 2.62x for auto-parallelized code. Parallel applications with OpenMP directives are also improved by the MCompiler, with a geometric mean performance improvement of 1.13x. The use of Machine Learning prediction achieves performance very close to the exploratory search for choosing the most suited code optimizer: within 4
In the realm of parallel computing, optimization plays a pivotal role in achieving efficient and scalable solutions. In this work, we present the parallelization of a hybrid genetic search for solving the Capacitated Vehicle Routing Problem with Pickup and Delivery (CVRPPD). The hybrid algorithm combines a customized version of local search with a genetic algorithm to compute an effective solution. Our implementation makes use of the Message Passing Interface (MPI) for data distribution and parallel execution. In addition, we run multi-threaded processes on NVIDIA graphical processors using the CUDA technology, which further increases the computation speed and consequently minimizes the runtime. Parallelization also allows the best improvement strategy to be used instead of the first-improvement strategy while maintaining the same runtime. We store the resulting routes in a bus route database which we created as the basis of an extensive library of optimal routes for our specific use case of optimizing bus routes in a rural area. The experimental results on real road data show that the parallel implementation of the Hybrid Genetic Search (HGS) achieves significant improvements in runtime over the sequential implementation above a certain problem size. We believe that our implementation of the parallel hybrid genetic search method can have a great influence on optimization strategies in parallel computing and can also be applied to other subproblems of the VRP.
This paper presents an optimized code generation for arbitrary out-of-place tensor transpositions using the MLIR compiler infrastructure, portable across CPU architectures. The proposed modular and reusable approach encodes optimizations such as multi-level tiling and explicit vectorization at multiple levels of abstraction as a sequence of transformation and conversion passes in MLIR. The efficient code generated is evaluated on AMD, Intel, and ARM processors and achieves performance comparable to the state-of-the-art HPTT library [27], a compelling speedup over Eigen [10], and a significant fraction of the STREAM memory bandwidth on these platforms. We further integrate this progressive lowering pipeline into COMET, an MLIR-based compiler for tensor contractions, and obtain an average speedup of 26% with the TTGT approach.
Deep Neural Networks (DNNs) are indispensable for AI robots and autonomous driving vehicles. Low power and high performance of DNN processors are critical to realize long battery life and long-term reliable controller operations, keeping their actions flexible. In addition, high program productivity is also essential for their cost-effective product development. We have been developing the OSCAR parallelizing and power-reducing compiler and its co-designed homogeneous and heterogeneous multicore processor chips. In this paper, the TVM, an open-source deep learning compiler, is utilized with the OSCAR compiler to automatically parallelize various DNN inference models. However, the current TVM does not generate a C program compatible with vectorization. In this paper, we propose a code generation method for TVM to generate vectorization-friendly code by transforming the memory layout of tensors to keep a long vector length at an innermost vectorized loop. The parallelized coarse grain task parallelization program is translated into NEC machine code with vector instructions by the NEC compiler. The execution performance of the proposed method with pre-trained DNN inference models is evaluated on NEC SX-Aurora TSUBASA vector multicore. The evaluation result shows that the proposed method achieves 31.3× speedup on seven cores with a ResNet model and 37.6× speedup with a VGG model, compared with the compilation flow that does not include the proposed method.
SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this paper, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, use of local memory, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.
We evaluate the use of the open-source Llama-2 model for generating well-known, high-performance computing kernels (e.g., AXPY, GEMV, GEMM) on different parallel programming models and languages (e.g., C++: OpenMP, OpenMP Offload, OpenACC, CUDA, HIP; Fortran: OpenMP, OpenMP Offload, OpenACC; Python: numpy, Numba, pyCUDA, cuPy; and Julia: Threads, CUDA.jl, AMDGPU.jl). We built upon our previous work that is based on the OpenAI Codex, which is a descendant of GPT-3, to generate similar kernels with simple prompts via GitHub Copilot. Our goal is to compare the accuracy of Llama-2 and our original GPT-3 baseline by using a similar metric. Llama-2 has a simplified model that shows competitive or even superior accuracy. We also report on the differences between these foundational large language models as generative AI continues to redefine human-computer interactions. Overall, Copilot generates codes that are more reliable but less optimized, whereas codes generated by Llama-2 are less reliable but more optimized when correct.
The performance of machine learning systems heavily relies on code generators tailored to tensor computations. We propose an approach to the design and implementation of such code generators leveraging the natural structure of tensor algebra and illustrating the progressive lowering of domain-specific abstractions in the MLIR infrastructure.
By comparing automatically versus manually parallelized NAS Benchmarks, we identify code sections that differ, and we discuss opportunities for advancing auto-parallelizers. We find ten patterns that challenge current parallelization technology. We also measure the potential impact of advanced techniques that could perform the needed transformations automatically. While some of our findings are not surprising and difficult to attain – compilers need to get better at identifying parallelism in outermost loops and in loops containing function calls – other opportunities are within reach and can make a difference. They include combining loops into parallel regions, avoiding load imbalance, and improving reduction parallelization. Advancing compilers through the study of hand-optimized code is a necessary path to move the forefront of compiler research. Very few recent papers have pursued this goal, however. The present work tries to fill this void.
In most programming languages, data is organized in structures that are explicitly mapped to machine words each containing a fixed number of bits. For example, a C variable declared as an int might be specified to be represented by a 32-bit word. Given computer hardware in which data are organized as fixed-size words, this seems intuitive and efficient. However, if the integer is known to always have a value between 0 and 100, at most only seven of those 32 bits are needed; the other 25 bits are always 0. Programming languages like C allow integer variables to be declared as having any of several bit precisions, so declaring the variable as uint8_t could reduce waste to just one bit. The catch is the index is only seven bits long when it holds a value greater than 63. Operating on more bits than necessary dramatically increases both the storage space and the number of gate operations needed to perform operations like addition or multiplication. The solution proposed here is to implement a programming model in which integer and floating-point variables are represented by just enough bits to represent the values they contain at that moment in execution. The overhead involved in dynamically adjusting precision is significant, thus it is only used for SIMD-parallel variables implemented using the PBP execution model.
Data races in distributed parallel programs, such as those developed with the message passing interface (MPI), can cause critical correctness and reliability issues. Therefore, it is highly necessary to detect and fix them. However, existing MPI programming error detection tools have rather limited support for data race detection. To address this problem, we present MPIRace, which is a static data race detector for MPI programs. It creates several novel and effective static program analysis techniques to overcome the technical challenges of conducting static data race detection for MPI programs. We also implement a research prototype of MPIRace based on LLVM, a widely-used compiler infrastructure. After applying MPIRace to MPI-CorrBench, a recent MPI correctness benchmark suite, and a broad range of real-world MPI applications, we successfully find 20 data races. Among them, 12 are found in the real-world MPI applications and it is the first time they are reported by a data race detector. Moreover, the detection speed of MPIRace is extremely fast, i.e., less than one minute for every evaluated application. We believe MPIRace will tremendously help developers in improving the correctness, reliability, and sustainability of MPI programs.
Power ISA™ Version 3.1 has introduced a new family of matrix math assist instructions, collectively known as the Matrix-Multiply Assist (MMA) facility. The instructions in this facility implement numerical linear algebra operations on small matrices and are meant to accelerate computation-intensive kernels. We advocate the use of compiler built-ins as the preferred way of leveraging these instructions. MMA built-ins are currently available in the GNU Compiler Collection and the LLVM-based IBM Open XL compilers. The built-ins are compatible across both compiler suites. We show that programming with these built-ins leads to efficient code that fully exploits the new facility.
HPC practitioners make use of techniques, such as parallelism and sparse data structures, that are difficult to reason about and debug. Here we explore the role of data refinement, a correct-by-construction approach, in verifying HPC applications via bounded model checking. We show how single program, multiple data (SPMD) parallelism can be modeled in Alloy, a declarative specification language, and describe common issues that arise when performing scope-complete refinement checks in this context.
