In the era of LLMs, dense operations such as GEMM and MHA are critical components. These operations are well-suited for parallel execution using a tilebased approach. While traditional GPU programming often relies on low level interfaces like CUDA or SYCL, Triton has emerged as a DSL that offers a more user-friendly and portable alternative by programming at a higher level. The current Triton starts at the workgroup (aka threadblock) level, and directly lowers to per-thread level. And then attempt to coalesce and amend through a series of passes, promoting information from low-level representation. We believe this is pre-mature lowering based on the below observations. 1. GPU has a hierarchical structure both physically and logically. Modern GPUs often feature SIMD units capable of directly operating on tiles on a warp or warpgroup basis, such as blocked load and blocked MMA. 2. Multi-level gradual lowering can make compiler decoupled and clean by separating considerations inter and intra a logical layer. 3. Kernel developers often need fine control to get good performance on the latest hardware. FlashAttention2 advocates explicit data partition between warps to make a performance boost. In this context, we propose ML-Triton which features multi-level compilation flow and programming interface. Our approach begins at the workgroup level and progressively lowers to the warp and intrinsic level, implementing a multilevel lowering align with the hierarchical nature of GPU. Additionally, we extend triton language to support user-set compiler hint and warp level programming, enabling researchers to get good out-of-the box performance without awaiting compiler updates. Experimental results demonstrate that our approach achieves performance above 95 the geometric mean.

Similar to other programming models, compilers for SYCL, the open programming model for heterogeneous computing based on C++, would benefit from access to higher-level intermediate representations. The loss of high-level structure and semantics caused by premature lowering to low-level intermediate representations and the inability to reason about host and device code simultaneously present major challenges for SYCL compilers. The MLIR compiler framework, through its dialect mechanism, allows to model domain-specific, high-level intermediate representations and provides the necessary facilities to address these challenges. This work therefore describes practical experience with the design and implementation of an MLIR-based SYCL compiler. By modeling key elements of the SYCL programming model in host and device code in the MLIR dialect framework, the presented approach enables the implementation of powerful device code optimizations as well as analyses across host and device code. Compared to two LLVM-based SYCL implementations, this yields speedups of up to 4.3x on a collection of SYCL benchmark applications. Finally, this work also discusses challenges encountered in the design and implementation and how these could be addressed in the future.

Recent years have raised awareness of the fact that many optimizing C++ compilers, such as Clang/LLVM, miss optimization opportunities due to the lack of a suitable high-level intermediate representation. The typical compilation flow of such a compiler would lower from a representation close to the original or pre-processed source code, e.g., an abstract syntax tree (AST), directly to a low-level, CFG- and SSA-based intermediate representation such as LLVM IR. However, this lowering loses much of the high-level information and structure of the original source code as it cannot always be represented accurately in the low-level intermediate representation. Compiler optimization passes working on this low-level IR try to recover relevant parts of the high-level information (e.g., loops) and programmer’s intent to optimize the code. If they fail to recover the necessary information, important optimization opportunities might be missed. This insight about loss of high-level information in compilers has driven the creation of the MLIR framework. The MLIR framework, through its core abstraction called “dialect”, enables the creation of a set of multiple intermediate representations capturing high-level semantics and domain-specific information. The progressive lowering process from source code to executable then happens in much smaller steps and allows optimization passes to operate at the appropriate level of abstraction to leverage high-level information. With SYCL being strongly based on C++, SYCL compilers likewise suffer from the same problem as current C++ compilers, which naturally raises the question of whether the MLIR framework can be used to improve SYCL compilation. Our poster will present important insights from our ongoing investigation into this question. It will present an overview of an architecture for an MLIR-based SYCL compiler, demonstrating how MLIR can be integrated into the typical compilation flow for SYCL applications and how the resulting compilation output interacts with existing SYCL runtime implementations. The poster will also report on the status of an ongoing collaboration project between Codeplay and Intel, developing an MLIR-based SYCL compiler as an open-source project based on Intel’s existing DPC++ SYCL compiler and runtime implementation. At the time of writing, the MLIR-based device compiler can already compile a substantial portion of the SYCL application tests in Intel’s fork of the LLVM test-suite, and we seek to further improve coverage and extend the compilation to the host-part of SYCL applications. The design principles and core abstractions of the MLIR dialect for SYCL, developed as part of the project, will be discussed in detail, demonstrating how MLIR enables compiler optimization passes to better understand the semantics of SYCL applications. The poster will outline several opportunities for MLIR to significantly improve the code generated for SYCL applications over existing, LLVM-based compilation flows. One instance of such a potential improvement is MLIR’s ability to represent host and device code together in its nested intermediate representation. The ability to access host and device code at the same time from compiler optimization passes will enable transformations across the border between host and device code for SYCL applications. An example for such a transformation is the propagation of constants from host code into device code, an ability not found in DPC++’s current LLVM-based compilation flow. This may also pave a new avenue to single-pass compilation of SYCL applications in the future. Overall, the poster will provide a clear vision of how MLIR in the future will allow SYCL programs to be better optimized during compilation and the necessary steps to implement such a compilation flow.

Ability to efficiently offload computational workloads to graphic processing units (GPUs) is critical for the success of hybrid CPU-GPU architectures, such as the Summit and Sierra supercomputing systems. OpenMP 4.5 is a high-level programming model that enables the development of architecture- and accelerator-independent applications. This article describes aspects of the OpenMP implementation in the IBM XL C/C++ and XL Fortran OpenMP compilers that aid programmers to achieve performance objectives. This includes an interprocedural static analysis the XL optimizer uses to specialize code generation of the OpenMP distribute parallel do loop within the dynamic context of a target region, and other compiler optimizations designed to reduce the overhead of data transferred to an offloaded target region. We introduce the heuristic used at runtime to select optimal grid sizes for offloaded target team constructs. These tuned heuristics lead to an average improvement of 2x in the runtime of several target regions in the SPEC ACCEL V1.2 benchmark suite. In addition to performance enhancement, this article also presents an advanced diagnostic feature implemented in the XL Fortran compiler to aid in debugging OpenMP applications ojjloaded to accelerators.

Iteration Point Difference Analysis is a new static analysis framework that can be used to determine the memory coalescing characteristics of parallel loops that target GPU offloading and to ascertain safety and profitability of loop transformations with the goal of improving their memory access characteristics. This analysis can propagate definitions through control flow, works for non-affine expressions, and is capable of analyzing expressions that reference conditionally defined values. This analysis framework enables safe and profitable loop transformations. Experimental results demonstrate potential for dramatic performance improvements. GPU kernel execution time across the Polybench suite is improved by up to 25.5× on an Nvidia P100 with benchmark overall improvement of up to 3.2×. An opportunity detected in a SPEC ACCEL benchmark yields kernel speedup of 86.5× with a benchmark improvement of 3.3×. This work also demonstrates how architecture-aware compilers improve code portability and reduce programmer effort.

Automating the device selection in heterogeneous computing platforms requires the modelling of performance both on CPUs and on accelerators. This work argues for the use of a hybrid analytical performance modelling approach is a practical way to build fast and efficient methods to select an appropriate target for a given computation kernel. The target selection problem has been addressed in the literature, however there has been a strong emphasis on building empirical models with machine learning techniques. We argue that the applicability of such solutions is often limited in production systems. This paper focus on the issue of building a selector to decide if an OpenMP loop nest should be executed in a CPU or in a GPU. To this end, it offers a comprehensive comparison evaluation of the difference in GPU kernel performance on devices of multiple generations of architectures. The idea is to underscore the need for accurate analytical performance models and to provide insights in the evolution of GPU accelerators. This work also highlights a drawback of existing approaches to modelling GPU performance - accurate modelling of memory coalescing characteristics. To that end, we examine a novel application of an inter-thread difference analysis that can further improve analytical models. Finally, this work presents an initial study of an OpenMP runtime framework for target-offloading target selection.

OpenMP is a directive-based shared memory parallel programming model and has been widely used for many years. From OpenMP 4.0 onwards, GPU platforms are supported by extending OpenMP's high-level parallel abstractions with accelerator programming. This extension allows programmers to write GPU programs in standard C/C++ or Fortran languages, without exposing too many details of GPU architectures. However, such high-level programming models generally impose additional program optimisations on compilers and runtime systems. Otherwise, OpenMP programs could be slower than fully hand-tuned and even naive implementations with low-level programming models like CUDA. To study potential performance improvements by compiling and optimising high-level programs for GPU execution, in this paper, we: 1) evaluate a set of OpenMP benchmarks on two NVIDIA Tesla GPUs (K80 and P100); 2) conduct a comparable performance analysis among hand-written CUDA and automatically-generated GPU programs by the IBM XL and clang/LLVM compilers.

As computer hardware continues to dramatically improve in transistor density and raw capability, the importance of compilers for bridging the gap between high-level programming languages and abundant hardware resources has never been greater. In the footsteps of ten previous iterations, this 11th workshop about Compiler-Driven Performance brought together academic faculty, students, industry researchers and developers from across Canada and the United States to meet and discuss state-of-the-art compiler technology, new innovations and future directions. The current state of computing technology motivates a number of key challenges for compilers to address. This workshop had a particular focus on the following: (1) Innovative Analysis, Transformation, and Optimization Techniques: Today's software systems are often composed of several different languages and programming models. At the same time the underlying processors and memory systems are typically complex and out-of-order superscalar processors. Managing this complexity and trying to produce efficient systems in a way that does not increase the burden on programmers requires constant innovation in compilation methods and optimizations. (2) Languages, Compilers, and Optimization Techniques for Multicore Processors and Other Parallel Architectures: As we approach and surpass one billion transistors on a single die, all processor vendors have switched to multicore product lines. Dual and quad core designs are now commodity products and the number of processors contained on future chips will likely double with each generation, soon reaching a thousand processor cores on one chip. This highly-parallel hardware represents a daunting challenge for average programmers; compilation technology is well-suited to ease this parallel programming crisis. Potential solutions range from new languages of expressing concurrency to speculative threading and transactional memory. (3) Compiling for Streaming or Heterogeneous Hardware: In addition to highly-parallel, multiple-core processors for general-purpose and scientific computing, the computer hardware industry is also aggressively pursuing custom computing cores in order to accelerate key applications. Such heterogeneous computing systems were once limited to the embedded domain, but are becoming increasingly common for general-purpose computing. Examples include cores for encryption, compression, pattern matching, systems that have FPGA co-processors, IBM/Sony/Toshiba's CELL processor, which contains multiple vector processors for media and other stream processing, and graphics processing units (GPUs), which will likely soon be incorporated on the chip with regular processors. The resulting heterogeneous hardware presents another key challenge that is addressed by the community targeted by this audience. (4) Dynamic Compilation for High-Performance and Real-Time Environments: Compilers that dynamically translate or optimize programs are of ever-increasing importance, not only to support interpreted languages such as Java, but also to exploit the run-time behaviour of programs written in C and C++ to improve efficiency and performance. Run-time adaptation can usually take advantage of more precise state information, which allows systems to present abstracted interfaces while ensuring an efficient implementation. Such layers of abstraction are important for allowing programmers to efficiently target the emerging highly-parallel and heterogeneous hardware. (5) Compilation and optimization for Scripting Languages: the performance of scripting languages such as python, ruby, php, javascript, and others are of increasing importance to the overall performance of most online systems. These interpreted and often dynamically-typed languages present many challenges in terms of optimization design. They require highly dynamic and adaptive techniques to overcome the lack of static information, language idioms, and novel workloads found in different execution contexts. (6) Compilation Techniques for Reducing Power: Reducing power consumption is a key challenge for all computer systems, from hand-held mobile devices to high-end supercomputers. Compilers that can optimize power or coordinate the power-reduction features of other parts of the system are of great significance. This extends to the compiler itself, incorporating power-friendly methods in the context of dynamic compilation. (7) Program Safety: The size and complexity of many modern software projects makes programming errors both difficult to find and easy to produce. Compiler approaches have shown potential to improve code safety by detecting common bugs ahead of time or by automatically trapping subtle errors at runtime. Such techniques are likely to play an increasing role in software development, ease many analysis and runtime optimization challenges, and represent an interesting further application domain for software analysis. (8) Whole System Optimization and Analysis: Many applications run in a non-trivial context along with other activities or programs. Individual program behaviours and resource competition will then affect overall system performance. Designs that assess complementary or competitive behaviours, or that dynamically adjust individual execution to improve global system usage extend program optimization and analysis techniques to higher-level execution goals. (9) Tools and Infrastructure for Compiler Research: The changing technology landscape highlights the need for ever-improving compiler-based tools as well as infrastructure for understanding programs and performing research. Development of new analysis techniques and optimizations is facilitated by basic program exploration, profiling, visualization and looking for further sources of semantic meaning that can be applied to improve performance, language design, or other optimization goals. Continued language and hardware development requires continued effort to identify new sources of potential optimization, and develop novel techniques for new contexts. This workshop on Compiler-Driven Performance allowed researchers to share progress in developing new approaches to existing problems and in identifying new optimization opportunities and performance vectors for current and future languages. The workshop had the additional benefit of both providing and demonstrating strong cooperation between academia and industry. Compiler optimization research is a highly practical domain, but also one in which cutting-edge research techniques can have direct application, and work in both academic and industrial contexts is improved through this level of cooperation.

In modern supercomputers, nodes are connected by networking hardware capable of up to 40 Gb/s. Data compression could allow for even higher effective bandwidth. However, data compression for such systems requires a unique tradeoff between the compression rate delivered by the compressing scheme and the speed of compression/decompression. While traditional software compression techniques may deliver high compression rates, they cannot maintain the high compression/decompression throughput that is needed. We present Run-Length Base-Delta (RLBD) encoding, a software compression format and algorithm that delivers a high-speed compression/decompression that is suitable for data transfers at up to 40GbE. RLBD can be implemented in CPUs or in parallel accelerator devices to improve network throughput by up to 57% while transmitting data from real datasets.

While GPUs are increasingly popular for high-performance computing, optimizing the performance of GPU programs is a time-consuming and non-trivial process in general. This complexity stems from the low abstraction level of standard GPU programming models such as CUDA and OpenCL: programmers are required to orchestrate low-level operations in order to exploit the full capability of GPUs. In terms of software productivity and portability, a more attractive approach would be to facilitate GPU programming by providing high-level abstractions for expressing parallel algorithms.OpenMP is a directive-based shared memory parallel programming model and has been widely used for many years. From OpenMP 4.0 onwards, GPU platforms are supported by extending OpenMP's high-level parallel abstractions with accelerator programming. This extension allows programmers to write GPU programs in standard C/C++ or Fortran languages, without exposing too many details of GPU architectures.However, such high-level parallel programming strategies generally impose additional program optimizations on compilers, which could result in lower performance than fully hand-tuned code with low-level programming models. To study potential performance improvements by compiling and optimizing high-level GPU programs, in this paper, we 1) evaluate a set of OpenMP 4.× benchmarks on an IBM POWER8 and NVIDIA Tesla GPU platform and 2) conduct a comparable performance analysis among hand-written CUDA and automatically-generated GPU programs by the IBM XL and clang/LLVM compilers.

Programs written in the Unified Parallel C (UPC) language can access any location of the entire local and remote address space via read/write operations. However, UPC programs that contain fine-grained shared accesses can exhibit performance degradation. One solution is to use the inspector-executor technique to coalesce fine-grained shared accesses to larger remote access operations. A straightforward implementation of the inspector executor transformation results in excessive instrumentation that hinders performance.This paper addresses this issue and introduces various techniques that aim at reducing the generated instrumentation code: a shared-data localization transformation based on Constant-Stride Linear Memory Descriptors (CSLMADs) [S. Aarseth, Gravitational N-Body Simulations: Tools and Algorithms, Cambridge Monographs on Mathematical Physics, Cambridge University Press, 2003.], the inlining of data locality checks and the usage of an index vector to aggregate the data. Finally, the paper introduces a lightweight loop code motion transformation to privatize shared scalars that were propagated through the loop body.A performance evaluation, using up to 2048 cores of a POWER 775, explores the impact of each optimization and characterizes the overheads of UPC programs. It also shows that the presented optimizations increase performance of UPC programs up to 1.8 x their UPC hand-optimized counterpart for applications with regular accesses and up to 6.3 x for applications with irregular accesses. (C) 2016 Elsevier B.V. All rights reserved.

Partitioned Global Address Space (PGAS) languages are a popular alternative when building applications to run on large scale parallel machines. Unified Parallel C (UPC) is a well known PGAS language that is available on most high performance computing systems. Good performance of UPC applications is often one important requirement for a system acquisition. This paper presents the memory management techniques employed by the IBM XL UPC compiler to achieve optimal performance on systems with Remote Direct Memory Access (RDMA). Additionally we describe a novel technique employed by the UPC runtime for transforming remote memory accesses on a same shared memory node into local memory accesses, to further improve performance. We evaluate the proposed memory allocation policies for various UPC benchmarks and using the IBM® Power® 775 supercomputer [1].

The workshop on Compiler-Driven Performance provided an important opportunity for academic researchers, and industry researchers and developers from across Canada and in the United States to meet to examine state-of-the-art compiler technology and to discuss future directions for research and development.

Significant progress has been made in the development of programming languages and tools that are suitable for hybrid computer architectures that group several shared-memory multicores interconnected through a network. This paper addresses important limitations in the code generation for partitioned global address space (PGAS) languages. These languages allow fine-grained communication and lead to programs that perform many fine-grained accesses to data. When the data is distributed to remote computing nodes, code transformations are required to prevent performance degradation. Until now code transformations to PGAS programs have been restricted to the cases where both the physical mapping of the data or the number of processing nodes are known at compilation time. In this paper, a novel application of the inspector-executor model overcomes these limitations and allows profitable code transformations, which result in fewer and larger messages sent through the network, when neither the data mapping nor the number of processing nodes are known at compilation time. A performance evaluation reports both scaling and absolute performance numbers on up to 32,768 cores of a Power 775 supercomputer. This evaluation indicates that the compiler transformation results in speedups between 1.15x and 21x over a baseline and that these automated transformations achieve up to 63 percent the performance of the MPI versions.

Programs written in Partitioned Global Address Space (PGAS) languages can access any location of the entire address space via standard read/write operations. However, the compiler have to create the communication mechanisms and the runtime system to use synchronization primitives to ensure the correct execution of the programs. However, PGAS programs may have fine-grained shared accesses that lead to performance degradation. One solution is to use the inspector-executor technique to determine which accesses are indeed remote and which accesses may be coalesced in larger remote access operations. A straightforward implementation of the inspector-executor in a PGAS system may result in excessive instrumentation that hinders performance. This paper introduces a shared-data localization transformation based on linear memory descriptors (LMADs) that reduces the amount of instrumentation introduced by the compiler into programs written in the UPC language and describes a prototype implementation of the proposed transformation. A performance evaluation, using up to 2048 cores of a POWER 775 supercomputer, allows for a prediction that applications with regular accesses can achieve up to 180% of the performance of handoptimized versions while applications with irregular accesses yield performance gain from 1.12X up to 6.3X speedup.