The Paradigm Compiler for Distributed-Memory Multicomputers

The authors describe a flexible compiler framework for distributed-memory multicomputers, called Paradigm (Parallelizing Compiler for Distributed-Memory, General-Purpose Multicomputers). To extract computational power from a multicomputer, users must often expend significant time and energy to write efficient software. Paradigm addresses this problem by automatically parallelizing sequential programs. Besides handling traditional compiler optimizations, Paradigm focuses on several other areas within a unified platform. These include automatic data distribution, communication optimizations, support for irregular computations, exploitation of functional and data parallelism, and multithreaded execution. Automatic data partitioning involves several choices. These include array alignment, distribution (block or cyclic), block size, and mesh configuration. Paradigm addresses these decisions in distinct phases. The compiler supports both regular and irregular computations. For regular computations, the compiler uses efficient processor-tagged descriptors to handle the simplest and most frequent cases. It uses more general, inequality-based representations for the difficult cases. This lets Paradigm compile a larger proportion of programs without jeopardizing compilation speed. In addition, to reduce the overhead caused by frequent communications, the compiler employs message coalescing, message verification, message aggregation, and coarse-grain pipelining. For irregular computations, Paradigm uses two sequences of code: an inspector for preprocessing and an executor for performing the actual computations. The Parallel Irregular Library with Application of Regularity (PILAR) provides Paradigm's irregular runtime support. Finally, the authors describe how Paradigm uses functional and data parallelism and multithreading to improve overall execution efficiency.