Program Centric Cost Models for Locality and Parallelism

Good locality is critical for the scalability of parallel computations. Many cost models that quantify locality and parallelism of a computation with respect to specific machine models have been proposed. A significant drawback of these machinecentric cost models is their lack of portability. Since the design and analysis of good algorithms in most machine-centric cost models is a non-trivial task, lack of portability can lead to a significant wastage of design effort. Therefore, a machineindependent portable cost model for locality and parallelism that is relevant to a broad class of machines can be a valuable guide for the design of portable and scalable algorithms as well as for understanding the complexity of problems. This thesis addresses the problem of portable analysis by presenting programcentric metrics for measuring the locality and parallelism of nested-parallel programs written for shared memory machines–metrics based solely on the program structure without reference to machine parameters such as processors, caches and connections. The metrics we present for this purpose are the parallel cache complexity (Q∗) for quantifying locality, and the effective cache complexity (̂Qα) for both locality and the cost of load-balancing for a parameter α≥ 0. These two program-centric metrics constitute the parallel cache complexity (PCC) framework. We show that the PCC framework is relevant by constructing a class of provablygood schedulers that map programs to the parallel memory hierarchy (PMH) model, represented by a tree of caches. We prove optimal performance bounds on the communication costs of our schedulers in terms of Q∗ and on …