The efficiency of scheduling algorithms is essential in order to attain optimal performances from parallel programming systems. In this paper we use a portable parallel programming environment we have implemented, the pSystem, to evaluate and compare the performance of various scheduling algorithms on shared memory parallel machines.
A fully distributed OR-parallel Prolog implementation scheme is shown in the paper based on the recomputation model. The main advantage of the recomputation model is that only little modification is needed to the sequential WAM engine in order to achieve a distributed Prolog system. The paper describes the new data structures of the parallel WAM engine and explains how the distributed scheduler can be organized. Finally a broadcast message based cavallier cut implementation scheme is shown in the paper.
A new routing scheme for concatenating two omega networks is presented. Routing the 2log2N-stage concatenated network has been an open problem for decades, and no clear solution has yet been proposed. It was shown in [5] that this asymmetric network can be converted into a symmetric network with butterfly connections between the two center stages. The proposed algorithm routes the network from center stages to outer stages at both the input and the output sides, simultaneously. The algorithm presented is simpler and more flexible than other algorithms known so far, in that it can be applied adaptively, according to the structure of the network.
We provide a new paradigm to treat insertion and deletion of a batch of edges in a graph, that makes use of the sparsification technique developed for on-line algorithms. In particular, we refer to the problems of minimal spanning forest (MSF), connected components (CC) and k-vertex-connectivity (k-VC). Our batch algorithms, improve of a log-factor over the classical one-by-one algorithms, for k- VC. This improvement is limited to batches of properly “large” size for MSF and CC. In parallel computation we discuss MSF and CC under single edge updating, batch insertions and batch updating, extending the use of the sparsification data structure 10 the CRCVV PRAM model and providing efficient parallel algorithms for these problems (the batch algorithms for CC are indeed work-optimal). We also discuss the difficulty of obtaining a parallel dynamic algorithm for the k-vertex connectivity problem.
In this paper, we propose a dataflow-based massively parallel programming language, called V, which is loosely based on a data-flow oriented functional programming language. The language provides a programming unit, or agent, to write parallel entities communicating with each other. In addition, we can connect agents explicitly and abstract an ensemble of agents on a predefined topology description in order to write a massively parallel program that naturally reflects the structure of a problem. We also present some implementation issues and a preliminary evaluation of our compiler and runtime system developed for the Fujitsu AP1000, a distributed-memory parallel machine with conventional processors.
Many inner loop bodies in numerical applications are short with a disproportionally large amount of array operations which are potentially remote operations. Consequently, fewer threads can remain active while the requested data is in transit. When such situations arise, a processor can either remain idle or switch to another activation. Both options result in poor processor utilization which adversely affects the overall performance of a program. Because these loops are often the kernel of an application, there is a strong incentive for improving the performance of such loops. As a solution, this paper proposes a fine-grain execution model called the Multiple Iterations per Activation (MIpA) model.
A framework is presented to show formally that a parallel system is fair with respect to any set of events. Systems are specified by means of a set of variables, a set of guarded transitions and a set of liveness conditions, using an already presented technique [PA93]. Their semantics are given in the CSP model [BRH84]. The proofs use normal predicate logic and noetherian induction. Invariants are an important piece of information in the development of the proofs. A non trivial case study is developed.
The load balancing scheme of a massively parallel computer system named PISMA is analysed in this paper. The theory behind the load balancing scheme is presented and several variations of the same scheme, which differ in complexity, are studied with the help of a simulator. It is shown that with a 10% overhead for load calculation a well balanced system can be obtained, which can execute a range of applications efficiently.
In this paper, we introduce the Datarol-II processor, that can efficiently execute a fine-grain multi-thread program, called Datarol. In order to achieve the efficient multi-thread execution by reducing context switching overhead, we introduce an implicit register load/store mechanism in the execution pipeline. A two-level hierarchical memory system is also introduced in order to reduce memory access latency. The simulation results show that the Datarol-II processor can tolerate remote memory access latencies and execute a fine-grain multi-thread program efficiently.
Although directory-based write-invalidate cache coherence protocols have a potential to improve the performance of large-scale multiprocessors, coherence misses limit the processor utilization. Therefore, so called competitive-update protocols — hybrid protocols between write-invalidate and write-update — have been considered as a means to reduce the coherence miss rate and have been shown to be a better coherence policy for a wide range of applications. Unfortunately such protocols may cause high traffic peeks for applications with extensive use of migratory objects. These traffic peeks can offset the performance gain of a reduced miss rate if the network bandwidth is not sufficient. We propose in this study to extend a competitive-update protocol with a previously published adaptive mechanism that can dynamically detect migratory objects and reduce the coherence traffic they cause. Detailed architectural simulations based on five scientific and engineering applications show that this adaptive protocol can outperform a write-invalidate protocol by reducing the miss rate and bandwidth need by as much as 71
Relational structures representing causality and weak causality can be used to provide a ‘truly concurrent’ semantics of Petri Nets with priorities. We show how to derive such structures by generalising the standard construction of causal partial orders based on occurrence nets.
We study the serial correctness of programs in a subset of Fortran X3H5, a control-parallel extension of Fortran. This property, an equivalence between a parallel program and its sequential version, follows from the preservation of dependences, defined on the sequential version, by the control flow and the synchronizations. To check this preservation, we propose an algorithm which builds a formula, using a new kind of block graph. Under a linearity assumption, the algorithm tries to prove that this formula is a tautology by means of the Omega test.
We introduce a technique for scheduling task graphs onto distributed memory architectures factoring in the realistic constraints of interconnection, routing and channel allocation. We depart from conventional schedulers in our use of a bottom up approach for scheduling the tasks to get a better estimate of their scheduling weights. A variety of heuristics are employed for load balancing, routing and the minimization of channel contention. Although we present experimental results for a message switched hypercube and mesh connected system, our proposed scheme can be easily adapted to other topologies like fat trees and X-meshes, as well as other routing strategies such as wormhole routing and circuit switching.
In this article, we describe the Massively AGent Execution system (MAGE), whose purpose is to facilitate the implementation of multiagent applications in a massively parallel environment. MAGE is a parallel programming environment that supports the implementation of each agent in a Prolog system extended with communication primitives used for the cooperation and the interaction among the agents.
We present a small language ℒ and its typing system, starting from the idea of interaction, one of the important notions in parallel and distributed computing. ℒ is based on, apart from such constructs as parallel composition and process creation, three pairs of communication primitives which use the notion of a session, a semantically atomic chain of communication actions which can interleave with other such chains freely, for high-level abstraction of interactionbased computing. The three primitives enable programmers to elegantly describe complex interactions among processes with a rigorous type discipline similar to ML [4]. The language is given formal operational semantics and a type inference system, regarding which we prove that if a program is well-typed in the typing system, it never causes run-time error due to type inconsistent communication patterns, offering a new foundation for type discipline in parallel programming languages.
Parallel functional programs based on the graph reduction execution model display considerable locality of reference, favouring the use of large cache lines in the implementation of the shared heap on a shared-memory multiprocessor. They also display a very high rate of synchronisation, making conventional weakly-consistent coherency protocols ineffective at avoiding unnecessary contention for write access to cache lines due to false sharing. We present the design of a specially adapted cache coherency protocol and show results of simulation experiments which demonstrate that the protocol allows spatial locality to be exploited to at least the level of a conventional invalidation protocol, but without the unnecessary serialisation and network transactions caused by false sharing.
This paper presents a parallel simulated annealing algorithm for solving the problem of mapping irregular parallel programs onto homogeneous processor arrays with regular topology. The algorithm constructs and uses joint transformations. These transformations guarantee a high degree of parallelism that is bounded below by ⌈¦N p /deg(G p +1⌋, where ¦N p ¦ is the number of task nodes in the mapped program graph G p and deg(G p ) is the maximal degree of a node in G p . The mapping algorithm provides good program mappings (in terms of program execution time and the number of processors used) in a reasonable number of steps.
Efficient utilisation and scalability of a parallel system strongly depend on the load distribution and balancing mechanisms. We analyse the behaviour of a load distribution and balancing mechanism by modeling the parallel system as an extended queueing model with subsequent evaluation by the modeling and analysis tool, MAOS. Our focus is on search-based parallel systems (here, the OR-parallel theorem prover PARTHEO), running on a loosely coupled multiprocessor environment.
Graph Traverse Scheduling, GTS [ALTB89], was developed as a technique suitable for parallelizing DO loops with cycles on their dependence graph, but actually, GTS is a generalization of many loop restructuring techniques. The improvements achieved with its application in front of other techniques are the reduction of synchronizations, the introduction of scalar optimizations and the better use of the memory hierarchy. The purpose of this work has been the implementation of this new technique for a particular existing shared-memory multiprocessor and its integration on a source-to-source restructuring compiler. We studied the limitations imposed by the architecture to the technique and modified it in order to adapt the generated parallel code to the architecture. Finally, we measured the improvements of the new scheduling comparing the execution time of some example loops obtained by the commercial parallelizer with the execution time obtained by GTS.
A scalable SIMD architecture has been developed for the most efficient implementation of binary pattern classification by nearestneighbor algorithms. A two-dimensional M × N array of asynchronous counters, reflecting an inherent two-fold data parallelism of the applications, reduces the data transfer to off-chip memory from \(\mathcal{O}(M \times N)\) to \(\mathcal{O}(M + N)\) which allows a high integration and efficient use of external memory. Here, we present the realization of a VLSI structure, the system architecture, and possible applications including binary kNN and a completely binary version of k-means.
