In this work, we investigate the idea of incorporating caches into selected switching elements of a multistage interconnection network (MIN)-based multiprocessor. Along with the processor private caches, these switch caches form a two-level cache hierarchy. Selected switch caches within a particular stage of the MIN are connected by a coherence control bus, through which a write-invalidate cache coherence protocol maintains coherence of private and switch caches. The impact of the protocol on system performance is evaluated through a simulation-based performance study, which shows that it is feasible to build large shared-memory cache-coherent multiprocessor systems.
The paper deals with mesh-connected massively parallel systems affected by failures. The complexity of a local diagnostic procedure, based on new definitions of the local k-diagnosability and the r-fault-tolerance, is analysed. It depends on distances between individual fault clusters and on fault cluster diameters. In particular cases the minimum distance between fault clusters can be enlarged on the account of the maximum fault cluster diameter. The criterion function for the optimization of the diagnostic examination is proposed.
We present efficient parallel algorithms for recognizing chordal graphs and locating all maximal cliques of a chordal graph G=(V,E). Our techniques are based on partitioning the vertex set V using information contained in the distance matrix of the graph. We use these properties to formulate parallel algorithms which, given a graph G=(V,E) and its adjacency-level sets, decide whether or not G is a chordal graph, and, if so, locate all maximal cliques of the graph in time O(k) by using δ2·n2/k processors on a CRCW-PRAM, where δ is the maximum degree of a vertex in G and 1≤k≤n. The construction of the adjacency-level sets can be done by computing first the distance matrix of the graph, in time O(logn) with O(nβ+DG) processors, where DG is the output size of the partitions and β=2.376, and then extracting all necessary set information. Hence, the overall time and processor complexity of both algorithms are O(logn) and O(max{δ2·n2/logn, nβ+DG}), respectively. These results imply that, for δ≤√nlogn, the proposed algorithms improve in performance upon the best-known algorithms for these problems.
The branching regions of data-parallel programs can lead to serial execution on a SIMD processor. However, recent investigations show that these branching regions can be compiled into instruction sequences that are placed in the Processing Element (PE) memories for interpretation. Thus, when the branching region is reached, the control unit of the SIMD processor executes an instruction interpreter that causes the concurrent execution (through interpretation) of the branching code at each PE. Interpretation incurs some overhead and therefore, it is necessary to carefully design the instruction set to be interpreted. In this paper, we present a software architecture called Compose with 9 functional units and show that these functional units can be composed together to realize (at least) 42 distinct useful instructions. Thus, each iteration of the interpreter loop pays only the overhead of broadcasting execution orders for a few operations and yet manages to interpret (in parallel) a much larger set of operations.
Data parallel languages such as Vienna Fortran and HPF can be successfully applied to a wide range of numerical applications. However, many advanced scientific and engineering applications are of a multidisciplinary and heterogeneous nature and thus do not fit well into the data parallel paradigm. In this paper we present new Fortran 90 language extensions to fill this gap. Tasks can be spawned as asynchronous activities in a homogeneous or heterogeneous computing environment; they interact by sharing access to Shared Data Abstractions (SDAs). These mechanisms support the integration of data and task parallelism as well as nested task parallelism and thus can be used to express multidisciplinary applications in a natural and efficient way.
The ”classical parallelizers” integrate more and more sophisticated and costly parallelization techniques. As a result, they are limited by the size of the program they are able to compile and are not well suited to parallelize real scientific programs whose parallelism detection complexity may differ very much from one code fragment to another.
This paper presents an overlapping technique of communications by computations based on pipelined communications. This allows to improve the execution time of most parallel numerical algorithms. Some simple examples are developed to illustrate the efficiency of this technique: matrix-vector product and bi-dimensional Fast Fourier Transform. Moreover, we propose an unified formalism to express easily the pipelined versions of these algorithms. Finally, we report some experiments on various parallel machines.
A visual language paradigm for the interactive specification of application algorithms is proposed for consideration. The approach is based on a set of computational schemes (”shapes” of computation) presented by color figures, pictures and animation films with sound accompaniment. Each film is related to series of frames (computational steps) and it reflects some knowledge about data processing. Each frame “brightens” up a substructure of data for which operations should be specified. As a rule, this substructure is a set of points and/or moving objects in a multi-dimensional space-time. A user embeds his algorithm into computational schemes by making these schemes more precise. In fact, he defines the specification by creating his new film. The corresponding program (sequential or parallel) is generated automatically.
Interval routing was introduced to reduce the size of the routing tables. This way of implementing routing functions is quite attractive but very few is known on the topological properties that should satisfy a network to admit an interval routing function satisfying particular constraints (shortest paths, limited number of intervals associated to each direction, etc). In this paper, we investigate the study of optimal interval routing functions, that is routing functions that construct shortest paths. In particular we derive practical tools that allow to determine if a network supports or not an optimal interval routing function. We describe large classes of networks that admit optimal interval routing functions. We also study the case of the usual networks that interconnect the processors of a distributed memory parallel computer.
This paper describes the methods used and experiences made with implementing a finite element application on three different parallel computers with either message passing or shared virtual memory as the programming model. Designing a parallel finite element application using message-passing requires to find a data domain decomposition to map data into the local memory of the processors. Since data accesses may be very irregular, communication patterns are unknown prior to the parallel execution and thus makes the parallelization a difficult task. We argue that the use of a shared virtual memory greatly simplifies the parallelization step. It is shown experimentally on an hypercube iPSC/2 that the use of the KOAN/Fortran-S programming environment based on a shared virtual memory allows to port quickly and easily a sequential application without a significant degradation in performance compared to the message passing version. Results for recent parallel architectures such as the Paragon XP/S for message-passing and the KSR1 for shared virtual memory are presented, too.
Volume rendering is a computational challenge even for todays high performance computers. In the field of supercomputing there is a trend towards massively parallel machines with distributed memory architectures. The algorithm presented here is capable of exploiting the processing power of such machines most efficiently. Arbitrary viewing directions are supported without the need for dynamic data redistribution during rendering. Nevertheless, only small amounts of volume data have to be replicated in local processor memories. A scheme is presented for accumulating partial images generated locally on each processor which avoids the bottleneck problems of previous approaches. We report on an implementation of this renderer on a distributed memory machine with 1024 processors connected in a hypercube topology.
The Rewrite Rule Machine (RRM) is a massively parallel MIMD/SIMD computer designed with the explicit purpose of supporting very-high-level parallel programming with rewrite rules. The RRM's node architecture consists of a SIMD processor, a SIMD controller, local memory, and network and I/O interfaces. A 64-node cluster board is already an attractive RRM system capable of extremely high performance on a variety of applications. A cluster is SIMD at the node level, but it is MIMD at the system level to flexibly exploit the parallelism of complex nonhomogeneous applications. In addition to reporting detailed simulation experiments used to validate the node design, we measure the performance of an RRM cluster on three relevant applications.
Self-organizing feature maps are presented as an effcient tool for mapping process graphs onto processor networks. Arbitrary process graphs can be mapped to most of the common parallel architectures (two-dimensional lattice, three-dimensinal torus, hypercube, etc.). Two extensions of the Kohonen algorithm for self-organizing feature maps were necessary. A special graph metric allows the support of arbitrary process graphs and a modification of the learning rule added a load balancing facility. The order of computational complexity is restricted to O(m^3) (m denoting the number of processes) in the worst case. The method can be adapted to a wide variety of further graph mapping problems (e.g. circuit design, production planning, scheduling).
It is well known that communication time is one of the most limiting factors for obtaining high performance programs for parallel architectures.
Linda, a general purpose coordination language, has been used to make a language parallel. Based on a logically shared tuplespace, Linda poses difficulties to be efficiently implemented on a distributed multiprocessor system. This paper reports our approach to solve the problem: processors are divided into groups, and each group has a group manager to provide a local view of global tuplespace, and handles the tuplespace operations incurred by processors within the group. To maintain the consistency and correctness of the Linda tuplespace operations, we propose the algorithms of a group manager. We also implement the algorithms on a network of transputers, and show the experiment results.
This paper reports on some experiments with the implementation of a concurrent version of a graph reduction system π-Red+ on an nCUBE/2 system of up to 32 processing sites. They primarily concern basic concepts of workload partitioning and balancing, the relationship between relative performance gains and the computational complexities of the investigated programs, resource management and suitable system topologies. All programs used for these experiments realize divide_and_conquer algorithms and have been run with varying (sizes of) data sets and system parameters (configurations).
Efficient data communication through an interconnection network in the presence of faulty nodes and links is an important issue in multiprocessor systems. The newly proposed folded Petersen networks, constructed by iteratively applying the cartesian product operation on the well-known Petersen graph, provide a very efficient topology for multiprocessor architectures. In this paper, we construct multiple arc-disjoint spanning trees on the folded Petersen networks and utilize them for deriving fault-tolerant algorithms for several communication primitives. These primitives include single node broadcasting, gossiping, scattering, and total exchange.
A new Mean Field Annealing (MFA) formulation is proposed for the mapping problem for mesh-connected architectures. The proposed MFA heuristic exploits the conventional routing scheme used in mesh interconnection topologies to introduce an efficient encoding scheme. An efficient implementation scheme which decreases the complexity of the proposed algorithm by asymptotical factors is also developed. Experimental results also show that the proposed MFA heuristic approaches the speed performance of the fast Kernighan-Lin heuristic while approaching the solution quality of the powerful simulated annealing heuristic.
The rapid advances in high-performance computer architecture and compilation techniques provide both challenges and opportunities to exploit the rich solution space of software pipelined loop schedules. In this paper, we develop a framework to construct a software pipelined loop schedule which runs on the given architecture (with a fixed number of processor resources) at the maximum possible iteration rate (a la rate-optimal) while minimizing the number of buffers-a close approximation to minimizing the number of registers. The main contributions of this paper are: First, we demonstrate that such problem can be described by a simple mathematical formulation with precise optimization objectives under a periodic linear scheduling framework. The mathematical formulation provides a clear picture which permits one to visualize the overall solution space (for rate-optimal schedules) under different sets of constraints. Secondly, we show that a precise mathematical formulation and its solution does make a significant performance difference. We evaluated the performance of our method against three leading contemporary heuristic methods. Experimental results show that the method described in this paper performed significantly better than these methods. The techniques proposed in this paper are useful in two different ways: 1) As a compiler option which can be used in generating faster schedules for performance-critical loops (if the interested users are willing to trade the cost of longer compile time with faster runtime). 2) As a framework for compiler writers to evaluate and improve other heuristics-based approaches by providing quantitative information as to where and how much their heuristic methods could be further improved.
The “arrowhead torus” is a broadcast graph that we define on the 6-valent grid as a Cayley graph. A 3-port wormhole broadcasting protocol is derived first from construction, then improved by using edgedisjoint forests. A store-and-forward broadcasting protocol is derived afterwards, then improved by mixing pipelining and arc-disjoint spanning trees. Costs are given in constant and linear time and compared with lower bounds.
