A novel distributed-resource abstraction environment is introduced. You can access any resource in a computer network as a memory mapped I/O device, as if it was attached to the local bus of your PC. This network technology gives us several benefits. From the application development viewpoint, no network-related programming is required, and we don't need to modify the applications even if the network topologies and protocols are changed. On the other hand, network maintenance and upgrading can be done anytime without worrying about the application users, because the environment completely separates or hides the network from the applications. The API (Application Program Interface), a resource abstraction mechanism, and a directory service are implemented. In addition, a reconfigurable hardware technology is adopted to perform autonomous network control using a lour layer protocol. Furthermore, we introduce a testbed that allows heterogeneous resources to be utilized, and demonstrate the feasibility of our concept using some applications
The efficient implementation of collective communication operations has received much attention. Initial efforts modeled network communication and produced "optimal" trees based on those models. However, the models used by these initial efforts assumed equal point-to-point latencies between any two processes. This assumption is violated in heterogeneous systems such as clusters of SMPs and wide-area "computational grids", and as a result, collective operations that utilize the trees generated by these models perform suboptimally. In response, more recent work has focused on creating topology-aware trees for collective operations that minimize communication across slower channels (e.g., a wide-area network). While these efforts have significant communication benefits, they all limit their view of the network to only two layers. We present a strategy based upon a multilayer view of the network. By creating multilevel topology trees we take advantage of communication cost differences at every level in the network. We used this strategy to implement topology-aware versions of several MPI collective operations in MPICH-G, the Globus-enabled version of the popular MPICH implementation of the MPI standard. Using information about topology discovered by Globus, we construct these topology-aware trees automatically during execution, thus freeing the MPI application programmer from having to write special files or functions to describe the topology to the MPICH library. We present results demonstrating the advantages of our multilevel approach by comparing it to the default (topology-unaware) implementation provided by MPICH and a topology-aware two-layer implementation.
Two different approaches have been commonly used to address problems associated with space sharing scheduling strategies: (a) augmenting space sharing with backfilling, which performs out of order job scheduling; and (b) augmenting space sharing with time sharing, using a technique called coscheduling or gang scheduling. With three important experimental results-impact of priority queue order on backfilling, impact of overestimation of job execution times, and comparison of scheduling techniques-this paper presents an integrated strategy that combines backfilling with gang scheduling. Using extensive simulations based on detailed models of realistic workloads, the benefits of combining backfilling and gang scheduling are clearly demonstrated over a spectrum of performance criteria.
This paper presents a new approach to improve performance of Java programs by extending the superthreaded speculative execution model to exploit coarse-grained parallelism on a shared-memory multiprocessor system. The parallelization model, called Java Speculative MultiThreading (JavaSpMT), combines control speculation with run-time dependence checking to parallelize a wide variety of loop constructs, including do-while loops, that cannot be parallelized using standard parallelization techniques. JavaSpMT is implemented using the standard Java multithreading mechanism and the parallelization is expressed using a Java source-to-source transformation. Thus, the transformed programs are still portable to any shared-memory multiprocessor system with a Java Virtual Machine implementation that supports native threads.
This paper presents experimental results for a parallel pipeline STAP system with I/O task implementation using the parallel file systems on the Intel Paragon and the IBM SP. In our previous work, a parallel pipeline model was designed for radar signal processing applications on parallel computers. Based on this model, we implemented a real STAP application which demonstrated the performance scalability of this model in terms of throughput and latency. In this paper we study the effect on system performance when the I/O task is incorporated in the parallel pipeline model. There are two alternative for I/O implementation: embedding I/O in the pipeline or having a separate I/O task. From these two I/O implementations, we discovered that the latency may be improved when the structure of the pipeline is reorganized by merging multiple tasks into a single task. All the performance results shown in this paper demonstrated the scalability of parallel I/O implementation on the parallel pipeline STAP system
Reconstructing a physical map of a chromosome from a genomic library presents a central computational problem in genetics. Physical map reconstruction in the presence of errors is a problem of high computational complexity which provides the motivation for parallel computing. Parallelization strategies for a maximum likelihood estimation-based approach to physical map reconstruction are presented. The estimation procedure entails gradient descent search for determining the optimal spacings between probes for a given probe ordering. The optimal probe ordering is determined using a stochastic optimization algorithm. A two-tier parallelization strategy is proposed wherein the gradient descent search is parallelized at the lower level and the stochastic optimization algorithm is simultaneously parallelized at the higher level. Implementation and experimental results on a distributed memory multiprocessor cluster running the Parallel Virtual Machine (PVM) environment are presented.
Dynamic replication algorithms aim at allocating, migrating and deleting copies of an object over various Internet hosts, according to the access patterns exhibited online, so as to improve object proximity for the end-users and/or load-balance the servers. Most of the existing algorithms try to disseminate the objects of an entire Internet Service Provider (ISP), without taking into account the needs and characteristics of specific web sites with large commercial value. In this paper we tackle the replication problem in an Internet environment, inspired by the need of news agencies and other information providers to include in their pages multimedia content without incurring high access delays. We consider an environment that consists of a central multimedia repository and various physically dispersed sires. We propose a cost model to formalize the replication of multimedia objects located at the repository which can result in decreasing the download time. Taking into account implementation issues, such as the storage and processing capacity constraints, the proposed replication policy is evaluated and compared with alternatives including an ideal LRU caching scheme. Qualitative comparisons with the other replication schemes are reported as well.
Cache coherence enforcement and memory latency reduction and hiding are very important and challenging problems in the design of large-scale distributed shared-memory (DSM) multiprocessors. We propose an integrated approach to solve these problems through a compiler-directed cache coherence scheme called the Cache Coherence with Data Prefetching (CCDP) scheme. The CCDP scheme enforces cache coherence by prefetching the potentially stale references in a parallel program. It also prefetches the non-stale references to hide their memory latencies. To optimize the performance of the CCDP scheme, some prefetch hardware support is provided to efficiently handle these two forms of data prefetching operations. We also developed the compiler techniques utilized by the CCDP scheme for stale reference detection, prefetch target analysis, and prefetch scheduling. We evaluated the performance of the CCDP scheme via execution-driven simulations of several numerical applications from the SPEC CFP95 and the Perfect benchmark suites. The simulation results show that the CCDP scheme provides significant performance improvements for the applications studied, comparable to that obtained with a full-map hardware cache coherence scheme.
Metacomputing systems are intended to support remote and/or concurrent use of geographically distributed computational resources. Resource management in such systems is complicated by five concerns that do not typically arise in other situations: site autonomy and heterogeneous substrates at the resources, and application requirements for policy extensibility, co-allocation, and online control. We describe a resource management architecture that addresses these concerns. This architecture distributes the resource management problem among distinct local manager, resource broker, and resource co-allocator components and defines an extensible resource specification language to exchange information about requirements. We describe how these techniques have been implemented in the context of the Globus metacomputing toolkit and used to implement a variety of different resource management strategies. We report on our experiences applying our techniques in a large testbed, GUSTO, incorporating 15 sites, 330 computers, and 3600 processors.
Instruction scheduling methods based on the construction of state diagrams (or automata) have been used for architectures involving deeply pipelined function units. However, the size of the state diagram is prohibitively large, resulting in high execution time and space requirement. We present a simple method for reducing the size of the state diagram by recognizing unique paths of a state diagram. Our experiments show that the number of paths in the reduced state diagram is significantly lower-by 1 to 3 orders of magnitude-compared to the number of paths in the original state diagram. Using the reduced MS-state diagrams, we develop an efficient software pipelining method. The proposed software pipelining algorithm produced efficient schedules and performed better than R.A. Huff's (1993) Slack Scheduling method, and the original Co-scheduling method, in terms of both the initiation interval (II) and the time taken to construct the schedule
Efficient performance tuning of parallel programs is often hard. We describe an approach that uses a uni-processor execution of a multithreaded program as reference to simulate a multiprocessor execution. The speed-up is predicted, and the program behaviour is visualized as a graph, which can be used in the performance tuning process. The simulator considers scheduling as well as hardware parameters, e.g., the thread priority, no. of LWPs, and no. of CPUs. The visualization part shows the simulated execution in two graphs: one showing the threads' behaviour over time and the other the amount of parallelism over time. In the first graph it is possible to relate an event in the graph to the code line causing the event. Validation using a Sun multiprocessor with eight processors and five scientific parallel applications shows that the speed-up predictions are within +/-6% of a real execution
A unicast-based multicasting algorithm is presented for arbitrary interconnection networks arising in switch-based networks of workstations. The algorithm is optimal with respect to the number of startups incurred and is provably free from depth contention. Specifically, no two constituent unicasts for the same multicast contend for a common channel, even if some unicasts are delayed due to unpredictable variations in latencies. The algorithm uses an underlying partially adaptive deadlock-free unicast routing algorithm. Simulation results indicate that the algorithm behaves as predicted by its theoretical properties and provides a promising approach to unicast-based multicasting.
In coming years, if not already, the parallel-processing community can expect to hear regularly from MPP advocates and cluster advocates about why their approach is better. Either pitch is apt to be a hard sell: hard to sell to an informed audience or reader, and dull. The attempt to distinguish between MPPs and clusters is in some cases an empty subject. By the term "cluster," I mean a group of high-performance computers-" commodity" PCs, workstations, or SMPs-connected by a fast network, and employing software layers for low-latency message handling and a standard interface for multiple-process application programs. Indeed, today's multicomputers (distributed-memory MPPs) fit this definition of a cluster except for being packaged in a cabinet rather than distributed across a room or building. MPP bigots can be expected to present positive arguments of tight coupling and the virtues of nodes employing processor types not used in commodity computers. They may also present negative arguments of the limits imposed eventually by the speed-of-light delays in clusters. They will be right on all points, at least eventually. Cluster bigots can be expected to present different arguments involving the limited size of the MPP market in comparison with the PC market, and the resulting differences in investment, technology-insertion schedule, and performance per unit cost. They will also be right on all points, at least empirically, and will have persuasive benchmarks to prove these points. The obvious trouble with blind advocacy is that it distracts from the opportunities for the parallel-processing community to embrace both approaches, and thereby to extend the applicability of parallel processing. In certain application domains, such as military systems, it becoming commonplace for people to build systems composed of specialized, "embedded" MPPs clustered together with conventional computers. On the software side, a number of streamlined, message-handling, software layers, such as the University of California, Berkeley, Active Messages (AM) layer and the University of Illinois, Urbana-Champaign, Fast Messages (FM) layer, operate interchangeably on both clusters and MPPs. At the level of application programming and libraries, standard application-programming interfaces such as MPI are likewise portable between MPPs and clusters. There are many other needs and opportunities for research that will unify rather than divide MPPs and clusters. Let me join you in hoping that this is the last talk that you will have to endure on MPPs "versus" clusters.
We present a novel optimization called Last Parallel Call Optimization (LPCO) for parallel systems. The last parallel call optimization can be regarded as a parallel extension of last call optimization found in sequential systems. While the LPCO is fairly general, we use and-parallel logic programming systems to illustrate it and to report its performance on multiprocessor systems.The last parallel call optimization leads to improved time and space performance for a majority of and-parallel programs. We also present a generalization of the Last Parallel Call Optimization called Nested Parallel Call Optimization (NPCO). A major advantage of LPCO and NPCO is that parallel systems designed for exploiting control parallelism can automatically exploit data parallelism efficiently.
In this paper, we explore the design issues of a shared bus with pipelined protocol, so called HiPi+Bus, which is implemented for a multiprocessor server. The characteristics and design parameters for the HiPi+Bus are described. In the viewpoint of a pipelined bus, a block transfer is no good because of involving complex and unbalanced pipeline. However, it is requested by a local cache memory of which line size tends to be increased. To get the best performance and compensate unbalanced data transfer characteristic caused by block transfer, a responder queue for the bus interface is also proposed. According to the simulation results, it is explored that the HiPi+Bus, with help of the responder queue, can provide balanced service for more than 16 processors, which is important in running commercial applications. The HiPi+Bus is implemented for the TICOM III, a successor of the TICOM II which is the main server of the national administrative information network in Korea.
Scalability has never been more a part of System/390 than with Parallel Sysplex. The Parallel Sysplex environment permits a mainframe or Parallel Enterprise Server to grow from a single system to a configuration of 32 systems (initially), and appear as a single image to the end user and applications. The IBM S/390 Parallel Sysplex provides capacity for today's largest commercial workloads by enabling a workload to be spread transparently across a collection of S/390 systems with shared access to data. By way of its parallel architecture and MVS operating system support, the S/390 Parallel Sysplex offers near-linear scalability and continuous availability for customers' mission-critical applications. S/390 Parallel Sysplex optimizes responsiveness and reliability by distributing workloads across all of the processors in the Sysplex. Should one or more processors fail, the workload is redistributed across the remaining processors. Because all of the processors have access to all of the data, the Parallel Sysplex provides a computing environment with near-continuous availability.
Recently, there has been a considerable upsurge in cluster based computing. Centre for Development of Advanced Computing, at Pune, India is also offering a cluster computing solution based on its proprietary network (More information is available on request from Business Division, C-DAC, Pune 411 007, India.). This network is built around a high speed interconnect which uses DS-link protocol, a part of IEEE 1355 standard. One of the disadvantages that prevents DS links to operate in a geographically distributed computing environment is its distance limitation. In this paper, we have sought to remove this limitation by suggesting a scheme for implementing DS links over optical fibers. This scheme has been designed and implemented by us. Here, we discuss the salient features of our scheme. Our preliminary studies indicate that a cluster computing solution using this extended link will be an attractive solution.
This paper presents the "Planned Direct Transfer" programming model, developed by Mercury Computer Systems to meet the requirements of embedded high-performance computing applications. In this model, data transfers are "Planned" before they occur, resulting in low software overhead execution; they are also "Direct"-they do not require intermediate data copying. This paper locates the Planned Direct Transfer (PDT) model in the landscape of the standard approaches of shared memory and message passing.
