Flexible data structures have become a common tool of programming also in the field of engineering simulation and scientific simulation in the last years. Standard programming languages like Fortran and C or C++ allow to specify user defined datatypes for such structures. For parallel programming this leads to a special problem when it comes to exchanging data between processors. Regular data structures occupy contiguous space in memory and can thus be easily transferred to other processes when necessary. Irregular data structures, however, are more difficult to handle and the costs for communicating them may be rather high. MPI (Message Passing Interface) provides so called “derived datatypes” to overcome that problem, and for many systems these derived datatypes have been implemented efficiently. However, when running MPI on a cluster of systems in wide-area networks, such optimized implementations are not yet available and the overhead for communicating them may be substantial. The purpose of this paper is to show how this problem can be overcome by considering both the nature of the derived datatype and the cluster of systems used. We present an optimized implementation and show some results for clusters of supercomputers.
Computations from many scientific and engineering domains use irregularmeshes and/or sparse matrices. The codes expressing these computations involve irregular reductions. The main characteristics of irregular reduction loops are 1) elements of left-hand-side arrays may be incremented in multiple iterations of the loop, but only using associative and commutative operations (these arrays are called reduction arrays), 2) there are no loop carried dependencies, except on elements of reduction arrays, and 3) one or more arrays are accessed using indirection arrays. It is very challenging to efficiently parallelize codes involving irregular reductions, especially on large parallel machines. Because of accesses through indirection arrays, communication and locality are hard to manage. Not only is the total communication volume large, but the communication requirements typically cannot be determined at compile time. It is also hard to efficiently allocate space for non-local elements. Particularly, there are no effective solutions for parallelization of adaptive irregular reductions. In an adaptive irregular reduction, the elements of the indirection arrays are modified after every few iterations. This significantly increases the overhead associated with partitioning and runtime preprocessing routines [7,11], which have been critical for achieving locality, communication efficiency, and effective buffer management. Recently, there has been much interest in multithreaded architectures. A multiprocessor based upon a multithreaded architecture supports multiple threads of execution on each processor. These architectures also support low-cost thread initiation, low-overhead communication, and efficient communication and synchronization between threads on different processors. Multithreaded architectures are considered a promising medium for scalable parallelization of irregular applications, where the frequent communication and synchronization make parallelization hard on conventional parallel machines. We have developed an execution strategy for irregular reductions on a multithreaded architecture. The key idea in our execution model is that the frequency and volume of communication is independent of the contents of the indirection arrays. Thus, unlike other approaches to scalable parallelization of irregular reductions, our approach does not require mesh partitioning [1], array renumbering Though the frequency and volume of communication is independent of the data distribution, other factors are not.
In this work, we study on distributed memory architecture, the parallelization of the continuous global optimization problem, based on interval arithmetic, with inequality constraints. Since this algorithm is dynamic and irregular, we propose, in particular, some techniques taking into account the load balancing problem.
In this paper, we propose a parallel I/O system utilizing parallel commodity network attached to multiple I/O processors of parallel processing systems. I/O requests from user application are automatically distributed to parallel I/O channels to achieve the best utilization of parallel network, and the system provides highly scalable performance according to the number of channels. We also provides an API for easy user-level programming and a set of utilities for high-speed data transfer and real-time parallelized visualization based on it.
Future high energy physics experiments will require huge distributed computational infrastructures, called data grids, to satisfy their data processing and analysis needs. This paper records the current understanding of the demands that will be put on a data grid around 2006, by the hundreds of physicists working with data from the CMS experiment. The current understanding is recorded by defining a model of this CMS physics analysis application running on a ‘virtual data grid’ as proposed by the GriPhyN project. The complete model consists of a hardware model, a data model, and an application workload model. The main utility of the HEPGRID2001 model is that it encodes high energy physics (HEP) application domain knowledge and makes it available in a form that is understandable for the CS community, so that architectural and performance requirements for data grid middleware components can be derived.
This paper presents the recent development of the environment of on-line tools for parallel programming support, based on a universal monitoring system, the OCM, which is built in compliance with the OMIS specification. Issues covered include enhancements needed both at the monitoring level and at the user interface level in order to achieve full tool support for message-passing parallel applications, and to enable interoperability of tools. We focus on the evolution of the environment towards support for performance analysis of MPI applications, and interoperability of two tools: the PATOP performance analyzer and the DETOP debugger. We also outline perspectives for further research to extend the environment’s capabilities to support other parallel programming paradigms.
We define a grid middleware comprising federated resources that facilitates a globally optimal mapping of applications to the available resources while satisfying the goals of both users and resource providers. Applications are annotated with performance and behavioural information to enable the ‘best’ resources to be found automatically. A computational currency is used by resource providers and consumers to express their goals (e.g. completion time, resource utilisation, etc.) enabling a globally optimal mapping of applications to resources. We describe a prototype implementation of this architecture using Java and Jini.
Simulation-visualization complexes combine tools for numerical simulation and data visual representation. They facilitate together the research process of investigating a phenomenon by decreasing necessary time and cost resources. The paper is devoted to the process of design and development of such kind of complexes. It introduces the approach that permits to combine simulation and visualization compounds together and represents at the same time a minimum of user discomfort related with the increasing functionality of a final complex.
Several studies have considered reducing instruction cache misses and branch penalty stall cycles by means of various forms of code placement. Most proposed approaches rearrange procedures or basic blocks in order to speed up execution on sequential architectures with branch prediction. Moreover, most works focus mainly on instruction cache performance and disregard execution cycles. To the best of our knowledge, no work has specifically addressed statically scheduled ILP machines like VLIWs, with control-transfer delay slots. We propose a new code positioning algorithm especially designed for VLIW-style architectures, which allows to trade off tighter schedule for program locality. Our measurements indicate that code positioning, as a result of tighter program schedule and removed unconditional jumps, can significantly reduce the number of execution cycles, by up to 21%, while improving program locality and instruction cache performance.
Network resource planning aims to determine cost-effective resources needed to carry current volumes of traffic. Determining where and when to add bandwidth or expand routing or switching capacity are the most fundamental resource planning decisions. This paper describes this problem in converged networks and an model-based approach as an efficient solution for it. Computational tests of the approach with different backbones ware carried
Large scale irregular applications involve data arrays and other data structures that are too large to fit in main memory and hence reside on disks; such applications are called out-of-core applications. We present a design and implementation of lip, a new parallel library that is intended to efficiently support parallel execution of irregular out-of-core codes on parallel systems. We discuss the lip structure and application interface, as well as an application that has been implemented using that interface. The library has been implemented on top of MPI/MPI-IO and, therefore, is fully portable.We introduce preliminary performance results from a template CFD codes to demonstrate the efficacy of the presented techniques.
STEM-II is an Eulerian numerical model to simulate the behavior of pollutant factors in the air. In this paper the computational requirements of the program in terms of memory storage and execution times are analyzed. The results of this analysis are conclusive as regards to the need of using parallel processing to achieve reasonable execution times. Then, the improvements achieved after the parallelization of the code on a distributed memory multiprocessor using the MPI standard message passing library are shown.
Caper is a parallel programming language, which supports declarative parallel computations and control of all architectures by Flynn [1]. Caper has a self-organization and asynchronous events processing programming means. Represented language has various variables with different scope, time of creation and survival time. Besides, Caper has so called “controlled variables” or variables with statuses, which allow to regulate usage of variables by different parallel processes. Caper based on virtual machines system, including own parallel virtual machine. This property allows to create programs not depended on multitasking management of operating systems.
This paper presents an experience in use of a parallel mathematical library, ScaLAPACK, on a network composed by heterogeneous workstations. The good performance results have been obtained by means of a distributed programming environment which is able to dynamically evaluate available computing power at each workstation and to distribute accordingly the set of parallel processes.
This paper gives an overview over different techniques necessary for the realisation of distributed image retrieval systems by considering a running prototype Cairo as an example. Cairo uses the performance capabilities of a cluster architecture for the realisation of a high-quality, object-based image analysis and search. Integrated scheduling algorithms balance the workload over the available nodes and minimise the communication within the cluster.
The access to distributed high performance computing facilities for execution of Java programs has generated considerable interest. A metacomputing system, or metasystem, allows uniform access to heterogeneous resources. Our case study is suma,a metasystem defined as a set of corba components,offerin g services for execution of both sequential and parallel applications. This document describes the most important aspects of suma design in terms of corba services. We present some experimental results related to execution overhead in a campus-wide environment.
In this paper, we show a Monte Carlo linear solver with chain reduction and optimization, coupled with PLFG, a parallel pseudo-random generator. PLFG, designed for MIMD architectures, is highly scalable and with the default parameters chosen, it provides an astronomical period of at least 229 (223209 - 1). Numerical experiment results show that Monte Carlo method with chain optimization and reduction gives much better estimates of the solution vector.
The practice of speculation in resolving data dependences based on value prediction has been studied as a means of extracting more instruction level parallelism. There are many studies on value prediction mechanisms with high predictabilities. However, to the best of our knowledge, the influence of compiler optimizations on value prediction has not been investigated. In this paper we evaluate efficiency of value prediction on several binaries which are compiled with different optimization levels. Detailed simulations reveal that value prediction is still effective for highly optimized binaries.
In this paper, we study the parallelization of the one-sided Jacobi method for computing the eigenvalues and the eigenvectors of a real and symmetric matrix. We use a technique to overlap the communications by the computations in order to decrease the global communication time. We also extend the obtained results to the block version for using the level-3 BLAS.
This paper presents a new technique that enhances the process and the methodology used in a performance prediction analysis. An automatic dynamic instrumentation methodology is added to Warwick's Performance Analysis and Characterization Environment PACE [1]. The automation process has eliminated the need to manually obtain application information and data. The Dynamic instrumentation has given PACE the ability to extract and utilize data that were hidden and unobtainable prior to execution. We give two examples to illustrate our methodology. While it was impossible to perform the analysis using the original method due to lack of essential information, the new technique successfully enabled PACE to conduct the prediction analysis in a dynamic environment. The results show that with the automated dynamic instrumentation, the performance prediction analysis of dynamic application execution is possible and the results obtained are reliable. We believe that the technique implemented here could eventually be used in other performance prediction tool-sets, and therefore enhance the ways in which the performance of systems and applications is analysed and predicted.
