RED-SEA is a H2020 EuroHPC project, whose main objective is to prepare a new-generation European Interconnect, capable of powering the EU Exascale systems to come, through an economically viable and technologically efficient interconnect, leveraging European interconnect technology (BXI) associated with standard and mature technology (Ethernet), previous EU-funded initiatives, as well as open standards and compatible APIs.To achieve this objective, the RED-SEA project is being carried out around four key pillars: (i) network architecture and workload requirements-interconnects co-design – aiming at optimizing the fit with the other EuroHPC projects and with the EPI processors; (ii) development of a high-performance, low-latency, seamless bridge with Ethernet; (iii) efficient network resource management, including congestion and Quality-of-Service; and (iv) end-to-end functions implemented at the network edges.This paper presents key achievements and results at the midterm of the project for each key pillar in the way to reach the final project objective. In this regard we can highlight: (i) The definition of the network requirements and architecture as well as a list of benchmarks and applications; (ii) In addition to initially planned IPs progress, BXI3 architecture has evolved to support natively Ethernet at low level, resulting in reduced complexity, with advantages in terms of cost optimization, and power consumption; (iii) The congestion characterization of target applications and proposals to reduce this congestion by the optimization of collective communication primitives, injection throttling and adaptive routing; and (iv) the low-latency high-message rate endpoint functions and their connection with new open technologies.

In order to enable Exascale computing, next generation interconnection networks must scale to hundreds of thousands of nodes, and must provide features to also allow the HPC, HPDA, and AI applications to reach Exascale, while benefiting from new hardware and software trends. RED-SEA will pave the way to the next generation of European Exascale interconnects, including the next generation of BXI, as follows: (i) specify the new architecture using hardware-software co-design and a set of applications representative of the new terrain of converging HPC, HPDA, and AI; (ii) test, evaluate, and/or implement the new architectural features at multiple levels, according to the nature of each of them, ranging from mathematical analysis and modeling, to simulation, or to emulation or implementation on FPGA testbeds; (iii) enable seamless communication within and between resource clusters, and therefore development of a high-performance low latency gateway, bridging seamlessly with Ethernet; (iv) add efficient network resource management, thus improving congestion resiliency, virtualization, adaptive routing, collective operations; (v) open the interconnect to new kinds of applications and hardware, with enhancements for end-to-end network services - from programming models to reliability, security, low- latency, and new processors; (vi) leverage open standards and compatible APIs to develop innovative reusable libraries and Fabrics management solutions.

Heading towards exascale, the challenges for process management with respect to flexibility and efficiency grow accordingly. Running more than one application simultaneously on a node can be the solution for better resource utilization. However, this approach of co-scheduling can also be the way to go for gaining a degree of flexibility with respect to process management that can enable some kind of interactivity even in the domain of high-performance computing. This chapter gives an introduction into such co-scheduling policies for running multiple MPI sessions concurrently and interactively within a single user allocation. The chapter initially introduces a taxonomy for classifying the different characteristics of such a flexible process management, and discusses actual manifestations thereof during the course of the reading. In doing so, real world examples are motivated and presented by means of ParaStation MPI, a high-performance MPI library supplemented by a complete framework comprising a scalable and dynamic process manager. In particular, four scheduling policies, implemented in ParaStation MPI, are detailed and evaluated by applying a benchmarking tool that has especially been developed for measuring interactivity and dynamicity metrics of job schedulers and process managers for high-performance computing.

In this paper we describe the design of fault tolerance capabilities for general-purpose offload semantics, based on the OmpSs programming model. Using ParaStation MPI, a production MPI-3.1 implementation, we explore the features that, being standard compliant, an MPI stack must support to provide the necessary fault tolerance guarantees, based on MPI's dynamic process management. Our results, including synthetic benchmarks and applications, reveal low runtime overhead and efficient recovery, demonstrating that the existing MPI standard provided us with sufficient mechanisms to implement an effective and efficient fault-tolerant solution.

Heading towards exascale, the challenges for process management with respect to flexibility and efficiency grow accordingly. Running more than one application simultaneously on a node can be the solution for better resource utilization. However, this approach of co-scheduling can also be the way to go for gaining a degree of flexibility with respect to process management that can enable some kind of interactivity even in the domain of high-performance computing. This chapter gives an introduction into such co-scheduling policies for running multiple MPI sessions concurrently and interactively within a single user allocation. The chapter initially introduces a taxonomy for classifying the different characteristics of such a flexible process management, and discusses actual manifestations thereof during the course of the reading. In doing so, real world examples are motivated and presented by means of ParaStation MPI, a high-performance MPI library supplemented by a complete framework comprising a scalable and dynamic process manager. In particular, four scheduling policies, implemented in ParaStation MPI, are detailed and evaluated by applying a benchmarking tool that has especially been developed for measuring interactivity and dynamicity metrics of job schedulers and process managers for high-performance computing.

Load balancing, maintenance, and energy efficiency are key challenges for upcoming supercomputers. An indispensable tool for the accomplishment of these tasks is the ability to migrate applications during runtime. Especially in HPC, where any performance hit is frowned upon, such migration mechanisms have to come with minimal overhead. This constraint is usually not met by current practice adding further abstraction layers to the software stack. In this paper, we propose a concept for the migration of MPI processes communicating over OS-bypass networks such as InfiniBand. While being transparent to the application, our solution minimizes the runtime overhead by introducing a protocol for the shutdown of individual connections prior to the migration. It is implemented on the basis of an MPI library and evaluated using virtual machines based on KVM. Our evaluation reveals that the runtime overhead is negligible small. The migration time itself is mainly determined by the particular migration mechanism, whereas the additional execution time of the presented protocol converges to 2ms per connection if more than a few dozen connections are shut down at a time.

SummaryHomogeneous cluster architectures, which used to dominate high‐performance computing (HPC), are challenged today by heterogeneous approaches utilizing accelerator or co‐processor devices. The DEEP (Dynamical Exascale Entry Platform) project is implementing a novel architecture for HPC, in which a standard HPC cluster is directly connected to a so‐called ‘Booster’: a cluster of many‐core processors. By these means heterogeneity is organized differently as in today's standard approach, where accelerators are added to each node of the cluster. In order to adapt application codes to this Cluster‐Booster architecture as seamless as possible, DEEP has developed a complete programming environment. It integrates the offloading functionality given by the Message Passing Interface standard with an abstraction layer based on the task‐based OmpSs programming paradigm. This paper presents the DEEP project with an emphasis on the DEEP programming environment. Copyright © 2015 John Wiley & Sons, Ltd.

Homogeneous cluster architectures dominating high-performance computing (HPC) today are challenged, in particular when thinking about reaching Exascale by the end of the decade, by heterogeneous approaches utilizing accelerator elements. The DEEP (Dynamical Exascale Entry Platform) project aims for implementing a novel architecture for high-performance computing consisting of two components - a standard HPC Cluster and a cluster of many-core processors called Booster. In order to make the adaptation of application codes to this Cluster-Booster architecture as seamless as possible, DEEP provides a complete programming environment. It integrates the offloading functionality given by the MPI standard with an abstraction layer based on the task-based OmpSs programming paradigm. This paper presents the DEEP project with an emphasis on the DEEP programming environment.

In distributed Java environments, locality of objects and threads is crucial for the performance of parallel applications. We introduce dynamic locality optimizations in the context of JavaParty, a programming and runtime environment for parallel Java applications. Until now, an optimal distribution of the individual objects of an application has to be found manually, which has several drawbacks. Based on a former static approach, we develop a dynamic methodology for automatic locality optimizations. By measuring processing and communication times of remote method calls at runtime, a placement strategy can be computed that maps each object of the distributed system to its optimal virtual machine. Objects then are migrated between the processing nodes in order to realize this placement strategy. We evaluate our approach by comparing the performance of two benchmark applications with manually distributed versions. It is shown that our approach is particularly suitable for dynamic applications where the optimal object distribution varies at runtime.

Concurrent programming is getting more and more important. Managing concurrency requires the usage of synchronization mechanisms, which is error-prone. Well-known examples for synchronization defects are deadlocks and race conditions. Detecting such errors is known to be difficult. There are several approaches to identify potential errors, but they either produce a high number of false positives or suffer from high computational overhead, catching only a small number of defects. Our approach uses static analysis techniques combined with points-to and may-happen-in-parallel (MHP) information to reduce the number of false positives. Additionally, we present code patterns indicating possible synchronization problems. We have implemented our approach using the Java framework Soot. Our tool was tested with small code examples, an open source web server, and commercial software. First results show that the number of false positives is reduced significantly.

This paper presents the design and implementation of the MPI-IO interface for the Clusterfile parallel file system. The approach offers the opportunity of achieving a high correlation between the file access patterns of parallel applications and the physical file distribution. First, any physical file distribution can be expressed by means of MPI data types. Second, mechanisms such as views and collective I/O operations are portably implemented inside the file system, unifying the I/O scheduling strategies of the MPI-IO library and the file system. The experimental section demonstrates performance benefits of more than one order of magnitude.

Today's cluster computers suffer from slow I/O, which slows down I/O-intensive applications. We show that fast disk I/O can be achieved by operating a parallel file system over fast networks such as Myrinet or Gigabit Ethernet. In this paper, we demonstrate how the ParaStation3 communication system helps speed-up the performance of parallel I/O on clusters using the open source parallel virtual file system (PVFS) as testbed and production system. We will describe the set-up of PVFS on the Alpha-Linux-Cluster-Engine (ALiCE) located at Wuppertal University, Germany. Benchmarks on ALiCE achieve write-performances of up to 1 GB/s from a 32-processor compute-partition to a 32-processor PVFS I/O-partition, outperforming known benchmark results for PVFS on the same network by more than a factor of 2. Read-performance from buffer-cache reaches up to 2.2 GB/s. Our benchmarks are giant, I/O-intensive eigenmode problems from lattice quantum chromodynamics, demonstrating stability and performance of PVFS over Parastation in large-scale production runs.

Remote method invocation in Java RMI allows the flow of control to pass across local Java threads and thereby span multiple virtual machines. However, the resulting distributed threads do not strictly follow the paradigm of their local Java counterparts for at least three reasons. Firstly, the absence of a global thread identity causes problems when reentering monitors. Secondly, blocks synchronized on remote objects do not work properly. Thirdly, the thread interruption mechanism for threads executing a remote call is broken. These problems make multi-threaded distributed programming complicated and error prone. We present a two-level solution: On the library level, we extend KaRMI (Philippsen et al. (2000)), a fast replacement for RMI, with global thread identities for eliminating problems with monitor reentry. Problem with synchronization on remote objects are solved with a facility for remote monitor acquisition. Our interrupt forwarding mechanism enables the application to get full control over its distributed threads. On the language level, we integrate these extensions with JavaParty's transparent remote objects (Philippsen et al. (1997)) to get transparent distributed threads. We finally evaluate our approach with benchmarks that show costs and benefits of our overall design.

We have demonstrated that the ParaStation3 communication system speeds up the performance of parallel I/O on cluster computers such as ALiCE. I/O benchmarks with PVFS using Parastation over Myrinet achieve a throughput for write operations of up to 1 GB/s from a 32-processor compute partition, given a 32-processor PVFS I/O partition. These results outperform known benchmark results for PVFS on 1.28 Gbit Myrinet by more than a factor of 2, a fact that is mainly due to the superior communication features of ParaStation. Read performance from buffer cache reaches up to 2.2 GB/s, while reading from hard disk saturates at the cumulative hard disk performance. The I/O performance achieved with PVFS using ParaStation enables us to carry out extremely data-intensive eigenmode computations on ALiCE in the application field of lattice quantum chromodynamics. For the future, we plan to utilize the I/O system for storing and processing mass data in high energy physics data analysis on clusters.