Due to a hard dependency between time steps, large-scale simulations of gas using the Direct Simulation Monte Carlo (DSMC) method proceed at the pace of the slowest processor. Scalability is therefore achievable only by ensuring that the work done each time step is as evenly apportioned among the processors as possible. Furthermore, as the simulated system evolves, the load shifts, and thus this load-balancing typically needs to be performed multiple times over the course of a simulation. Common methods generally use either crude performance models or processor-level timers. We combine both to create a timer-augmented cost function which both converges quickly and yields well-balanced processor decompositions. When compared to a particle-based performance model alone, our method achieves 2x speedup at steady-state on up to 1024 processors for a test case consisting of a Mach 9 argon jet impacting a solid wall.
A buffer k-d tree is a k-d tree variant for massively-parallel nearest neighbor search. While providing valuable speed-ups on modern many-core devices in case both a large number of reference and query points are given, buffer k-d trees are limited by the amount of points that can fit on a single device. In this work, we show how to modify the original data structure and the associated workflow to make the overall approach capable of dealing with massive data sets. We further provide a simple yet efficient way of using multiple devices given in a single workstation. The applicability of the modified framework is demonstrated in the context of astronomy, a field that is faced with huge amounts of data.
This paper presents LRMalloc, a lock-free memory allocator that leverages lessons of modern memory allocators and combines them with a lock-free scheme. Current state-of-the-art memory allocators possess good performance but lack desirable lock-free properties, such as, priority inversion tolerance, kill-tolerance availability, and/or deadlock and livelock immunity. LRMalloc’s purpose is to show the feasibility of lock-free memory management algorithms, without sacrificing competitiveness in comparison to commonly used state-of-the-art memory allocators, especially for concurrent multithreaded applications.
Concurrent execution of tasks in GPUs can reduce the computation time of a workload by overlapping data transfer and execution commands. However it is difficult to implement an efficient runtime scheduler that minimizes the workload makespan as many execution orderings should be evaluated. In this paper, we employ scheduling theory to build a model that takes into account the device capabilities, workload characteristics, constraints and objective functions. In our model, GPU tasks scheduling is reformulated as a flow shop scheduling problem, which allow us to apply and compare well known methods already developed in the operations research field. In addition we develop a new heuristic, specifically focused on executing GPU commands, that achieves better scheduling results than previous techniques. Finally, a comprehensive evaluation, showing the suitability and robustness of this new approach, is conducted in three different NVIDIA architectures (Kepler, Maxwell and Pascal).
Performance prediction of applications has always been a great challenge, even for homogeneous architectures. However, today’s trend is the design of cluster running in a heterogeneous architecture, which increases the complexity of new strategies to predict the behavior and time spent by an application to run. In this paper we present a strategy that predicts the performance of an application on different architectures and rank then according to the performance that the application can achieve on each architecture. The proposed strategy was able to correctly rank three of four applications tested without overhead implications. Our next step is to extend the metrics in order to increase the accuracy.
Iterative linear algebra methods are the important parts of the overall computing time of applications in various fields since decades. Recent research related to social networking, big data, machine learning and artificial intelligence has increased the necessity for non-hermitian solvers associated with much larger sparse matrices and graphs. The analysis of the iterative method behaviors for such problems is complex, and it is necessary to evaluate their convergence to solve extremely large non-Hermitian eigenvalue and linear problems on parallel and/or distributed machines. This convergence depends on the properties of spectra. Then, it is necessary to generate large matrices with known spectra to benchmark the methods. These matrices should be non-Hermitian and non-trivial, with very high dimension. This paper highlights a scalable matrix generator that uses the user-defined spectrum to construct large-scale sparse matrices and to ensure their eigenvalues as the given ones with high accuracy. This generator is implemented on CPUs and multi-GPU platforms. Good strong and weak scaling performance is obtained on several supercomputers. We also propose a method to verify its ability to guarantee the given spectra.
The problem of Protein Structure Prediction (PSP) is one of the major challenges in computational biology, which has attracted the interest of many researchers in several areas. However, determining the three dimensional structure is an expensive and a time-consuming taks, even considering the current advances made in High-Performance Computing (HPC) systems. This paper presents a systematic mapping on HPC for PSP problems, aiming to determine: protein energy models applied to solve PSP problems with HPC, parallel programming models considered in such HPC solutions, and the heuristics and metaheuristics implemented with HPC for PSP. We considered 5732 papers published until January 2018 and, from this set, a total of 72 studies were selected for this paper. Our results show an increasing interest to apply HPC for this hard computing problem, using MPI as programming model and clusters as infrastructure to solve PSP problems. Some hybrid programming models with OpenMP and CUDA can also be found. There is a recurrent use of genetic algorithms to optimize PSP problems and the full atom model was the widest energy representation applied.
We evaluate the performance of two well-known Deep Learning frameworks – Caffe and TensorFlow – on two different types of computing devices – GPU and NUMA CPU architecture – using two popular network models as benchmark – AlexNet and GoogLeNet. We variate batch sizes between trainings and estimate the average training time per iteration and per image on each configuration. Both frameworks presented similar times for the AlexNet model, and TensorFlow outperforms Caffe by presenting times up to 2 times lower than Caffe for the GoogLeNet Model. The work also presents the impact of lack of support by the frameworks for NUMA Architectures, and relates a problem stated on loss computation by the Caffe Framework.
This is the companion data for the VECPAR2018 submission paper entitled: Design, Implementation and Performance Analysis of a CFD task-based Application for Heterogeneous CPU/GPU Resources by Lucas Leandro Nesi, Lucas Mello Schnorr, and Philippe Olivier Alexandre Navaux. All the data, source code, and images generation scripts used in the paper are present here.
Often unstructured grid methods, such as finite elements, are used in high fidelity simulations. In these methods, solution accuracy is associated to the interpolation order and to the grid size. Unstructured parallel mesh refinement is computationally expensive due to subdomains interface communication. In the present work we develop a uniform edge-based parallel tetrahedral mesh refinement scheme completely free of communication, fast, simple to implement and highly scalable. This is achieved by an index generation for subdomain interface grid points based on special pairing functions.
This paper presents a new benchmark to evaluate performance and energy consumption of different Parallel Programming Interfaces (PPIs). The benchmark is composed of 11 algorithms implemented in PThreads, OpenMP, MPI-1 and MPI-2 (spawn) PPIs. Previous studies have used some of these applications to perform this type of evaluation in different architectures, since there is no benchmark that offers this variety of PPIs and communication models. In this work we measure the energy and performance of each application in a single architecture, varying the number of threads/processes. The goal is to show that this set of applications has enough features to form a parallel benchmark. The results show that there is no single best case that provides both better performance and low energy consumption in the presented scenarios. However, PThreads and OpenMP achieve the best trade-offs between performance and energy in most cases.
CUDA Dynamic Parallelism (CDP) is an extension of the GPGPU programming model proposed to better address irregular applications and recursive patterns of computation. However, processing memory demanding problems by using CDP is not straightforward, because of its particular memory organization. This work presents an algorithm to deal with such an issue. It dynamically calculates and configures the CDP runtime variables and the GPU heap on the basis of an analysis of the partial backtracking tree. The proposed algorithm was implemented for solving permutation combinatorial problems and experimented on two test-cases: N-Queens and the Asymmetric Travelling Salesman Problem. The proposed algorithm allows different CDP-based backtracking from the literature to solve memory demanding problems, adaptively with respect to the number of recursive kernel generations and the presence of dynamic allocations on GPU.
Scientific workflows are being used today in a number of areas. As they grow in complexity and importance, cloud computing emerges as an important execution environment. In this scenario, scheduling the workflow tasks and data on the cloud ensuring proper use of the computational resources is one of the key issues in the management of workflow execution. Although many workflow schedulers have been proposed in the literature, few of them deal with heterogeneous computing resources and data file assignment. The Hybrid Evolutionary Algorithm to Task Scheduling and Data File Assignment Problem (HEA-TaSDAP) addresses these two problems simultaneously, but the scheduling is time consuming, especially if we consider large scale workflows. In this work, we propose optimizations on HEA-TaSDAP by taking advantage of the massive parallelism provided by GPUs, leveraging the scheduling of larger instances in a reasonable amount of time. Our parallel solution provided about 98.83% of reductions in the scheduling time, keeping the quality of the solutions.
This paper concentrates on an OpenMP implementation of the TVD–Hopmoc method with executions performed on Intel® Many Integrated Core and Xeon® Scalable Processor architectures. Specifically, this paper evaluates an improved OpenMP implementation of the TVD–Hopmoc method based on a cluster of points when applied to the convection–diffusion equation in 1–D. Aiming at avoiding fine-grained parallelism employed in a basic OpenMP implementation of the TVD–Hopmoc method, this approach groups variables (located at stencil points) to be calculated simultaneously in parallel instead of calculating them individually. Numerical experiments performed on Intel® Many Integrated Core and Scalable Processor architectures show that the improved OpenMP implementation of the TVD–Hopmoc method based on a cluster of points provides further worthwhile gains when compared both with our previous implementation based only on parallel chunk loops and a basic OpenMP implementation of this method.
High Performance Computing relies on accelerators (such as GPGPUs) to achieve fast execution of scientific applications. Traditionally these accelerators have been programmed with specialized languages, such as CUDA or OpenCL. In recent years, OpenMP emerged as a promising alternative for supporting accelerators, providing advantages such as maintaining a single code base for the host and different accelerator types and providing a simple way to extend support for accelerators to existing code. Efficiently using this support requires solving several challenges, related to performance, work partitioning, and concurrent execution on multiple device types. In this paper, we discuss these challenges and introduce a library, HybridOMP, that addresses several of them, thus enabling the effective use of OpenMP for accelerators. We apply HybridOMP to a scientific application, PlasCom2, that has not previously been able to use accelerators. Experiments on three architectures show that HybridOMP results in performance gains of up to 10x compared to CPU-only execution. Concurrent execution on the host and GPU resulted in additional gains of up to 10% compared to running on the GPU only.
Exact string matching, searching for all occurrences of given pattern P on a text T, is a fundamental issue in computer science with many applications in natural language processing, speech processing, computational biology, information retrieval, intrusion detection systems, data compression, and etc. Speeding up the pattern matching operations benefiting from the SIMD parallelism has received attention in the recent literature, where the empirical results on previous studies revealed that SIMD parallelism significantly helps, while the performance may even be expected to get automatically enhanced with the ever increasing size of the SIMD registers. In this paper, we provide variants of the previously proposed EPSM and SSEF algorithms, which are originally implemented on Intel SSE4.2 (Streaming SIMD Extensions 4.2 version with 128-bit registers). We tune the new algorithms according to Intel AVX2 platform (Advanced Vector Extensions 2 with 256-bit registers) and analyze the gain in performance with respect to the increasing length of the SIMD registers. Profiling the new algorithms by using the Intel Vtune Amplifier for detecting performance bottlenecks led us to consider the cache friendliness and shared-memory access issues in the AVX2 platform. We applied cache optimization techniques to overcome the problems particularly addressing the search algorithms based on filtering. Experimental comparison of the new solutions with the previously known-to-be-fast algorithms on small, medium, and large alphabet text files with diverse pattern lengths showed that the algorithms on AVX2 platform optimized cache obliviously outperforms the previous solutions.
The use of heterogeneous architectures has become indispensable in optimizing application performance. Nowadays, one of the most popular heterogeneous architectures is discrete CPU+GPU. Despite the high computational power present in such architectures, in many cases, memory data transfers between CPU and GPU are significant performance bottlenecks. As an attempt to mitigate performance costs involved in data transfers, chipmakers started to integrate CPU and GPU cores in the same fabric sharing the same main memory but with different memory address spaces in architectures denominated APUs (Accelerated Processing Unit). To efficiently exploit heterogeneous CPU+GPU architectures it is needed to split the data so that both processing units (PUs) can perform the computations in parallel. Although this approach results in significant performance improvements, some applications can also be functionality split, as is the case of the Lattice-Boltzmann Method (LBM). In this work, we evaluate the performance of each kernel resulting from the functional decomposition of an OpenCL Lattice-Boltzmann method implementation using non-uniform domain decomposition between CPU and GPU on an APU to better understand the performance impact of different non-uniform domain decompositions between CPU and GPU on each kernel. The experimental results performed on an AMD APU A10-7870K show that uniform domain decompositions between each kernel on the same PU but non-uniform domain decompositions between CPU and GPU affect each kernel differently. These results suggest that non-uniform domain decompositions between each kernel on the same PU and not only between the different PUs can improve even more the performance of the application.
Accelerated Processing Units (APUs) are an emerging architecture that integrates, in a single silicon chip, the traditional CPU and the GPU. Due to its heterogeneous architecture, APUs impose new challenges to data parallel applications that want to take advantage of all the processing units available on the hardware to minimize its execution time. Some standards help in the task of writing parallel code for heterogeneous devices, but it is not easy to find the data division between CPU and GPU that will minimize the execution time. In this context, this work further extends and details load balancing algorithms designed to be used in a data parallel problem. Also, a sensitivity analysis of the parameters used in our models was performed. The results have shown that the algorithms are effective in their purpose of improving the performance of an application on an heterogeneous environment.
On future large-scale systems, the mean time between failures (MTBF) of the system is expected to decrease so that many faults could occur during the solution of large problems. Consequently, it becomes critical to design parallel numerical linear algebra kernels that can survive faults. In that framework, we investigate the relevance of approaches relying on numerical techniques, which might be combined with more classical techniques for real large-scale parallel implementations. Our main objective is to provide robust resilient schemes so that the solver may keep converging in the presence of the hard fault without restarting the calculation from scratch. For this purpose, we study interpolation-restart (IR) strategies. For a given numerical scheme, the IR strategies consist of extracting relevant information from available data after a fault. After data extraction, a well-selected part of the missing data is regenerated through interpolation strategies to constitute a meaningful input to restart the numerical algorithm. In this paper, we revisit a few state-of-the-art methods in numerical linear algebra in the light of our IR strategies. Through a few numerical experiments, we illustrate the respective robustness of the resulting resilient schemes with respect to the MTBF via qualitative illustrations.
The smoothed aggregation algebraic multigrid (SA-AMG) method is among the fastest solvers for large-scale linear equations. It achieves good convergence by generating small matrices from the original matrix problem. However, the convergence of the method can be further improved by using near-kernel vectors. Our research investigates the effectiveness of using multiple near-kernel vectors and finds the near-kernel vectors that are most important for obtaining rapid convergence. We apply our method to the three-dimensional problem in elasticity. The known near-kernel vectors (the parallel translation and rotation vectors) improve the convergence and execution time of the SA-AMG method. We use an iterative process known as the V-cycle to extract multiple near-kernel vectors. In numerical experiments, we show that a suitable choice of the near-kernel vectors reduces the number of iterations by up to two-thirds and halves the execution time, compared to use of the known near-kernel vectors. Our method will be effective for cases in which the same matrix problem is solved repeatedly.
