This paper presents a novel adaptive machine learning (ML) framework for predicting computational tasks execution time in OpenCL environments, specifically designed to address load balancing challenges within Central Processing Unit (CPU) and Graphics Processing Unit (GPU) Heterogeneous Computing Systems (HCS). Finding a universal solution for load balancing is complex given the diverse hardware configurations and computational tasks, ranging from cybersecurity to ML workloads. The focus on CPU-GPU systems enables our solution to effectively navigate the specific demands of modern heterogeneous computing environments. The framework proposed in the paper implements a regression-based deep learning network to analyze OpenCL kernels and monitor the computing environment's state. The ML model, which is a part of the proposed framework, excels at predicting kernel execution times across various devices, enabling dynamic and responsive load balancing. The method underlying the framework operation differs from traditional static code analyzer–based approaches, which specify execution devices without considering critical runtime features extracted through kernel profiling. The framework enhances performance through dynamic selection of devices based on system status real-time monitoring. The proposed solution, integrated with a comprehensive profiling system, analyzes memory access patterns, data transfer rates, and others. The regression-based task modeling approach provides precise performance measurements, avoiding the limitations of classification methods that require redundant task executions. Empirical evaluation demonstrates substantial improvements achieving up to 9.8% enhancement in load balancing efficiency during operational fine-tuning.
Cloud is a distributed computing system that offers resources and services depending on the need. The primary concerns with managing cloud resources will be task scheduling and resource provisioning in a cloud environment. Computational complexity and the interdependency of scientific procedures restrict the use of cloud resource management. Balanced load is accomplished by emphasis on workflow detection and scheduling based on utilization percentage. A novel two stage framework is designed to reduce the number of jobs in the workflow scheduling jobs that cannot meet their deadlines. This paper proposes a new approach called Hybridized Fennec Rat Swarm Optimization (HFe-RSO), which assigns priority to the tasks. Then, a workflow scheduling model, Enhanced Soft Actor-Critic Deep Reinforcement Learning (ESAC-DRL), is developed for efficient resource management by considering the task priority and utilization rate. Workflow benchmarks such as Montage, CyberShake, Sipht, Epigenomics, and Inspiral are used to assess the performance of the proposed model. The performance of the proposed approach is analysed using cost, makespan, and utilization rate, and compared with state-of-the-art approaches. By using the proposed approach, the cost of $259.11, makespan of 99.84ms, and the utilization rate 96.20%, was achieved. The experimental results demonstrate the significance of the proposed approach over other existing approaches.
Planning unmanned aerial vehicles (UAVs) by considering the optimization concerns about the enemy threats, fuel consumption or usage and maneuver limitations has a critical importance on the task being performed and safety of the flight. The Back-and-Forth (BaF) algorithm has been introduced recently as a solving technique for the geometrical description of the UAV path planning problem and its sole performance and positive contribution on the capabilities of another methods have been validated. In this study, the greedy heuristic guided by the BaF algorithm was redesigned with the purpose of getting the power of a multi-core architecture and a new geometrical UAV path planner named as the Combined Paths for short ComPaths was proposed. The performance of the ComPaths was evaluated over 3 different battlefield scenarios and 12 test cases derived from them. Moreover, the paths found by the ComPaths were compared to the paths of other 15 heuristic or meta-heuristic backed planners. Comparative studies between the ComPaths and other tested techniques allowed to conclude that the ComPaths is among the top three best planners for all cases even though it makes nearly 43 to 49 times less function calls than the remaining competitors.
The arrival of heterogeneous (or hybrid) multicore architectures has brought new performance trade-offs for applications, and efficiency opportunities to systems. They have also increased the challenges related to thread scheduling, as tasks’ execution times will vary depending if they are placed on big (performance) cores or little (efficient) ones. In this paper, we focus on the challenges heterogeneous multicore processors bring to partially-replicable task chains, such as the ones that implement digital communication standards in Software-Defined Radio (SDR). Our objective is to maximize the throughput of these task chains while also minimizing their power consumption. We model this problem as a pipelined workflow scheduling problem using pipelined and replicated parallelism on two types of resources whose objectives are to minimize the period and to use as many little cores as necessary. We propose two greedy heuristics (FERTAC and 2CATAC) and one optimal dynamic programming (HeRAD) solution to the problem. We evaluate our solutions and compare the quality of their schedules (in period and resource utilization) and their execution times using synthetic task chains. We also study an open source implementation of the DVB-S2 communication standard based on the StreamPU runtime. Leading processor vendors are covered with ARM, Apple, AMD, and Intel platforms. Both the achieved throughput and the energy consumption are evaluated. Our results demonstrate the benefits and drawbacks of the different proposed solutions. On average, FERTAC and 2CATAC achieve near-optimal solutions, with periods that are less than 10% worse than the optimal (HeRAD) using fewer than 2 extra cores. These three scheduling strategies now enable programmers and users of StreamPU to transparently make use of heterogeneous multicore processors and achieve a throughput that differs from its theoretical maximum by less than 6% on average. On the DVB-S2 receiver, it is also shown that the heterogeneous solutions outperform the best homogeneous ones in terms of energy efficiency by 8% on average.
The MPI standard provides the foundational building blocks for most parallel applications running on large-scale HPC systems. Collective communication operations in MPI are critical components for the scalability of these applications. Most MPI libraries offer several algorithms for each specific collective operation, and each library selects the algorithm to be used based on the number of processes, the message size, and possibly other factors. Each collective algorithm may perform better in certain scenarios, and thus, selecting the most suitable algorithm for each use case is essential. However, even the best algorithm in a given MPI library may deliver suboptimal performance.Self-consistent MPI performance guidelines capture semantic relationships between different collective operations and exploit these to express performance expectations that collectives should reasonably satisfy to be considered performance-consistent. For collective communication, such performance guidelines typically state that a specialized collective call should not be slower than less specialized counterparts.In this article, we demonstrate how the consistency of MPI libraries with respect to performance guidelines can be analyzed. For this purpose, we present a tool that checks guideline compliance. For regular collective operations such as MPI_Bcast, the tool contains multiple emulated versions of the collective by composing less specialized operations. Then, for a specific number of processes and message sizes, the tool experimentally assesses whether the algorithm selected by the MPI library is slower than its emulated counterparts. If that is the case, a performance-guideline violation is detected. In a broader empirical study, we assess the current state of performance consistency in MPI libraries on modern supercomputers.
High-performance computing (HPC) applications are increasingly executed in heterogeneous environments, introducing new challenges for programming and software portability. SYCL has emerged as a leading model designed to simplify heterogeneous programming and make it more accessible to developers. Intended as a single-source, cross-platform parallel programming framework, SYCL promises portability, productivity, and performance across a variety of architectures.However, these goals have not been consistently defined or realized, leaving developers with varying expectations. This paper addresses this gap by evaluating SYCL from the perspective of application developers. We analyze whether SYCL meets essential criteria for cross-platform development, including code portability, development productivity, and runtime efficiency.Our evaluation draws on benchmarks and illustrative examples and focuses on SYCL’s memory management and parallelism abstractions. We provide detailed comparisons between Unified Shared Memory (USM) and buffer-accessor approaches, as well as between NDRange and hierarchical kernel models. In addition to presenting our own benchmark results on Intel platforms, we synthesize findings from recent studies across multiple SYCL implementations and compilers.Our results expose key limitations and inconsistencies in current SYCL implementations and offer insights into the steps needed to improve the framework’s reliability and cross-platform usability.
Recursion, as a common programming paradigm, is widely applied in numerous applications. By treating recursive problems as tasks, the recursion process generates many independent subtasks, which reveals the potential for parallelism. To harness this parallelism on heterogeneous CPU-GPU systems, this paper introduces HRPF (Heterogeneous Recursive Parallel Programming Framework). HRPF provides a set of programming interfaces to define recursive algorithms, shielding users from the complexities of task allocation, scheduling, and data movement. This facilitates the efficient and straightforward implementation of parallel recursive programs on CPU-GPU systems. HRPF dispatches tasks between CPU and GPU workers by combining depth-first search (DFS) and breadth-first search (BFS) strategies. It adopts a hybrid work-stealing scheduling algorithm incorporating both work-first and help-first policies to achieve dynamic load balancing. The HRPF runtime system ensures data consistency between the host and the device and overlaps computation with data transfer. Additionally, HRPF provides a set of parallel loop programming interfaces. To evaluate HRPF, we implement several benchmarks including merge sort, quicksort, Strassen-Winograd matrix multiplication, and parallel loops in four commonly used algorithms. Experimental results on a CPU-GPU platform demonstrate that HRPF achieves superior performance across a range of benchmarks compared to OpenMP, StarPU and Taskflow.
Sparse generalized matrix multiplication (SpGEMM) has been widely applied to sparse neural network models. However, the arbitrary distribution of non-zero elements in sparse matrices leads to the processing elements (PEs) in the systolic array (SA) architecture being idle and further affecting computing efficiency. Reviewing existing methods, we found three main drawbacks to calculating SpGEMM in multi-core SAs. First, the sparse matrix calculation format is unsuitable for the SA architecture. Second, when the SA calculates SpGEMM, the load is unbalanced among PEs. Third, the computational load is unevenly distributed across different SA cores during the process above. To address the above problems, we proposed a load-balancing SpGEMM accelerating framework for multi-core SAs. First, we introduced the SCSR sparse matrix compression format and the PE fast sparse matrix matching and calculation method in SA. Second, we present a runtime dynamic data flow packaging algorithm, GrePack. Third, we propose a compile-time sparse data flow multi-core static partitioning algorithm. Compared with the state-of-the-art work, our dynamic packaging algorithm accelerates SpGEMM speed by up to 2.08x, our static multi-core partitioning method improves the computing unit utilization by up to 1.54x, and our collaborative inference framework improves SpGEMM speed by up to 2.29x.
Three big semiconductor companies in HPC are currently competing in the race for the best CPU: AMD, Intel, and NVIDIA. There are significant differences among their state-of-the-art CPU designs, spanning the entire range from instruction execution to cache behavior and main memory bandwidth. In this work, we analyze the performance of CPUs based on the Zen 4, Golden Cove, and Neoverse V2 microarchitectures. We create accurate in-core performance models for use with the Open Source Architecture Code Analyzer (OSACA) tool and compare its prediction accuracy with llvm-mca. Beyond the tool aspect, this reveals interesting differences in in-core design points but also some commonalities. Beyond the single core, we extend our comparison by measuring data-transfer behavior through the memory hierarchy using a variety of microbenchmarks. We thoroughly investigate the “write-allocate (WA) evasion” feature, which can automatically reduce the memory traffic caused by write misses. We show that the Grace Superchip has a next-to-optimal implementation of WA evasion while the Sapphire Rapids CPU can avoid write allocates completely only in specific scenarios. The only way to eliminate WAs on AMD Genoa is the explicit use of non-temporal stores. Finally, we study the cache hierarchy of the CPUs in view of the Execution-Cache-Memory (ECM) performance model, revealing overlapping cache hierarchies on Genoa and Grace in contrast to Sapphire Rapids.
Source code-level auto-tuning enables applications to adapt their implementation to maintain peak performance under varying execution environments (i. e.hardware, input, or application settings). However, the performance of the auto-tuned code is inherently tied to the design of the tuning space (the space of possible changes to the code). An ideal tuning space must include configurations diverse enough to ensure high performance across all targeted environments while simultaneously eliminating redundant or inefficient regions that slow the tuning space search process. Traditional research has focused primarily on identifying optimization opportunities in the code and on efficient tuning space search. However, there is no rigorous methodology or tool supporting analysis and refinement of the tuning spaces, allowing for the addition of configurations that perform well in an unseen environment or the removal of configurations that perform poorly in any realistic environment. In this short communication, we argue that hardware performance counters should be used to analyze tuning spaces, and that such an analysis would allow programmers to refine the tuning spaces by adding configurations that unlock additional performance in unseen environments and removing those unlikely to produce efficient code in any realistic environment. While our primary goal is to introduce this research question and foster discussion, we also present a preliminary methodology for tuning-space analysis. We validate our approach through a case study using a GPU implementation of an N-body simulation. Our results demonstrate that the proposed analysis can detect the weaknesses of a tuning space: based on its outcomes, we refined the tuning space, improving the average configuration performance 3.3x, and the best-performing configuration by 2-18%.
The article presents the results of several studies to assess the impact of the configuration of NUMA (Non-Uniform Memory Access) nodes on the performance of GPU-accelerated applications in hybrid computing system with shared memory. Using Crossroads/N9 DGEMM (NVBLAS library) as a model application, the performance in various NUMA modes with one or more GPUs was analyzed, and the throughput of the memory subsystem and data transfer channels between the host memory and graphics processors was also measured. The impact of coprocessor distribution across NUMA nodes on the efficiency of the model application was also examined.Results showed that configuration of NUMA nodes can have a significant impact on the performance of applications that offload calculations to graphics coprocessors in a hybrid computing system with shared memory, and this impact could have an effect in different ways. For example, using one NUMA node for the entire computing system is the least optimal approach in terms of memory bandwidth, but it provided the highest bandwidth for communication between host memory and coprocessors during active data transfer to several accelerators. Thus, this mode achieves maximum performance when performing calculations on multiple GPUs that actively exchange data through host memory. Other modes showed advantages in different situations. Overall, to achieve maximum performance during active data transfer to coprocessors, they should be part of one NUMA node. These results will help to develop approaches to configuration of hybrid computing systems on processors with a chiplet layout, and help to improve the performance of software that offloads calculations to graphics accelerators with the Ampere architecture, such as NVIDIA A800 and NVIDIA A100, which are currently widely represented in the high-performance computing industry.
Hardware specialization is a promising direction in the post-Moore era, particularly for high-performance computing (HPC). In this work, we present a lightweight prototyping example of hardware specialization using open-source tools. Focusing on the Monte Carlo cross-section lookup kernel, a computation with low resource utilization on general-purpose architectures, we implement a custom hardware pipeline in Chisel and generate Verilog for resource usage estimation. We explore hardware optimization techniques that trade off throughput and resource usage, and show that, as SRAM scaling stalls and memory dominates chip area, using additional logic, even in brute-force forms, can lead to better overall efficiency. Our estimation demonstrates significant performance gain over general-purpose CPUs. While this is a case study, the methodology provides a practical path for quick feasibility studies in hardware specialization.
Caffe, one of the most popular deep learning frameworks, trains models by reading training data from the storage engine, LevelDB, and feeding it into the computation engine. This paper analyzes the challenges faced by data reading in Caffe Training: (1) Fetch, Parse, and Transform-the three steps of reading each image-are serial, and each image is read sequentially; (2) Frequent disk I/O-each image read triggers an I/O operation- significantly increases the data reading time; (3) Caffe calls LevelDB's range query method to read training data, but this leads to unnecessary pointer comparison operations, wasting CPU resources; (4) Since LevelDB reads training data in key order during range queries, the fixed order of training data across epochs may cause overfitting and lower the model's test accuracy. Based on these challenges, this paper proposes Parallel Read, Out-of-Order Optimization, and Adaptive Design strategies to design a new I/O layer, PROAD, for Caffe that systematically reconstructs and optimizes LevelDB's original data reading mechanism, thus improving LevelDB I/O performance for Caffe Training. The Parallel Read method pipelines the Fetch, Parse, and Transform steps and accelerates reading via large block reads; Out-of-Order Optimization discards the range scan feature of LevelDB, allowing Caffe to read training data in a random manner during training, avoiding the original key comparison overhead and providing a boost to model accuracy; while the Adaptive Design method supports efficient reading of training data with different resolutions. Based on these designs, this paper implements PROAD and deploys it in Caffe for performance evaluation. Experimental results show that Caffe with PROAD significantly improves data reading performance during training, especially for high-resolution datasets, where data reading time in Caffe with PROAD is reduced by 14%-42% compared to Caffe with LevelDB and 6%-34% compared to Caffe with LMDB. Furthermore, Caffe with PROAD improves model test accuracy due to the Out-of-Order Optimization strategy, while consuming relatively reasonable memory resources.
One of the novel features of the Fujitsu A64FX CPU is the sector cache. This feature enables hardware-supported partitioning of the L1 and L2 caches and allows the programmer control of which partition is used to place data in. This paper performs an in-depth study of applying the sector cache to sparse matrix-vector multiplication (SpMV) in the Compressed Sparse Row (CSR) format using a collection of 490 sparse matrices. A performance model based on reuse analysis is used to better understand situations in which and how the sector cache leads to improved cache reuse and to predict cache behavior. The model predicts the number of L2 cache misses within an error of 2% without cache partitioning. With sector cache enabled, depending on the configuration, the model predicts the number of L2 cache missed within 2-3% and 4-18% for sequential and parallel SpMV with 48 threads, respectively. Further experiments show the effect of various sector cache configurations on performance. A median speedup of about 1.05x is achieved, whereas the maximum speedup is about 1.6x.
This article presents a comparative analysis of two storage approaches: a SAN disk array, exemplified by an HPE 3PAR device, and a software-defined storage cluster constructed with the Ceph software. The objective of this comparison is to ascertain whether a software-defined storage cluster built with commodity servers can achieve comparable performance to a SAN disk array with a similar hardware configuration. The configuration used identical numbers of components of matching speeds, capacities, and hardware generation from the same manufacturer. By relaxing some requirements on the software-defined storage, we were able to benchmark all RAID levels with corresponding replication and erasure code settings. The results revealed that 3PAR performed 31 times better for 4 KiB data block writes than Ceph. On the contrary, the Ceph cluster surpassed 3PAR by a factor of 1.4 in 16 MiB large-block reads. The differences are explained in the text based on the theory of operation of the two types of storage. We propose criteria for choosing the correct type of technology for individual use cases.
The utilization of machine learning (ML) in architectural design shows great potential, especially in addressing the challenges posed by complex design spaces where traditional approaches may fall short. Network-on-chip (NoC) architecture has emerged as an efficient solution for on-chip communication among processors. However, with the increasing device scaling and component density, the likelihood of processor failures also rises, making fault-tolerant design a critical aspect of chip development to ensure system reliability. In this paper, we present a novel ML framework for fault-tolerant core mapping that effectively overcomes issues encountered in previous methodologies, such as re-transmission and re-mapping. The proposed framework intelligently learns optimal core mapping strategies and effectively addresses fault tolerance concerns in NoCs with diverse application core graphs. The approach begins with efficient NoC mapping and scheduling as the primary step. In the event of any faults during this process, an error detection and correction mechanism is applied within the NoC itself, eliminating the need for time-consuming re-transmissions. Furthermore, if faults persist even after error correction, the tasks assigned to the failed core are seamlessly migrated to a designated spare core, ensuring continuous system operation. Comparisons with conventional methods demonstrate considerable improvements in processor speed-up, energy efficiency, as well as reductions in re-transmission, latency, and dynamic power consumption. Hardware results indicate enhanced performance, reduced area, and lower power consumption compared to related algorithms when implemented on an FPGA board. The proposed technique showcases significant advancements in fault-tolerant core mapping for NoCs, thereby enhancing overall chip reliability and performance.
Recent advances in machine learning have led to increasingly large and complex models, placing significant demands on computation and memory. Techniques such as Butterfly factorization have emerged to reduce model parameters and memory footprints while preserving accuracy. Specialized hardware accelerators, such as Graphcore's Intelligence Processing Units (IPUs), are designed to address these challenges through massive parallelism and efficient on-chip memory utilization. In this paper, we extend our analysis of Butterfly structures for efficient utilization on single and multiple IPUs, comparing their performance with GPUs. These structures drastically reduce the number of parameters and memory footprint while preserving model accuracy. Experimental results on the Graphcore GC200 IPU chip, compared with an NVIDIA A30 GPU, demonstrate a 98.5% compression ratio, with speedups of 1.6x and 1.3x for Butterfly and Pixelated Butterfly structures, respectively. Extending our evaluation to Vision Transformer (ViT) models, we compare Multi-GPU and Multi-IPU systems on the M2000 machine: Multi-GPU reaches a maximum accuracy of 84.51% with a training time of 401.44 min, whereas Multi-IPU attains a higher maximum accuracy of 88.92% with a training time of 694.03 min. These results demonstrate that Butterfly factorization enables substantial compression of ViT layers (up to 97.17%) while improving model accuracy. The findings highlight the promise of IPU machines as a suitable platform for large-scale machine learning model training, especially when coupled with sparsification methods like Butterfly factorization, thanks to their efficient support for model parallelism.
ShyLU-node is an open-source software that implements linear solvers and preconditioners on shared-memory multicore CPUs or on a GPU. ShyLU-node can be used as a stand-alone global linear solver, as a local subdomain solver for a domain decomposition (DD) preconditioner, or as a coarse linear solver in an algebraic multigrid preconditioner. It is part of the Trilinos framework and is designed to play a role in computing robust and efficient solutions of large-scale linear systems from real-world applications on current and emerging computer architectures. In this paper, we discuss the improvements and the current status of two sparse direct solvers, Basker and Tacho, and an algebraic preconditioner, FastILU, within ShyLU-node. To demonstrate the portability of our solvers, we present the performance of the sparse direct solvers for real application problems, namely, Basker for Xyce Circuit Simulations on Intel Xeon CPUs and Tacho for the Albany Land-Ice Simulation of Antarctica on the Perlmutter Supercomputer with NVIDIA A100 GPUs. FastILU has also been used in real-world applications, but in this paper, we illustrate its performance using 3D model problems on an NVIDIA H100 GPU and on an AMD MI250 GPU.
