This paper presents the design and implementation of a multi-GPU concurrent queue system using NVIDIA's NVSHMEM. The Bellman-Ford algorithm is used as a case study to evaluate the performance of the proposed concurrent FIFO queue, with this multi-GPU implementation being the first known instance of its kind. Experimental results demonstrate that the multi-GPU queue implementation achieves a maximum speedup of 3.92x and an average speedup of 3.04x over the singleGPU baseline on four NVIDIA A100 GPUs. When applied to the Bellman-Ford Single-Source Shortest Path (SSSP) algorithm, the multi-GPU system achieves a maximum speedup of 3.03x and an average speedup of 2.65x compared to the single-GPU implementation, tested on 10 graphs of different kinds taken from the SuiteSparse Matrix Collection.
We present a solution to unify various High Performance Computing (HPC) resources and enable environmentally sustainable (“green”), automated job assignment for independent HPC systems. The goal is to increase utilization of carbon-neutral computing resources and reduce the time to solution with simple job management using a job specification file and command line interface modelled after established batch systems. Our workload-agnostic approach is motivated by a comprehensive overview of existing software, which reveals the lack of a solution for executing arbitrary jobs for multiple users across multiple targets with low technical requirements and complexity. The proposed solution is transparent to HPC administrators through SSH-based instrumentation and to users with regard to the utilized batch system. We demonstrate the capabilities of our prototype in a case study of Molecular Dynamics (MD) simulations. The individual jobs are allocated over four different target systems and the scheduling policy takes into account the availability of resources which can be powered exclusively by Renewable Energy (RE). This results in a reduction of Carbon Emissions (CEs) by at least 9% compared to running all jobs on a conventional HPC system. In the future, we plan to extend our scheduling policy to exploit job execution timing using grid carbon intensity over time and investigate additional simulation parameters to minimize the economical and ecological impact of HPC jobs.
Asymmetric multicore processors such as Intel Arrow Lake, integrate performance (P) and efficiency (E) cores within the same die, offering new opportunities for flexible execution. However, these architectures also introduce additional challenges for energy-efficient operation, as balancing power and performance across such heterogeneous cores of these architectures is nontrivial. In this work, we first derive nearoptimal energy-efficient regions for three classes of bottleneckoriented microbenchmarks. We then investigate to what extent microbenchmark-derived insights translate to real applications. For this purpose, we characterize applications using the Roofline Model and additional performance metrics, to identify their compute-, bandwidth-, or latency-bound behavior. Based on this classification, we derive simple guidelines for selecting core type, operating frequency, and core number. We evaluate these guidelines using representative workloads from SPEC CPU 2017 and GAP graph benchmark suite. Our results show that microbenchmark-based insights can provide practical guidance for energy-efficient execution while also highlighting limitations when applied to more complex applications.
The exponential growth of data-intensive workloads in high-performance computing has turned traditional shared parallel file systems into a performance bottleneck. To address this, ad-hoc parallel file systems have emerged as a highperformance alternative, taking advantage of local node resources to provide ephemeral, low-latency storage. However, their temporary nature introduces a critical reliability challenge, since any node failure or job termination results in the total loss of the ad-hoc data. In addition, traditional checkpointing is ineffective here, as it usually focuses on application memory and does not account for potential data loss within the ad-hoc file system itself. This work addresses this limitation by presenting the integration of a transparent checkpointing workflow with the Expand Ad-Hoc parallel file system. The proposed solution employs the MPI-Agnostic Network-Agnostic (MANA) framework, providing a unified solution that saves both application memory and ad-hoc storage data. By leveraging the fault tolerance of Expand AdHoc to store checkpoint images, the system ensures data integrity against node failures. Additionally, an intelligent mechanism selectively captures only modified files, effectively reducing I/O redundancy and overhead for read-only files. This solution was evaluated on the C3 supercomputer using the IOR benchmark and the Quantum ESPRESSO, a real-world scientific application. The experimental results demonstrate that our integrated solution reduces checkpointing time by 44% to 83% compared to traditional backend storage systems, as illustrated in these experiments using Lustre. In scenarios with frequent checkpointing intervals, the implementation of Expand Ad-Hoc results in a substantial reduction in total execution time, demonstrating its effectiveness as a high-performance, transparent resilience solution for large-scale computing.
The Nvidia Grace Hopper Superchip has seen a wide adoption across both HPC sites and AI data centers. This hybrid architecture places the Grace CPU and Hopper GPU on the same board, with a shared and adjustable power budget. We present an end-to-end analysis of its energy efficiency, from a verification of power sensor accuracy using microbenchmarks, an investigation of power knob effects to an energy-efficiency and TCO investigation for selected Large Language Model (LLM) inference use cases. Our study shows that the internal power sensors provide precise readings when compared to an external power meter, unlike contemporary x86 processors. Some power budget configurations are not enforced by the system, the power draw can exceed the configured limit. This behavior is not documented by Nvidia. We analyzed the energy efficiency of LLM inference for prefill and decoding-dominated workloads, with the latter having emerged as the driving factor. Increasing the module power limit while maintaining a constant GPU power did not affect the system's energy efficiency. According to our TCO analysis, no cost savings can be expected from reduced power limits for output lengths of up to 2048 tokens for our LLM setup.
An independent set of an undirected graph is a subset of vertices where no two vertices included in the set are neighbors. A Maximal Independent Set (MIS) of a graph is an independent set such that no additional vertices can be added to the set without violating the requirements of an independent set. To support modern large graphs, parallel MIS algorithms and their multi-core, GPU, and distributed memory implementations have attracted widespread research interests in recent years. We present a distributed MIS solution to support applications that demand high throughput and scalability. Our solution is designed to support various randomized priority assignment techniques and to compute MIS in a fully asynchronous or round-synchronous manner, the latter is required by some priority assignment techniques. Our MPI-based prototype attains strong scaling by way of multilevel load-balanced graph partitioning, accelerates distributed processing through computationcommunication overlap and message prioritization, and produces deterministic results. We demonstrate scalability using massive datasets: e.g., a real-world web graph with 128 billion edges and 512 compute nodes (16 K CPU-cores), and competitiveness of our solution through empirical comparisons: our implementation comfortably outperforms both CombBLAS-MIS and AGM-MIS, and identifies MIS' of size comparable to those of the key parallel and distributed solutions found in the literature.
The progress of non-blocking two-sided communication remains one of the key challenges in MPI applications. Mainstream MPI implementations such as MPICH and Intel MPI can be configured to use internal asynchronous progress threads. In contrast, Open MPI does not provide a general-purpose MPIinternal progress thread; instead, communication progress is primarily driven when the application explicitly invokes MPI test routines or through transport-specific offloading on RDMAcapable network interface cards (R-NICs). Although both threadbased progress and network offloading can improve communication progress and, consequently, communication-computation overlap, they may also degrade MPI process performance when the number of outstanding communication requests becomes large. In this study, we introduce a new approach that dedicates a single CPU core to manage communication progress of nonblocking two-sided communication in MPI applications while also performing actual data transfers, either across the network or within shared-memory regions. Our results demonstrate near-perfect overlap between computation and communication, achieving reliable and energy-efficient data transfers in sharedmemory environments and outperforming Intel MPI, even with its progress thread enabled. All experiments were conducted on a 32-node cluster equipped with high-performance Intel Sapphire Rapids processors.
Particle-in-cell (PIC) codes enable high-performance simulations of plasma dynamics, providing accurate modeling of electron and ion interactions in realistic configuration of fusion devices. Such codes have very long execution times even if they are parallelized using MPI so the need to optimize them is critical. Vector processors can improve performance and parallel execution for many types of applications. This paper studies BIT1, a representative PIC code, to assess the benefit from Vector/SIMD architectures using automatic and manual vectorization. We provide an extensive evaluation study on x86 and RISC-V platforms and improve BIT1 code to enable better vectorization and increase its performance. Our findings show that a portable solution (with minor code updates) enhances the performance using the out-of-the-box compiler's auto-vectorization capabilities on x86 by 76 % and on RISC-V by $2 \times$ using auto or manual vectorization with 2 MPI ranks.
The performance analysis (benchmarking) of the MPI runtime environment is a mandatory step after commissioning a new supercomputer. When doing so, the performance of the MPI library in charge, which forms the communication substrate and the progress engine for the parallel applications, can generally be optimized to suit the specific hardware through various parameters (tuning for the platform). A new approach, pursued in the context of the IFCES2 project, is to complement these optimizations with adjustments to the MPI library based on the performance characteristics of the applications used by specific research communities (tuning for the code). In this paper, we will present our findings from IFCES2 in this regard, and we will show how especially adjustments to the critical path within the MPI library can be used to achieve performance improvements for the target applications. For this, the ICON Earth System Model serves as our target application, for which we have optimized the ParaStation MPI library.
This work investigates the use of SIMD for the computation of three-body interactions in molecular dynamics. Our main focus are non-additive potentials in the form of the Axilrod-Teller-Muto (ATM). Given the high computational load required to calculate these interactions, the use of vectorization is of high relevance. Literature on the vectorization of the ATM potential is limited. In this paper, we propose two different techniques for the SIMD parallelization of these calculations, one based on register broadcast and another on permutation. These techniques are implemented using AVX2 intrinsic functions. Since in SIMD memory read/write operation management is critical to achieve a high performance, our work also explores the use of the two commonly found memory layouts in HPC applications, i.e. array-of-structures (AoS) and structure-of-arrays (SoA). These proposed vectorization techniques were tested at the node level, for which our simulations were implemented with shared-memory parallelization based on OpenMP. Our approach allows us to test the proposed parallelization schemes without the complications of more intricate molecule container implementations. With this purpose, the effect on the SIMD performance of other optimization aspects often used in molecular dynamics are described, such as the implementation of Newton's third law of motion. We present results that describe the achieved speedup and runtime gains for each technique. These were carried out for up to 64 cores in a single node and they illustrate the interplay between our proposed approaches, the use of shared-memory parallelization, and the described memory layouts. We were able to show a runtime speedup of up to $2.6 \times$ and up to $12 \times$ increase in performance with respect to the scalar calculations, without loss of performance at high core counts. A conclusion wraps up our studies.
This paper introduces the new FlatterScatter algorithm as part of mallocMC, a heterogeneous, configurable C++ framework for performant and portable dynamic memory allocation on highly multithreaded hardware such as GPUs. Configurability is achieved by a policy-based design enabling choice and configuration of the core algorithm and other behavior with a single line of code. Performance is achieved by shipping with state-of-the-art allocation algorithms including our new FlatterScatter algorithm improving its inspiration ScatterAlloc algorithm in terms of applicability, stability and maintainability while retaining performance. This paper assesses performance using a synthetic benchmark written in the CUDA language used to evaluate other allocators. It is also evaluated in the exascale particle-in-cell simulation code PIConGPU wherein mallocMC excels over its competitors. Platform independence is achieved by building on alpaka (abstraction library for parallel kernel acceleration). This enables running on NVIDIA, AMD and Intel accelerators as well as on CPU without changing a single line of code. We also evaluate performance on AMD hardware, a capability unique to the open-source library mallocMC.
Serverless platforms are increasingly used across edge-oriented environments, where heterogeneous resources must serve time-varying function workloads while balancing latency and energy. In this context, placement decisions directly affect both performance and resource consumption. Yet, comparing scheduling policies remains difficult because existing studies often rely on different workloads, platforms, and evaluation procedures. This paper presents a comparative experimental study of serverless placement policies, including heuristic baselines, a greedy heuristic, a contextual bandit scheduler based on LinUCB, and a deep reinforcement learning scheduler based on Deep QNetworks (DQN). The evaluation is conducted on the Grid'5000 testbed under shared workloads and platform conditions. We compare the policies under steady workloads, workload changes, and varying cluster sizes, and assess them through latency, energy per request, online overhead, failure rate, and learning-related cost. The results show that DQN achieves the strongest overall latency and energy performance across the evaluated workloads, while the greedy heuristic remains consistently competitive without incurring offline training cost. LinUCB improves selected metrics but does not match DQN or Greedy uniformly across conditions. The study also shows that scheduler comparisons cannot rely solely on latency, as energy, failure behavior, and learning-related overhead materially affect the interpretation of results.
Different approaches have already been proposed to automatically tune properties of compute-intensive applications such as runtime behavior, energy consumption or throughput. However, existing autotuning solutions typically require modifications to the application source code, for example, by inserting specific API calls. This work presents an alternative autotuning approach based on debugger interfaces that enables online tuning of binary programs by observing the program behavior during successive occurrences of a breakpoint in the tuning loop. The approach has been implemented as a proof of concept in the ONACADI online tuner. A case study that optimizes the cache blocking of a thread-parallel solver for systems of ordinary differential equations demonstrates the functionality of the approach, with an overhead per iteration on the order of milliseconds.
Pipeline parallelism is a key technique for distributed training of large language models because it reduces per-device parameter and activation memory. However, comparing pipeline schedules is difficult: analytical models expose structural quantities such as bubble ratios, while end-to-end hardware experiments are costly and system-specific. In this work, we introduce a tabular schedule abstraction and a unified multi-abstraction methodology that connects formula-based reasoning, idealized schedule tables, and communication-aware execution simulation. Using this framework, we compare GPipe, 1F1B, Chimera, and Hanayo in its restricted regime across multiple modeled system configurations. Our results show that schedule rankings are not abstraction-invariant: communication can negate structural advantages suggested by bubble analysis alone. Under the assumptions considered here, GPipe and 1F1B are runtime-equivalent, but 1F1B achieves a lower activation-memory peak. Chimera is advantageous mainly at low microbatch counts and in communication-favorable regimes, while Hanayo is effective in its intended restricted operating point but remains sensitive to network bottlenecks. We further study an asymmetric Chimera-style placement, which does not reduce the global peak memory requirement but reveals limited runtime gains in shallow pipelines. Overall, pipeline schedule quality is meaningful only in the context of the modeled execution environment.
While detailed resource usage monitoring is possible on the low-level using proper tools, associating such usage with higher-level abstractions in the application layer that actually cause the resource usage in the first place presents a number of challenges. Suppose a large-scale scientific data analysis workflow is run using a distributed execution environment such as a compute cluster or cloud environment and we want to analyze the I/O behaviour of it to find and alleviate potential bottlenecks. Different tasks of the workflow can be assigned to arbitrary compute nodes and may even share the same compute nodes. Thus, locally observed resource usage is not directly associated with the individual workflow tasks. By acquiring resource usage profiles of the involved nodes, we seek to correlate the trace data to the workflow and its individual tasks. To accomplish that, we select the proper set of metadata associated with low-level traces that let us associate them with higher-level task information obtained from log files of the workflow execution as well as the job management using a task orchestrator such as Kubernetes with its container management. Ensuring a proper information chain allows the classification of observed I/O on a logical task level and may reveal the most costly or inefficient tasks of a scientific workflow that are most promising for optimization.
In this paper, we address the Byzantine Agreement problem in synchronous systems where Byzantine agents can move from process to process, corrupting their host. We focus on the model defined by Bonnet et al. where a cured process may send messages (based on a state corrupted by the malicious agent), however it will behave correctly in the way it sends those messages: i.e., send messages according to the algorithm. Interestingly, in this model it has been proven that at least 5 t+1 processors are needed in order to tolerate t mobile Byzantine agents. Our contribution is twofold: first, we prove that while integrating trusted counters into the Bonnet et al. model, at least 4 t+1 processors are needed in order to tolerate t mobile Byzantine agents. Furthermore, we propose a novel algorithm that solves the Mobile Byzantine Agreement problem in Bonnet et al.’s model enhanced with trusted counters and prove that the algorithm matches the lower bound.
Modern in-vehicle networks are increasingly vulnerable to cyberattacks due to the lack of built-in security in protocols such as the Controller Area Network (CAN). This paper introduces bitrate hopping, a proactive intrusion prevention technique that defends against message injection, denial-of-service (DoS), and passive eavesdropping. Bitrate hopping periodically changes the CAN bitrate among synchronized, trusted ECUs using a cryptographically secure pseudorandom sequence, preventing unauthorized nodes from communicating. Unlike cryptographic authentication mechanisms that often require a change in the CAN packet structure or add overhead that may result in violation of real-time constraints in some applications, bitrate hopping operates at the physical layer with minimal overhead. We first implement and evaluate a CAN-only version of bitrate hopping to maintain full backward compatibility with existing networks, demonstrating that it can be effective in short bursts to disrupt attacks without requiring additional hardware. Building on these results, we develop a more robust Ethernet-based implementation, which maintains consistently high message success rates at faster hop intervals and represents our primary focus for future system development. Our contributions include the design of a synchronization protocol for bitrate transitions, a dual-mode implementation over CAN and Ethernet, and an empirical study validating its feasibility and effectiveness.
In recent years, machine learning models such as deep neural networks (DNN) and convolutional neural networks (CNN) have been widely adopted across various applications. Image processing, in particular, has seen rapid advancement with the development of high-accuracy models such as the Vision Transformer (ViT). Performing inference using image processing on the edge devices is important from several aspects such as communication latency, preserving privacy and ensuring realtime response. However, in the edge environment where highperformance computing resources such as the cloud cannot be utilized, the inference time tends to be long, causing delays in processing tasks such as anomaly detection using video or image data. To address this issue, we propose a method that reduces its latency applying stepwise distributed executions that consists of simplified multiple CNNs and the optimized ViT. We show its performance through experiments on image classification tasks using edge devices.
This paper presents $\mu$ xer, a deep learning architecture designed for cross-domain image classification in both healthcare and eco-health applications. The model leverages split mixer blocks and a hybrid SiReLU activation function to achieve efficient spatial and channel-wise feature extraction. This will not affect the maintenance of computational efficiency. The $\mu$ xer is tailored for edge AI deployment and addresses critical challenges such as domain shifts, limited labeled data, and inference constraints. It integrates multitask learning and CycleGAN-based domain adaptation. This enables a robust generalization across heterogeneous datasets without extensive fine-tuning. Experiments conducted on five binary classification tasks (malaria detection, plant disease recognition, biomedical textures, colorectal polyps, and cervical cells) demonstrate $\mu$ xer’s superior performance compared to ZF Net, ViT-base-patch16, and MobileNet in terms of accuracy, sensitivity, specificity, and latency. The results confirm that $\mu$ xer delivers high diagnostic precision with low resource consumption. This makes it a scalable solution for real-time medical and environmental monitoring on low-power devices.
Vehicle-to-Vehicle (V2V) communication is critical for intelligent transportation systems (ITS), requiring secure, efficient, and privacy-preserving solutions for real-time collision detection. This study proposes a novel framework integrating Federated Learning (FL) with Blockchain, leveraging a deep neural network (DNN) with 64-32-1 layers and dropout for collision prediction, enhanced by parallel and distributed computing. Using the Federated Averaging algorithm for distributed training and a permissioned Blockchain with Practical Byzantine Fault Tolerance (PBFT) consensus, the framework was evaluated on a simulated environment with three vehicles over 10 rounds, running on a 4 -core Intel i5 CPU with 8 GB RAM. The global model achieves an average accuracy of 75.42 % (improving from 70.00 % to 77.33%) and a loss reduction from 0.5874 to 0.5404, with latency decreasing from 2.82 seconds to 0.85 seconds (average 1.04 seconds)—outperforming a centralized baseline (2.50 s latency, 82.00% accuracy). Blockchain enhances security, maintaining 73.00% accuracy under a hypothetical attack, and preserves privacy by keeping data local. However, a recall of 33.89% indicates challenges in detecting all collisions, and performance variability is limited by synthetic data and the 3-vehicle setup. The 4-core CPU constraint and potential resource contention in the Docker-based deployment highlight scaling challenges, reflecting the preliminary nature of this work. This framework advances V2V communication by offering a scalable, secure, and low-latency solution, contributing to safer autonomous driving. Future work includes real-world testing with diverse datasets, integration with 6 G and edge AI, and hardware upgrades (e.g., multi-core CPUs, increased RAM) to target >90% accuracy and optimize scalability. By harnessing parallel and distributed computing, this study sets a new benchmark for secure V2V systems, with significant potential for ITS applications.
