Cholesky factorization is a core operation in scientific computing, yet its scalability is often constrained by memory limitations when processing extremely large dense matrices. This work introduces an out-of-core Cholesky factorization algorithm for symmetric positive-definite matrices that integrates GPU acceleration, block-wise lossless compression, and parallel I/O to overcome these limitations. The approach leverages the OMPC runtime for asynchronous task scheduling and employs HDF5 to store the matrix on disk, taking advantage of Lustre's parallel I/O capabilities in distributed environments. Tiles are decompressed just-in-time on the GPU, significantly reducing host memory usage, storage footprint, and end-to-end data movement overhead-from disk through the CPU to the GPU-without compromising numerical accuracy. Experimental results show that the proposed method scales across 8 GPU nodes, successfully factorizing matrices up to 3M x 3M. In comparison, SLATE could only handle sizes up to 700K x 700K, with the proposed algorithm achieving up to 41% higher throughput. These results demonstrate the algorithm's scalability and competitiveness beyond memory-constrained in-core solutions, offering a practical path for enabling extreme-scale scientific applications.
The rapid scaling of deep learning (DL) models raises the need for accurate and understandable performance/energy prediction tools to support efficient resource management and sustainable AI development. Existing modeling approaches often lack both sufficient granularity to capture nuanced hardware-software interactions and suitable flexibility to adapt to diverse modern architectures. This paper introduces an analytical framework for time/energy prediction of DL training workloads on GPU. Our framework integrates detailed workload characterization that includes FLOPs, memory access, kernel activities, and novel structural features to derive an architecture-aware efficiency model, which considers a saturation-based function to capture dimensional scaling effects on hardware utilization. We propose an iterative refinement methodology, which incorporates model-specific scalars to address particular architectures like ALBERT and precision-specific calibrations for BF16 operations. Our benchmark with six advanced DL models (including CNNs, BERT-style Transformers, and LLMs like TinyLlama) on NVIDIA A100 GPUs under various configurations (1/4 GPUs, FP32/TF32/mixed BF16) shows that our approach achieves a high predictive accuracy, with an overall relative error of 4.14% (3.05% for time, 5.78% for power). The framework is intended to provide valuable insights for HPC-AI co-design, energy-aware scheduling, and performance optimization.
Training in supervised machine learning is based on the availability of datasets; however, medical datasets must comply with stringent privacy regulations. Generative Adversarial Networks (GANs) are a relevant alternative to solve the limitation of small medical datasets due to their ability to generate additional data with desired features. A significant drawback of these models is that they may produce unrealistic, blurred, or insufficiently diverse images. This paper proposes a data augmentation technique using GANs to create synthetic Magnetic Resonance Imaging (MRI) of four stages of Alzheimer's Disease (AD): non-demented, very mild demented, mild demented, and moderate demented. We designed a GAN based on the Pix2Pix model, which learns the features of each AD stage. Generated images are evaluated by multistage Convolutional Neural Network (CNN) models, greyscale histograms of the distribution of pixel intensities, and brain mass measurements on binarized images. The results indicate that AD synthetic MRI effectively captures disease patterns, demonstrating the potential of GANs to improve training and diagnosis of neurodegenerative diseases.
Recently, Graph Neural Networks (GNN) have been integrated into various local applications, such as local community detection and local code assistant, making edge inference increasingly important. To support diverse workloads, state-of-the-art edge devices have evolved into heterogeneous platforms, integrating components like CPU, GPU, and NPU. To this end, we propose GNX, a novel GNN system that accelerates GNN inference on edge heterogeneous platforms by leveraging all the heterogeneous processing units. Given a GNN model and a heterogeneous platform, GNX automatically generates parallel execution plans, consisting of both data and pipeline parallelism. To reduce the complexity of the design space, GNX converts GNN models into coarse-grained blocks and performs the search at the block level. By leveraging the APIs provided by state-of-the-art heterogeneous frameworks, GNX can flexibly schedule various parallel execution plans and seamlessly adjust the workload across the heterogeneous processing units for load-balanced execution. Our study shows that GNX effectively accelerates three widely-used GNN models on two state-of-the-art edge heterogeneous platforms. Compared with the baseline approach that uses only a single processing unit, GNX achieves up to a 2.57x speedup. Compared with adopting data parallelism and a state-of-the-art scheduler, GNX achieves up to 1.90x and 1.79x speedup, respectively. We also discuss the applicability of and extensions to GNX to support other GNN models.
As GPU architectures continue to diversify across high-performance computing (HPC) systems, ensuring code portability and minimizing environmental impact have become critical challenges. This paper investigates how different programming models affect the carbon-performance efficiency of a Reverse Time Migration (RTM) application, a key workload in geophysics. We provide twelve implementations of the RTM code using CUDA, HIP, Kokkos, RAJA, and OpenMP Target, and evaluate their behavior on eleven GPUs from NVIDIA and AMD. Our analysis covers execution time, energy consumption, and carbon footprint. Results show that HIP, when tuned per architectures and RAJA, achieve the highest code portability in terms of carbon-efficiency, reaching up to 93.7% and 93.4% efficiency across all platforms, respectively. Overall, while HIP and CUDA deliver peak performance and the lowest emissions when properly optimized, the gap between these native models and high-level abstractions such as RAJA and Kokkos is steadily narrowing, indicating growing potential for portable and sustainable HPC development.
Computing continuum systems are emerging as a solution for organizations to process data across diverse infrastructures, reducing latency compared to traditional cloud computing. However, managing I/O operations in such distributed and heterogeneous environments remains an open research challenge. In this paper, we present A-Flow, a model for constructing I/O systems to manage data exchange in computing continuum environments. These systems are built around data distribution patterns defined by structures called abstract communication channels (ACCs). ACCs are established between processing stages using memory, file system, and network resource connections. To prevent resource overload during execution, A-Flow automatically selects the appropriate communication channel based on user-defined criteria such as throughput or resource utilization. We implemented this model in a prototype and evaluated it through a case study focused on managing medical data in HDF5 format across different environments. The evaluation revealed that A-Flow’s ACCs can be integrated into existing stage-based systems found in the state of the art. The results highlight the efficiency and effectiveness of A-Flow in enabling dataflows across heterogeneous infrastructures, addressing key challenges in computing continuum.
Modern scientific research often relies on powerful computing systems that use graphics processing units (GPUs) to run complex applications. However, running these systems requires a large amount of energy, which contributes to carbon emissions and raises concerns about environmental impact. Given this scenario, we explore how sharing a single GPU between multiple applications can improve both performance and sustainability when running scientific workflows. We consider three execution strategies: running applications one after another, running two at the same time, and replacing finished tasks with new ones right away, using eighteen widely used scientific applications on three different GPUs (AMD MI250X, AMD RX 7900XT, and NVIDIA RTX 4090). To demonstrate that finding the best co-execution combination of applications improves resource efficiency, we use a mathematical approach based on linear programming to schedule which applications run together. Our results show that this approach can reduce total execution time by up to 47% and lower carbon emissions by as much as 34%, with minimal impact on the performance of individual applications. Additionally, when optimal combinations of parallel applications are used, the overall performance of a complete scientific workflow can improve by 36%, while carbon emissions are reduced by 25%.
Scientific workflows are evolving from relying on a monolithic storage subsystem at a single High-Performance Computing (HPC) facility to using geographically distributed file systems, repositories, and cloud storage. As a result, storing, accessing, transferring, and managing scientific data have become highly complex and prone to performance inefficiencies. This paper delves into these challenges by exploring an optimized end-to-end interface designed to seamlessly connect various local and remote storage systems, enabling efficient data movement of objects across HPC-Cloud and HPC-HPC environments. We showcase this capability through an object-focused data management runtime system, discuss the effects of relaxed consistency semantics in distributed object scenarios, and illustrate its application in an earthquake simulation workflow. Besides reducing the amount of data by selectively transferring regions of interest, our facility-local results achieved a speedup of 45x over an optimized HDF5 usage and 15x over the HDF5 with caching by using the new interface in PDC-XF.
Wave propagation is a fundamental component of seismic imaging, a technique crucial to oil and gas exploration. Traditional finite-difference (FD) methods advance the wavefield one time step at a time. While effective, these methods are memory-bound and require multiple high-end GPU nodes to achieve acceptable performance. To address this, the superstep wavefield propagation technique was introduced, grouping multiple time steps into a single large operator to increase computational intensity. However, it incurs a prohibitive memory overhead, requiring the storage of hundreds of large matrices for realistic problems. In this work, we introduce Super-Stencil, a novel symbolic formulation of superstep propagation that eliminates the need to store intermediate operators. This drastically reduces memory consumption-by about 1009x compared to the original superstep method for 20 time steps-shifting the computational bottleneck from memory to compute. Although Super-Stencil incurs up to 9.1x longer execution time than superstep and 337.3x longer than the FD baseline in a sequential setting, it unlocks a new dimension of parallelism through a Parallel-in-Time execution strategy. By enabling simultaneous time and space parallelism, Super-Stencil transforms wave propagation into a compute-bound kernel, opening the door to aggressive optimization on modern parallel architectures. This makes it a compelling alternative for next-generation seismic imaging workflows where scalability is paramount.
Sparse computations are common in practical HPC, AI and graph-based applications. Such computations often exhibit scattered and fragmented data accesses, which negatively impact data transfer efficiency to/from accelerators. We propose, implement and evaluate an algorithm for extracting or mining sparsity patterns that exist in sparse matrices. The algorithm extracts multiple pattern types in a matrix, including blocks, bands, triangles or regular compositions of each. It does so without a priori knowledge of the presence of these patterns in the matrix. The patterns may contain, under user control, zero elements, or imperfections, to facilitate the extraction of larger patterns. Additionally, we introduce the Compressed Sparse Pattern (CSP), a novel compressed representation for sparse matrices that is based on these patterns. The use of CSP combined with extensions to Address Generation Units (AGUs) of accelerators regularize data accesses and improve data transfer efficiency. Evaluation of the pattern mining algorithm and CSP using 26 real-world sparse matrices is conducted on an Ubuntu system with an 8 core Intel CPU (3.6 GHz i7-9700K) and 32 GB of memory. The evaluation shows that patterns of different sizes and shapes are common, representing similar to 82% of the non-zero elements in these matrices. The patterns can be efficiently extracted in time, with an average of 4.6 seconds. The evaluation also shows that the mining of composite patterns contributes similar to 8% to the number of non-zeros in patterns and that imperfections increase pattern sizes with a minimal impact of only similar to 7% zero elements in patterns. Finally, using CSP leads to up to 90% reduction in data transfer overhead, compared to CSR and CSC, both common compressed sparse matrix representations. These results validate our approach of extracting and representing patterns to improve data transfer efficiency.
Applications handling massive graphs deployed within distributed High-Performance Computing (HPC) systems require careful allocation of vertices across processing elements (PEs) to maximize the utilization of the available resources. This allocation should minimize the number of edges cut between PEs while ensuring a balanced workload. Different graph partitioning tools are available to address this issue, providing both static and dynamic methods for efficiently distributing the application graph across the system.Among existing partitioners, multilevel graph partitioning (MGP) approaches produce high-quality partitions with even workload distribution across PEs while minimizing inter-process communication. However, state-of-the-art MGP frameworks such as Zoltan and ParMETIS often struggle with real-world networks. Although ParHIP provides better support for these cases, it lacks dynamic workload balancing, making it unsuitable for dynamic graphs.This paper presents a novel methodology for a distributed, hierarchical, and dynamic multilevel graph partitioning (HDMGP) framework designed for load-balancing large-scale simulations handling real-world graphs. The HDMGP framework is validated through a proof of concept implementation using ParHIP as a baseline. Initial tests demonstrate that the tool maintains repartition quality comparable to the baseline MGP while achieving a repartitioning time up to 8,8 times faster than recomputing the entire graph partition.
The maturity of Cloud Computing (CC) and the advent of Edge Computing (EC) have paved the way to the concept of Cloud-Edge Continuum (CEC), where both of these domains are transparently available for applications to use. However, provisioning an application composed of many services in this Continuum can be challenging due to the dynamic nature of the environment, especially regarding the users’ geographical mobility. This requires a careful analysis from the perspectives of service providers, regarding application’s requirements and users’ Quality-of-Service (QoS) constraints. In this context, we propose, implement and analyze Mobility-aware Cloud-Edge Provisioning Heuristic (MCEPH), a mobility-aware orchestrator for placing service-composed applications atop CEC substrates. MCEPH relies on distances and delay metrics to dynamically provision services on CEC. In short, re-configurations are triggered to improve the overall performance perception of end users. For composing the simulation campaign, we developed a mobility model representing the citizens’ mobility in a Brazilian’s city. The model comprises points of interest, daily activity profiles, and multiple edge and cloud servers. The simulation campaign demonstrates how to configure MCEPH and discusses the overall benefits and drawbacks facing the specialized literature. Overall, the simulation campaign, aided by the more realistic mobility model, indicated great benefits to the users’ experienced quality of service from applications that fit the CEC pattern.
Diabetic retinopathy (DR) diagnosis delays pose a critical challenge for public healthcare systems such as the Brazilian Unified Health System (SUS), where long referral queues often prevent timely treatment and increase the risk of vision loss. Deep learning (DL) models offer an effective solution by automating retinal image analysis, but choosing an appropriate model requires balancing diagnostic accuracy with computational and energy efficiency. This study evaluates 38 convolutional neural networks (CNN) across four key dimensions: Area under the curve (AUC), energy consumption, model size, and training time. Our analysis identifies MobileNet as the superior architecture, demonstrating 77% lower energy use, 83% faster training, and 85% smaller model size than the InceptionV3 baseline, while achieving 3% higher AUC. We further optimize MobileNet through systematic hyperparameter tuning and evaluate its scalability on the ARM-based NVIDIA Grace Superchip, revealing peak efficiency at 36-thread configurations where energy use, CPU utilization, and memory access patterns reach optimal balance. All implementation scripts are publicly available to foster reproducible, sustainable AI development for clinical applications.
Patched inference is a widely used technique in machine learning (ML) that enables fixed-shape models to process arbitrarily large or variably sized inputs by dividing them into smaller, compatible patches. This approach is particularly useful in domains such as seismic processing, medical imaging, and electron microscopy, where data samples often exceed the memory capacity of individual computing nodes. While patched inference is effective for leveraging pre-trained models and operating on resource-constrained hardware, there remains a lack of tools supporting its efficient, distributed execution at scale.To address this gap, we introduce SPINN (Scalable Parallel INference Network), a Python library designed to streamline and accelerate patched inference on high-performance computing (HPC) systems. SPINN supports data partitioning, patch-wise processing using user-defined ML models, and result aggregation, all while leveraging distributed computing frameworks such as Dask and Ray.We validate SPINN on two seismic interpretation tasks, fault detection and facies segmentation, using both public and large-scale private data (up to 272 GB). Experiments demonstrate that SPINN enables smoother prediction outputs via overlapping patches and achieves superlinear scalability with Dask in HPC environments, significantly outperforming conventional solutions such as the NVIDIA Triton Inference Server in large-scale scenarios. SPINN thus emerges as a robust and scalable solution for applying deep learning inference to massive data samples in memory-constrained or compute-intensive settings.
High-performance computing is essential for scientific innovation. With the advent of exascale computing and the growing scale of scientific workloads, novel tools and methodologies are extremely important to analyze and model the performance of large-scale scientific applications. Existing profiling and tracing tools have certain limitations: profiles do not allow fine-grained performance modeling, whereas trace-based simulation or performance modeling is expensive and often infeasible for large applications. In this work, we propose fine-grained communication phases as a level of abstraction for analyzing and modeling application performance. Given the iterative nature of HPC applications, we propose a methodology to automatically group MPI communication events into phases and identify the unique and repeating phases. Our approach enables modeling only the unique phases, in contrast to a complete trace simulation. Further, we address the limitation of existing analytical communication models to model runtime delays in communication time. We propose a new delay-aware communication model (DACM) that achieves a best-case prediction error of 10.11% across six diverse HPC benchmarks and applications, in contrast to 53% using existing analytical models.
Graphics Processing Units (GPUs) are powerful, massively parallel processors that have become ubiquitous in modern computing. In recent years, the GPU market has diversified, with vendors like AMD and Intel offering high-performance alternatives to NVIDIA. However, most applications are written using NVIDIA’s CUDA API, which is incompatible with non-NVIDIA GPUs, creating significant challenges for developers who must port their code to different architectures. To address this issue, AMD developed the Heterogeneous-Compute Interface for Portability (HIP), an open-source API for cross-vendor GPU programming. However, HIP is relatively new, leaving gaps in the literature regarding its performance, portability, and productivity. In this paper, we evaluate HIP using the NAS Parallel Benchmarks (NPB), a CFD-based suite maintained by NASA. We present the first HIP-based implementation of NPB and conduct experiments on integrated and discrete GPUs from NVIDIA, AMD, and Intel. Our results provide novel insights into HIP’s performance and portability, particularly for integrated GPUs and Intel discrete GPUs, which have been underrepresented in prior studies. We also assess productivity using different metrics to quantify the programming effort of HIP-based implementations. This work addresses key gaps in the literature, offering valuable data and insights for developers targeting emerging GPU architectures.
Approximate Computing (AC) has emerged as a promising paradigm by enabling the execution of operations with reduced precision in scenarios where minor errors are acceptable. AC has shown efficiency in domains such as image processing, machine learning, and communication systems, which exhibit a natural tolerance to inaccuracy without significantly degrading outcomes. This work investigates the benefits of AC in improving energy savings across a range of applications. Specifically, this paper presents heuristics that explore the design space of applications to apply instruction-level approximate computation. The vast design space renders the combinatorial optimization problem computationally infeasible; therefore, a branch-and-bound heuristic has been designed to prune the design space, enabling a feasible exploration of the trade-off between accuracy and power consumption. An automated infrastructure is also proposed to guide the selection of approximate instruction sets in error-tolerant scenarios, aiming to achieve energy reduction targets while meeting user-defined accuracy constraints.
Community detection is a common graph workload used in various domains. With the rapid increase in volumes of data, a lot of research has been done on accelerating different community detection algorithms through parallelization. However, the vast majority of such works focus on static graph structures. In recent years, dynamic graphs have been gaining a lot of attention, since various applications require the ability to change their data and execute new analytics. Most of the work on dynamic community detection has been done on sequential modularity-based approaches. In this paper, we present a Map Equation-based approach to dynamic community detection. The Map Equation is used by the Infomap algorithm and achieves better community structures on static graphs compared to modularity. We design our approach to be easy to parallelize, increasing its applicability to real-world graphs. We show that a parallel implementation of our approach can be faster than modularity-based implementations and as fast as parallel naive ones, with a minimal impact on accuracy, providing a positive impact in the efficiency and efficacy of dynamic graph workflows.
High Performance Computing (HPC) is increasingly transitioning to the cloud, although cost remains a significant barrier. While On-demand instances and Committed Use Discounts provide predictable pricing, the Spot market offers an appealing opportunity for substantial cost savings - though it does not guarantee resource availability. Effectively managing multiple instances for parallel HPC applications is essential. Services like AWS EC2 Fleet or Spot Fleet help with this, but they come with limitations, notably being constrained to a single region. Furthermore, simply selecting the lowest-priced instance often leads to suboptimal performance and, surprisingly, higher overall costs. To truly economize, a more sophisticated approach is required: one that involves profiling applications and instances to understand their intricate cost-performance trade-offs. The most cost-effective execution prioritizes instances that strike a better balance between the price per hour being charged and the actual performance they offer the application, even if its current Spot price is not the lowest available. This paper addresses these challenges by conducting a thorough analysis of existing EC2 (Spot) Fleet policies and introducing Fleet-MR, a novel multi-region instance selection framework. Fleet-MR aims to improve execution times or reduce costs, and its effectiveness is validated through experimental evaluations.
Distributed task-based execution models are well-suited for parallelizing irregular applications across clusters. OpenMP Cluster (OMPC) extends the traditional OpenMP tasking model to support distributed memory systems, leveraging a HEFT-based scheduler to improve resource utilization. However, the efficiency of such a scheduler depends heavily on accurate estimates of task execution and communication costs - information that is often difficult to obtain reliably and efficiently. To address this limitation, we propose a novel scheduling framework that combines the recent taskgraph directive introduced in OpenMP 6.0 with partial online profiling of iterative applications. Our approach performs quasi-static scheduling by recording task graphs at runtime and selectively profiling representative iterations to estimate performance. This information is interpolated and fed back into the scheduler to enhance decision-making. We demonstrate that our framework can improve scheduling quality with minimal overhead, making it suitable for long-running or repetitive workloads commonly found in High-Performance Computing (HPC) applications. We achieve up to 20% speedup for the total application and 4x speedup for scheduling.
