Achieving peak GPU performance remains a significant challenge as the system throughput is constrained by host-device synchronization delays and kernel scheduling overheads, even with aggressive kernel optimizations and batch processing. Furthermore, existing approaches often underutilize hardware resources such as compute cores and copy engines due to scheduling overheads. To address these problems, we propose a CUDA runtime framework for task-parallel pipelines to minimize the synchronization overheads and the gap between kernel executions. The proposed solution combines two innovations: (1) a multi-stream task-parallel pipeline programming model that leverages event-chaining and work-stealing mechanisms to fully utilize available hardware resources; (2) a graph-based execution flow with per-stream buffers to ensure memory safety for multiple in-flight jobs running concurrently. Extensive evaluations on representative real-world workloads show 1.15–1.44 × speedup and reduce scheduling overheads by 18–54
Alpine livestock monitoring is critical for ecological preservation and agricultural efficiency. However, existing solutions struggle with energy constraints, limited network availability, and intermittent connectivity in remote environments. To address this, we propose an on-device federated learning framework tailored for PV-powered IoT sensors to optimize energy-communication tradeoffs. Our approach introduces staleness-aware aggregation and solar-aware training scheduling to address intermittent connectivity and PV variability in remote alpine environments. Deployed on a real-world testbed with collar sensors, the framework achieves 92
Data partitioning, i.e. rearranging data according to a Boolean predicate, is needed for many operations on large data. This includes algorithms such as sorting, convex hull computations, in load balancing across a cluster, or in graph algorithms. Several efficient implementations for data partitioning have been proposed in the literature. Some of these focus on the ability to compute the result with only a constant amount of additional memory, known as in-place implementations. Others focus on efficient parallel executions. Our described strategy of implementing an in-place algorithm on GPUs includes keeping the memory requirements and movements as low as possible while maintaining enough parallelism and coalesced access patterns. We present an in-place partitioning algorithm that can be executed on massively parallel systems. Our implementation maps well to GPU architectures while moving only a negligible amount of data more than necessary, for non-adversarial input. We quantify ‘negligible’ by providing a probabilistic bound for random input and derive a worst-case bound. Our performance evaluation demonstrates that our algorithm achieves between 94
This work aims at designing and evaluating scheduling algorithms that minimize carbon cost on edge platforms. When a job is released to some edge server, difficult scheduling questions arise: should the job be executed on that server? If yes, when? If no, which other edge server should the job be transferred to? Typically, jobs are submitted online, and have a deadline to enforce. Online scheduling problems are already difficult without accounting for different energy sources, so one should not expect any optimal solution. Still, an important research goal is to revisit standard algorithms such as Earliest Completion Time (ECT) and Earliest Deadline First (EDF) in order to design and evaluate carbon-aware variants. This paper introduces several new algorithms that use sophisticated scheduling policies to efficiently decrease carbon cost; these algorithms maximize the use of green energy both on local and remote edge servers, by re-evaluating previous decisions whenever needed to accommodate newly released jobs. We provide a comprehensive simulation campaign based on actual platform/job data and carbon traces and report an average gain of 42
Each new generation of GPUs significantly enhances the resources available for diverseGPGPU applications, with kernel concurrency playing a crucial role in maximizing utilization and boosting performance. However, existing kernel concurrency strategies usually tend to neglect cache contention, where concurrent kernels potentially target the same cache levels. Traditional cache management methods are inadequate for addressing this issue, as they focus on individual kernels without heavily considering inter-kernel interactions. To overcome these challenges, we propose CacheC, a method that utilizes large language models (LLMs) to analyze cache affinity at the granularity of individual load instructions. For each kernel pair, CacheC extracts detailed features of all loads, evaluates their cache affinity across levels, and scores their suitability for concurrency. Based on these scores, CacheC not only selects kernel pairs with appropriateoptimal cache compatibility but also formulates load-specific cache bypassing strategies to enhance utilization. By iteratively scheduling kernel pairs and adjusting their cache policies, CacheC dynamically optimizes cache utilization and reduces cache contention during concurrent kernel execution. Experiments on off-the-shelf GPUs demonstrate that CacheC achieves a 19.67 × across scheduled kernel pairs, showcasing its effectiveness in alleviating cache contention and enhancing kernel concurrency performance.
Stochastic gradient descent (SGD) is a crucial optimisation algorithm due to its ubiquity in machine learning applications. Parallelism is a popular approach to scale SGD, but the standard synchronous formulation struggles due to significant synchronisation overhead. For this reason, asynchronous implementations are increasingly common. These provide an improvement in throughput at the expense of introducing stale gradients which reduce model accuracy. Previous approaches to mitigate the downsides of asynchronous processing include adaptively adjusting the number of worker threads or the learning rate, but at their core these are still fully asynchronous and hence still suffer from lower accuracy due to more staleness. We propose Interval-Asynchrony, a semi-asynchronous method which retains high throughput while reducing gradient staleness, both on average as well as with a hard upper bound. Our method achieves this by introducing periodic asynchronous intervals, within which SGD is executed asynchronously, but between which gradient computations may not cross. The size of these intervals determines the degree of asynchrony, providing us with an adjustable scale. Since the optimal interval size varies over time, we additionally provide two strategies for dynamic adjustment thereof. We evaluate our method against several baselines on the CIFAR-10 and CIFAR-100 datasets, and demonstrate a 32
Diffusion models have emerged as powerful tools for generative AI tasks. While prior research primarily focuses on eliminating redundancy across timesteps, models like Stable Diffusion introduce a ResNet-Transformer Alternating Execution (RTAE) Pattern, where convolution and attention operators execute sequentially within each timestep. This execution pattern leads to excessive on-chip memory access and poor computational resource utilization due to the mismatched characteristics of convolution and Transformer operations. To tackle these challenges, we propose FDHA, an accelerator designed for efficient diffusion model inference. First, to mitigate redundant on-chip memory access, FDHA introduces an inter-operator dataflow fusion mechanism that strategically aligns ResNet’s convolution and Transformer’s matrix multiplication dimensions, enabling efficient kernel reuse. Second, to maximize computational resource utilization, FDHA employs a heterogeneous architecture with dedicated Processing Elements for convolutions and Tensor Processing Elements for matrix multiplications, allowing for pipelined execution. Experimental results demonstrate that FDHA achieves 3.28 × speedup over an NVIDIA A100 GPU and 2.62 × speedup over a SoTA diffusion accelerator.
Graph Neural Networks (GNNs) commonly employ sampling-based methods for inference on large-scale real-world graphs. However, the inherent characteristics of sampling lead to redundant data loading during GNN inference, while slow data transfer between the host and GPU exacerbates the issues of slow inference and low resource utilization. Current methods to accelerate GNN inference face several challenges: (1) low GPU resource utilization; (2) neglect of adjacency matrix locality; and (3) long preprocessing time. To address these issues, we propose DCI, a system designed to accelerate GNN inference. The system provides a simple and effective cache capacity allocation and filling strategy that can adapt flexibly to different workload demands. During the pre-sampling phase, DCI allocates and fills cache capacities for node features and adjacency matrices based on workload patterns. Experimental results show that DCI accelerates sampling and node feature loading, achieving end-to-end inference speedups of 1.18 × to 11.26 × compared to DGL, and 1.14 × to 13.68 × compared to RAIN, while reducing preprocessing time by 52.8 × to 1.32 × . We also compared DCI with DUCATI’s dual-cache population strategy, and DCI achieves nearly identical inference speeds while reducing preprocessing time to less than 20
Replication-aided partitioning (RAP) has recently been introduced to facilitate the design of parallel logic simulation algorithms. By replicating overlapped work, RAP can significantly reduce the cost of inter-thread synchronization. However, the state-of-the-art RAP algorithm, RepCut, relies on time-consuming hypergraph construction and partitioning, where minimizing cut size corresponds to reducing replication. To overcome this runtime challenge, we introduce SimPart, a simple yet highly effective and efficient GPU-parallel replication-aided partitioner. SimPart tackles the partitioning problem directly without solving another proxy problem and proposes a hybrid strategy that can maximally utilize GPU threads for simulation atop our partitions. Compared to RepCut, SimPart achieves an average speedup of 23 × in partitioning and 1.58 × in GPU-parallel simulation, while increasing the original graph size by only 0.3
Compound operators, such as Log_softmax and RMSNorm, have been widely studied to enhance performance in deep neural networks (DNNs). Nonetheless, these operators often suffer from high hardware adaptation costs and limited optimization effects. AI compilers optimize them through operator splitting and successive operator fusion strategies. However, prior studies indiscriminately split all compound operators and failed to fully explore the fusion search space, incurring inefficient fusion schemes. To overcome these limitations, we propose a Co-Optimization Framework for Operator Splitting and Fusion (CoSF). In the operator splitting phase, we analyze memory reuse levels among operators and classify the compound operators into three types, according to their data locality. Then, we propose a fusion-aware splitting strategy. For each type of compound operator, it evaluates the successive fusion benefits after splitting the compound operator and automatically generates operator splitting strategies. In the operator fusion phase, to reduce the massive computation graph resulting from operator splitting, we propose a dominator tree-based graph partitioning algorithm to efficiently partition the computation graph. We then employ dynamic programming for each partitioned subgraph to generate an optimized fusion strategy. Finally, we propose a hardware-agnostic evaluation model to select the most effective fusion solution from multiple candidates. Experimental results demonstrate that CoSF achieves a 1.3–3.4 × speedup on GPU and a 1.59–3.93 × speedup on CPU compared to TVM, Pytorch, and TF-XLA.
Multi-DNN accelerators enable the simultaneous execution of multiple DNN workloads which improves performance by overlapping computations and memory accesses of multiple DNN workloads. However, on-chip memory must accommodate the footprint of all workloads. Batching allows DNN inferences using the same model to share weights which improves weight reuse and reducing off-chip access costs over a batch. Batching determines the batch size statically, leading to stalls when there is not enough on-chip memory available at runtime. This paper introduces BATCH-DNN, a dynamic method for adapting batch size on a layer-by-layer basis to available on-chip memory. It employs two techniques: adaptive cascaded sub-batching and adaptive sub-batch merging. Offline profiling establishes the footprint, while run-time adjustment establishes the maximum batch size on a layer-by-layer basis based on available on-chip memory. BATCH-DNN can improve the utilization of accelerator compute fabrics by 60
Simultaneous multithreading (SMT) processors improve system throughput by sharing core resources among the threads running on the same core. However, intra-core interference can cause co-running applications to degrade each other’s performance significantly. To address this issue, some approaches have focused on balancing contention at the core shared resources (e.g. the shared L1 data cache). A key advantage of these approaches is that they are workload-agnostic. Other approaches improve the previous ones by modeling the inter-application interference across the intra-core shared resources. Unfortunately, these approaches require off-line model training for specific workloads. This paper presents WAPA, a CPI-based thread-to-core allocation approach that incorporates the best of both worlds. WAPA is a workload-agnostic policy that implicitly accounts for inter-thread interference across all the shared resources by leveraging the CPI. The proposed approach relies on the optimal transport (OT) theory, a mathematical theory to dynamically select symbiotic pairs of applications. Experimental results in an Intel Xeon show that WAPA outperforms the default Linux scheduler on average by 8.4
The exponential growth of digital data poses a significant storage challenge, straining current storage systems in terms of cost, efficiency, maintainability, and available resources. For large-scale data archiving, highly efficient data compression techniques are vital for minimizing storage overhead, communication efficiency, and optimizing data retrieval performance. This paper presents a scalable parallel workflow designed to compress vast collections of files on high-performance computing systems. Leveraging the Permute-Partition-Compress (PPC) paradigm, the proposed workflow optimizes both compression ratio and processing speed. By integrating a data clustering technique, our solution effectively addresses the challenges posed by large-scale data collections in terms of compression efficiency and scalability. Experiments were conducted on the Leonardo petascale supercomputer of CINECA (leonardo-supercomputer.cineca.eu), and processed a subset of the Software Heritage archive, consisting of about 49 million files of C++ code, totaling 1.1 TB of space. Experimental results show significant performance in both compression speedup and scalability.
With Moore's law approaching its end, traditional von Neumann architectures are struggling to keep up with the exceeding performance and memory requirements of artificial intelligence and machine learning algorithms. Unconventional computing approaches such as neuromorphic computing that leverage spiking neural networks (SNNs) to perform computation are gaining traction and seek the paradigm shift necessary to sustain the increasing demands of modern applications. Novel memory technologies, such as resistive RAM (ReRAM), employ a crossbar architecture that possesses the inherent capability of efficiently computing vector-matrix multiplication-a dominant operation in SNNs. The prospect of naturally mapping SNNs to the crossbar structures provides a unique opportunity for achieving a high-performance, powerefficient neuromorphic system. In this work, we present ReSpike, which is a new framework, behavioral simulator, and architectural design based on ReRAM crossbar architectures, enabling modeling and co-design to achieve efficient execution of SNNs. We drive this co-design forward by quantifying the impact that ReRAM cell nonidealities have on the corresponding accuracy of an SNN application.
Sparse matrix LU factorization is a critical method in direct solvers, playing a significant role in the field of first-principles materials simulation. Matrices in quantum chemistry problems often exhibit locally dense properties, yet their spatial structural characteristics have been overlooked in previous efforts. This paper proposes a novel LU factorization algorithm that leverages application-specific locally dense structures by partitioning sparse matrices into uniform dense blocks. Through systematic integration of level-3 BLAS kernels, the method transforms traditionally memory-bound LU operations into compute-intensive tasks, achieving significant improvements in both computational efficiency and CPU utilization. We conducted performance tests on CPUs from three different vendors, including the x86-based Intel Xeon Platinum 8375C and AMD EPYC 7543, as well as the ARM-based Kunpeng 920. Experimental results demonstrate significant performance improvements compared to the state-of-the-art sparse direct solvers.
Sequence alignment is a crucial step in next-generation sequencing data analysis. However, most sequence aligners face performance challenges due to high computational complexity and extensive random memory access patterns, making them a significant bottleneck in the overall analysis pipeline, such as the industry gold standard BWA-MEM. The next-generation Sunway platform, with its high computational power and unique heterogeneous architecture, presents new opportunities for enhancing the efficiency of sequence alignment. In this work, we introduce SWBWA, a high-accuracy and high-performance sequence aligner designed for the new Sunway architecture. By redesigning the parallel framework tailored for Sunway, performing software prefetching optimization, vectorizing the striped Smith-Waterman algorithm, and addressing memory access bottlenecks in bigshare mode, SWBWA achieves a 330 × speedup over the single-threaded unoptimized version. Additionally, SWBWA running on a Sunway workstation can achieve 1.2–1.4 × speedups compared to BWA-MEM running on a dual-socket 48-core x86 server, while ensuring nearly identical output. The source code is publicly available at https://github.com/RabbitBio/SWBWA .
Recent leaps in large language models (LLMs) caused a revolution in programming tools (like GitHub Copilot) that can help with code generation, debugging, and even performance optimization. In this paper, we focus on the capabilities of the most recent reasoning models to generate optimized CUDA code for predefined, well-known tasks. Our objective is to determine which types of code optimizations and parallel patterns the LLMs can perform by themselves and whether they can be improved by tutoring (providing more detailed hints and guidelines in the prompt). The generated solutions were evaluated both automatically (for correctness and speedup) and manually (code reviews) to provide a more detailed perspective. We also tried an interactive approach where the LLM can fix its previous mistakes within a session. The results indicate that LLMs are quite skilled coders; however, they require tutoring to reach optimized solutions provided by parallel computing experts.
We examine the problem of scheduling tasks on identical parallel machines using the SLACK heuristic. This method sorts tasks in non-increasing order of processing times, partitions them into sets of size m (corresponding to the number of machines), and subsequently schedules them in non-increasing order of slack with a list-based heuristic. Similar to LPT, SLACK has a time complexity of O(n log n) , where n denotes the number of tasks, and exhibits strong empirical performance in some scenarios. However, no formal approximation guarantee has been established for this heuristic. In this work, we provide a 4/3-approximation ratio, which, while slightly worse than with LPT, is tight. Moreover, we derive improved bounds under the constraint that processing times do not exceed a fraction of the optimal makespan. Specifically, we show that SLACK is a ( 1+m-1/m(k+1)) -approximation algorithm when the processing time of any task is at most OPT/ k for k ≥ 2 .
For fast processing of increasingly large graphs, triangle counting – a common building block of graph processing algorithms, is often performed on GPUs. However, applying massive parallelism to triangle counting is challenging due to the algorithm’s inherent irregular access patterns and workload imbalance. In this work, we propose WeTriC, a novel wedge-parallel triangle counting algorithm for GPUs, which, using fine(r)-grained parallelism through a lightweight static mapping of wedges to threads, improves load balancing and efficiency. Our theoretical analysis compares different parallelization granularities, while optimizations enhance caching, reduce work-per-intersection, and minimize overhead. Performance experiments indicate that WeTriC yields 5.63× and 4.69× speedup over optimized vertex-parallel and edge-parallel binary search triangle counting algorithms, respectively. Furthermore, we show that WeTriC consistently outperforms the state-of-the-art (i.e., on avg. 2.86× faster than Trust and 2.32× faster than GroupTC).
Compute Express Link (CXL) is recognized as a revolutionary technology in high-performance computing (HPC) system design, driven by the growing demand for efficient and scalable memory solutions tailored to memory-centric workloads. However, despite its potential, evaluating CXL performance in real-world scenarios is challenging due to the lack of CXL hardware and the high costs of building a large-scale distributed system. To address this, we propose TSim4CXL, a novel trace-driven simulation framework for CXL-based HPC systems that provides accurate timing simulations within a practical timeframe. TSim4CXL separates computing resources from the CXL memory system, generating traces and simulating the memory system using SystemC's discrete-event modeling. By modeling the CXL interconnect at the protocol level with various configuration parameters, TSim4CXL allows us to explore the design space of HPC architecture. The accuracy of our CXL simulation model is validated using CXL hardware provided by Samsung Electronics. First, we compare load latencies using a custom microbenchmark on CXL hardware with simulation results and adjust the CXL parameters in our simulator accordingly. Second, we assess communication latency by running LAMMPS applications, ensuring the simulation results align with real-world performance. In addition, we perform design space exploration with two memory-centric applications, up to 25 CPU nodes for LAMMPS and 4 GPU nodes for LLM training. Furthermore, we compare the performance of target applications by executing multiple DRAM simulators, demonstrating how the memory bandwidth affects simulated time. These experiments prove the viability of the proposed simulation framework.
