Spiking Neural Networks (SNNs) are widely used in brain-inspired computing and neuroscience research. Several many-core accelerators have been built to improve the running speed and energy efficiency of SNNs. However, current accelerators generally need explicit synchronization among all cores after each timestep of SNNs, which poses a challenge to overall efficiency. This paper proposes DepAsync, an asynchronous architecture that eliminates inter-core synchronization, facilitating fast and energy-efficient SNN inference with commendable scalability. The main idea is to exploit the dependency of neuromorphic cores predetermined at compile time. We design a DepAsync scheduler for each core to trace the running state of its dependencies and control the core to safely forward to the next timestep without waiting for other cores to complete their tasks. This approach prevents the necessity for global synchronization, allowing DepAsync to minimize core waiting time facing inherent core and time imbalance in SNN workloads. The comprehensive evaluations using five SNN workloads show that DepAsync achieves 2.47x speedup and 1.55x energy efficiency compared to the state-of-the-art synchronization architectures.

Existing external graph processing systems struggle with low I/O efficiency, high computational overhead, and substantial development costs on modern NVMe SSDs, due to their reliance on complex loading and computing models that transform random I/Os into sequential access. While in-memory graph systems with general-purpose memory-storage caches (like OS page cache or TriCache) offer improved support for fine-grained I/Os and simplified programming, they often fail to leverage specific graph access patterns, resulting in suboptimal performance. This paper aims to enhance the I/O efficiency of large-scale graph processing on NVMe SSDs. We first introduce a novel chunk-based graph representation model, featuring classified and hierarchical vertex storage and chunk layout optimization, to improve I/O utilization. Additionally, we present a latency-optimized access mechanism featuring user-space asynchronous I/O execution and hotness-aware chunk caching management to accelerate I/O and boost cache efficiency. Our prototype, ChunkGraph, enables flexible graph algorithm implementation and efficient execution. Experiments demonstrate ChunkGraph significantly outperforms existing external graph systems and in-memory graph systems using general-purpose cache subsystems, delivering several-fold speedups on overall performance and superior I/O efficiency.

In response to the increasing demand for largescale machine learning training jobs, many organizations have deployed GPU clusters across geographically distributed regions. However, existing ILP- or genetic-based cross-cluster training approaches largely overlook the topology of decentralized clusters, lacking both topologyaware task scheduling mechanisms and automated model parallelization strategies. As a result, naively applying these optimization-based methods in cross-cluster settings leads to prohibitive scheduling overhead, due to the drastically enlarged search space induced by complex inter-cluster topologies. To address these challenges, we propose SpiderFlow, a topologyaware scheduling system specifically designed for decentralized GPU clusters. We formulate cross-cluster task scheduling as a graph optimization problem and introduce SpinSearch, a low-overhead topology-aware scheduling algorithm. In addition, for automated model parallelization, we propose TPA, a two-level scheduling framework that combines heuristic methods at the inter-cluster level with ILP-based optimization within clusters, effectively reducing the search space while maintaining high training throughput with substantially lower scheduling overhead. We evaluate SpiderFlow on a physical platform comprising 8 decentralized clusters, as well as on a simulation platform with up to 64 decentralized clusters. Experimental results demonstrate that SpiderFlow reduces job completion time (JCT) by 1.2-1.3×, improves throughput by 1.12-1.25×, and reduces scheduling overhead by 20-90× on average compared to state-of-the-art scheduling systems.

Processing-in-memory (PIM) accelerators perform in-situ dot-product operations between input feature maps (IFMs) and convolutional neural network (CNN) weights to accelerate inference. The efficiency of these computations is heavily influenced by the weight mapping strategy. However, existing approaches often adopt IFM-oriented mapping schemes that provide limited parallelism, leading to high latency and low energy efficiency. To address this issue, we propose SIFMAP, a subIFM-oriented weight mapping framework designed to enable highly parallel CNN inference. SIFMAP leverages the observation that different regions within an IFM contribute unevenly to computation. It first partitions the IFM into fine-grained sub-maps, or subIFMs, based on their computational characteristics. Then, it applies a reinforcement learning (RL) technique to determine optimal weight mapping strategies tailored to each subIFM, optimizing global performance. Furthermore, SIFMAP introduces a subIFM-oriented resource allocation scheme to further boost computational parallelism across subIFMs. Experimental results show that SIFMAP reduces inference latency by 4.8× and improves energy efficiency by 5.2× compared to state-of-the-art mapping methods under equivalent resource constraints.

Dynamic Graph Neural Networks (GNNs) combine temporal information with GNNs to capture structural, temporal, and contextual relationships in dynamic graphs simultaneously, leading to enhanced performance in various applications. As the demand for dynamic GNNs continues to grow, numerous models and frameworks have emerged to cater to different application needs. There is a pressing need for a comprehensive survey that evaluates the performance, strengths, and limitations of various approaches in this domain. This paper aims to fill this gap by offering a thorough comparative analysis and experimental evaluation of dynamic GNNs. It covers 81 dynamic GNN models with a novel taxonomy, 12 dynamic GNN training frameworks, and commonly used benchmarks. We also conduct experimental results from testing representative nine dynamic GNN models and three frameworks on six standard graph datasets. Evaluation metrics focus on convergence accuracy, training efficiency, and GPU memory usage, enabling a thorough comparison of performance across various models and frameworks. From the analysis and evaluation results, we identify key challenges and offer principles for future research to enhance the design of models and frameworks in the dynamic GNNs field.

Training large language models (LLMs) on heterogeneous GPU clusters is increasingly common, but resource provisioning remains a core bottleneck. Existing systems either require users to manually choose GPU types and counts or rely on static memory formulas that miss runtime factors (e.g., allocator behavior and communication buffers), leading to OOM failures or conservative over-provisioning. At the same time, heterogeneous scheduling often depends on expensive optimization procedures that introduce high scheduling latency. To solve this problem, we propose Frenzy, a memory-aware serverless LLM training system for heterogeneous GPU clusters. Frenzy first estimates peak training memory using analytical modeling calibrated by runtime feedback to infer robust GPU type-and-quantity plans. It then applies a pricing-driven greedy scheduler with lightweight local swaps to achieve efficient near-optimal placement with low overhead. We evaluate Frenzy using multi-task LLM training on a physical heterogeneous cluster with 8 nodes and 4 GPU types. Compared with state-of-the-art baselines, Frenzy reduces average job completion time by 12 × , and achieves over 97

Modern advanced large language model (LLM) applications often prepend long contexts before user queries to improve model output quality. These contexts frequently repeat, either partially or fully, across multiple queries. Existing systems typically store and reuse the keys and values of these contexts (referred to as prefix KVs) to reduce redundant computation and time to first token (TTFT). When prefix KVs need to be stored on disks due to insufficient CPU memory, reusing them does not always reduce TTFT, as disk I/O latency is high. In this paper, we present HyperInfer, an importance-informed multi-tier prefix KV caching and prefetching system designed to reduce TTFT in LLM inference. HyperInfer first employs an I/O-efficient algorithm to identify and load only the most important KVs, thereby minimizing I/O overhead. Then, to further hide loading latency, it leverages the inter-layer similarity of token importance to speculatively prefetch the next layer’s critical KVs during the current layer’s computation. Finally, HyperInfer optimizes prefix-KV storage and cache utilization via importance-informed KV placement, further reducing TTFT in end-to-end inference. Our experimental results show that HyperInfer can reduce TTFT by up to 1.75× compared to state-of-the-art systems, while maintaining comparable inference accuracy.

Graph convolutional networks (GCNs) are popular for a variety of graph learning tasks. ReRAM-based processing-in-memory (PIM) accelerators are promising to expedite GCN training owing to their in-situ computing capability. However, existing accelerators can be severely underutilized even with pipelines, due to the oversight of the skewed execution times of various GCN stages and the ignorance of skewed degrees of graph vertices. In this work, we propose GOPIM, a GCN-oriented pipeline optimization for PIM accelerators to expedite GCN training. First, GOPIM proposes an ML-based scheme that allocates crossbar resources to the most needed stages to streamline the overall pipeline. Second, GOPIM utilizes a selective vertex updating technique that evenly distributes vertices on crossbars by interleaved mapping. These techniques collectively reduce the overall execution time without losing much accuracy. We also provide a practical architecture design for GOPIM. Our experimental results show that, GOPIM achieves up to 191x speedup and 16.1x energy saving, compared to the state-of-the-art work.

Due to the diversity of edge devices (EDs) and applications, edge systems are heterogeneous and have been applied in artificial intelligence fields, such as smart factories and intelligent transportation, which is called heterogeneous edge intelligence. Many studies employ computation offloading to transfer processing data from resource-scarce EDs to resource-rich edge servers. These studies primarily focus on the overall resource consumption of homogeneous edge systems, neglecting the system heterogeneity and the details of resource consumption. In this article, we construct a system model from a parallel perspective for the heterogeneous edge system with different processors, memory, and applications, which perceives the cost of energy and delay from three levels: system, application, and component. A hybrid metaheuristic algorithm combined with a greedy rule, hybrid mutation, and whale optimization algorithm (GHMWOA) is proposed to realize partial computation offloading. A partial offloading architecture of heterogeneous edge intelligence is proposed to validate our model and algorithm with real-world hardware and software. Experiment results not only show GHMWOA outperforms multiple classical optimization algorithms in minimizing energy consumption, but also discover on which system component energy consumption depends, and how properties of application and system influence the cost of energy.

Training large language models (LLMs) is often constrained by GPU memory limitations. To alleviate memory pressure, activation recomputation and data compression have been proposed as two major strategies. However, both approaches have limitations: recomputation introduces significant training overhead, while compression can lead to accuracy degradation and computational inefficiency when applied naively. In this paper, we propose Adacc, the first adaptive memory optimization framework that unifies activation recomputation and data compression to improve training efficiency for LLMs while preserving model accuracy. Unlike existing methods that apply static, rule-based strategies or rely solely on one technique, Adacc makes fine-grained, tensor-level decisions, dynamically selecting between recomputation, retention, and compression based on tensor characteristics and runtime hardware constraints. Adacc tackles three key challenges: (1) it introduces layer-specific compression algorithms that mitigate accuracy loss by accounting for outliers in LLM activations; (2) it employs a MILP-based scheduling policy to globally optimize memory strategies across layers; and (3) it integrates an adaptive policy evolution mechanism to update strategies during training in response to changing data distributions. Experimental results show that Adacc improves training throughput by 1.01x to 1.37x compared to state-of-the-art frameworks, while maintaining accuracy comparable to the baseline.

Because of the increasing speed gap between speed of compute and storage, caching is critical for improving the throughput of distributed file systems. It has been shown that prefetching can hide the latency resulted by network communication or disk operations. However, conventional client-based prefetching schemes are not efficient in distributed file systems as the limited computing and memory power of client nodes. In this paper, we present an effective and load-aware server-side prefetching scheme for distributed file systems, name SSPF. As an orthogonal approach, SSPF can be coupled with any existing caching scheme. First, SSPF exploits spatial locality to improve the efficiency of the prefetching cache and minimize memory requirement. Then, for maximizing the efficiency of cache, a multi-queue based cache manager is designed to coordinate between the prefetching blocks and other caching blocks. Furthermore, a heuristic-based request distribution strategy is proposed to optimize the balance between data server nodes and improve the overall performance. Finally, we have implemented and evaluated SSPF on the real distributed file system. Experimental results show that SSPF can significantly improve the read performance with negligible memory over-head.

Neuromorphic hardware systems-designed as 2D-mesh structures with parallel neurosynaptic cores-have proven highly efficient at executing large-scale spiking neural networks (SNNs). A critical challenge, however, lies in mapping neurons efficiently to these cores. While existing approaches work well with regular, fully functional mesh structures, they falter in real-world scenarios where hardware has irregular shapes or non-functional cores caused by defects or resource fragmentation. To address these limitations, we propose a novel mapping method based on an innovative space-filling curve: the Adaptive Locality-Preserving (ALP) curve. Using a unique divide-and-conquer construction algorithm, the ALP curve ensures adaptability to meshes of any shape while maintaining crucial locality properties-essential for efficient mapping. Our method demonstrates exceptional computational efficiency, making it ideal for large-scale deployments. These distinctive characteristics enable our approach to handle complex scenarios that challenge conventional methods. Experimental results show that our method matches state-of-the-art solutions in regular-shape mapping while achieving significant improvements in irregular scenarios, reducing communication overhead by up to 57.1%.

Maximal biclique enumeration (MBE) in bipartite graphs is an important problem in data mining with many real-world applications. Parallel MBE algorithms for GPUs are needed for MBE acceleration leveraging its many computing cores. However, enumerating maximal bicliques using GPUs has three main challenges including large memory requirement, thread divergence, and load imbalance. In this paper, we propose GMBE+, an advanced GPU solution for the MBE problem. To overcome the challenges, we design (1) a node-reuse approach to reduce GPU memory usage with advanced node pruning, (2) a bitmap-based set intersection approach to minimize thread divergence, and (3) a load-aware task scheduling framework to achieve load balance among threads within GPU warps, facilitated by a novel set union approach. Our experiments reveal that GMBE+ is 1.2x faster than the latest GPU-based MBE algorithm GMBE on average when running on the same NVIDIA A100 GPU.

Delta compression can complement data deduplication by further minimizing redundancy through the compression of non-duplicate data chunks. When adding delta compression to deduplication-based backup systems, however, two primary challenges arise that degrade performance of inline deduplication. First, extra I/Os are introduced along the critical paths of backup and restoration for retrieving base chunks, slowing the system. Second, rewriting techniques prohibit specific data chunks from serving as base chunks for delta compression to improve restore performance, resulting in a loss of compression efficiency. In this paper, we introduce LoopDelta, a framework that seamlessly integrates delta compression into inline deduplication for backup storage, addressing the aforementioned challenges by using three techniques: (1) dual-locality-based similarity tracking leverages both logical and physical locality to detect most of the similar chunks, which, due to their locality, can be prefetched by piggybacking on routine operations during deduplication, thereby eliminating extra I/Os during backup; (2) cache-aware filter identifies base chunks requiring extra I/Os during restore and prevents their referencing, thus eliminating extra restore I/Os; and (3) inversed delta compression, which reverses the roles of base and target chunks in the traditional delta compression approach, thereby allowing for the delta compression of data chunks that are otherwise prohibited as base chunks due to rewriting techniques. Experiments show that LoopDelta increases the compression ratio by 1.28 to 11.33 times over basic deduplication, without significantly affecting backup throughput, and enhances restore performance by up to 3.57 times.

Distributed training of graph neural networks (GNNs) has become a crucial technique for processing large graphs. Prevalent GNN frameworks are model-centric, necessitating the transfer of massive graph vertex features to GNN models, which leads to a significant communication bottleneck. Recognizing that the model size is often significantly smaller than the feature size, we propose LeapGNN, a feature-centric framework that reverses this paradigm by bringing GNN models to vertex features. To make it truly effective, we first propose a micrograph-based training strategy that leverages a refined structure to enhance locality, combined with the model migration technique, to minimize remote feature retrieval. Then, we devise a feature pre-gathering approach that merges multiple fetch operations into a single one to eliminate redundant feature transmissions. Finally, we employ a micrograph-based merging method that adjusts the number of micrographs for each worker to minimize kernel switches and synchronization overhead. Our experimental results demonstrate that LeapGNN achieves a performance speedup of up to 4.2. compared to the state-of-the-art method, namely P-3.

With the increasing time and frequency resolution of modern radio telescopes and the exponential growth in observational data volumes, real-time single-pulse detection has become a critical requirement for time-domain radio astronomy. Heimdall, as a representative GPU-accelerated single-pulse search tool, offers substantial performance advantages over CPU-based approaches. However, its sequential execution model and resource contention in intermediate processing stages limit GPU utilization, leading to suboptimal throughput and increased computational latency. To address these limitations, we present Heimdall++, an optimized successor to Heimdall that incorporates fine-grained GPU parallelization, enhanced memory management, and a multi-threaded framework to decouple CPU-bound and GPU-bound processing stages. This design mitigates the GPU stall problem and improves end-to-end efficiency. We evaluated Heimdall++ on a system equipped with NVIDIA RTX 3080 Ti GPUs using both a single large-scale observational file and multiple files. Experimental results demonstrate that Heimdall++ achieves up to 2.66x speedup in single-file processing and 2.05x speedup in multi-file batch processing, while maintaining full consistency with the original Heimdall's search results.

Graph neural networks (GNNs) are effective models for analyzing graph-structured data, but encounter challenges when training on large distributed graphs. Existing GNN training frameworks use sampling parallelism and historical embedding methods to support distributed training and enhance efficiency. However, these methods suffer from issues like stale historical embeddings, imbalanced communication messages, and redundant storage and computation costs. In this paper, we present Emma, a distributed GNN training framework that incorporates source node centric chunking for frequent updates of embeddings and balanced communication, as well as a moving message aggregation technique to boost training efficiency and reduce storage costs. Experimental results show that Emma significantly enhances training efficiency by reducing computation and communication overhead, leading to a notable speedup while maintaining convergence accuracy compared to state-of-the-art distributed GNN training methods.

Modern advanced large language model (LLM) applications often prepend long contexts before user queries to improve model output quality. These contexts frequently repeat, either partially or fully, across multiple queries. Existing systems typically store and reuse the keys and values of these contexts (referred to as prefix KVs) to reduce redundant computation and time to first token (TTFT). When prefix KVs need to be stored on disks due to insufficient CPU memory, reusing them does not always reduce TTFT, as disk I/O latency is high. In this paper, we propose IMPRESS, an importance-informed multi-tier prefix KV storage system to reduce I/O delay for LLM inference by only loading important prefix KVs. IMPRESS first leverages the insight that there is significant similarity in important token index sets across attention heads and introduces an I/O-efficient important KV identification algorithm. It then optimizes prefix KV storage and caching through importance-informed KV management, reducing TTFT during model inference. Our experimental results show that IMPRESS can reduce TTFT by up to 2.8x compared to state-of-the-art systems, while maintaining comparable inference accuracy.

Neuromorphic systems, designed to execute Spiking Neural Networks (SNNs) efficiently, feature a many-core architecture equipped with a Network-on-Chip (NoC) to facilitate spike communication. The distributed memory and decentralized computation require partitioning SNNs into multiple subnetworks, each executed on a neuromorphic core in parallel. However, the increasing scale of SNNs leads to high inter-core spike traffic and expensive computational cost of partitioning algorithms. To address these challenges, we propose SNNcut, an efficient partitioning method for large-scale SNNs using spike-sharing. This approach fully exploits spike-sharing by using one NoC packet to deliver multiple spikes of information to reduce spike traffic. Meanwhile, instead of exhaustively searching over neurons in SNN, SNNcut indexes neurons based on the layer connection pattern and clusters sequentially, which significantly reduces the computational complexity and ensures the efficiency of large-scale SNN partitioning. Experimental results demonstrate that SNNcut achieves an average reduction in spike traffic of 98.97%, compared to 74.98% from state-of-the-art existing methods. In addition, the SNNs partitioned by SNNcut require only 5.7%∼15.7% neuromorphic core consumption of previous works. In a large-scale scenario, existing methods take more than 100 hours to accomplish the partitioning, while SNNcut only needs 10 minutes. We also validate the scalability of SNNcut by testing on edge cases with 3.2B neurons and 902B synapses.

Existing in-memory graph storage systems that rely on DRAM have scalability issues because of the limited capacity and volatile nature of DRAM. The emerging persistent memory (PMEM) offers us a chance to solve these issues through its larger capacity and non-volatile characteristics. However, simply adapting existing DRAM-based graph storage systems to PMEM would result in inefficient PMEM stores and accesses, including high read and write amplification to PMEM, imbalanced work division for PMEM accesses, and costly remote PMEM access across NUMA nodes. These issues severely limit the performance of large graph processing. In this paper, we aim to achieve scalable and high-performance graph processing in PMEM. We first propose an XPLine-friendly graph storage model that uses vertex-centric graph buffering, hierarchical vertex buffer managing, and in-place vertex block merging to optimize PMEM graph storage. Furthermore, we develop a scalable graph processing model that leverages multi-threaded work dividing and NUMA-friendly graph accessing to optimize PMEM graph accesses. Based on these techniques, we implement XPGraph, a PMEM-based graph storage system for large-scale evolving graphs, and several variants for different system settings. Our experiments demonstrate that XPGraph surpasses the state-of-the-art in-memory graph storage system on a PMEM-based system by 3.07 × to 4.99 × in update performance and up to 5.87 × in query performance, and performs much better in highly parallel multi-threaded scenarios.