Sparse tensor computation is widely used in deep learning and scientific computing. However, diverse sparse data and algorithmic characteristics at the application level, combined with the diversity of hardware platforms, pose significant challenges for efficient sparse tensor program optimization. Manually crafted operator libraries are time-consuming to develop and lack portability. To address this, we propose UniSparTa, a unified sparse tensor program tuning framework that automatically generates high-performance programs. First, we extract unified optimization principles for high-performance sparse tensor programs and propose a domain-specific language (DSL) to automatically generate a high-quality design space without manual intervention. Second, by analyzing the general distribution of the design space, we introduce an adaptive search strategy combining deep Q-networks (DQNs) and simulated annealing (SA). Finally, to avoid the unacceptable time cost of real measurement during tuning, we propose a unified cost model based on multimodal fusion to accurately predict program performance. Furthermore, by leveraging data augmentation and transfer learning, we enable low-cost transfer prediction across different sparse data patterns, algorithms, and hardware platforms. Results show that, compared with the state-of-the-art operator library MKL, the manually optimized scheme ASpT, the tensor compiler TVM, and the sparse tensor tuning framework WACO, UniSparTa achieves average speedups of 1.98 & times;, 2.75 & times;, 6.13 & times;, and 1.75 & times;, respectively. Moreover, UniSparTa significantly accelerates the tuning process.

Tensorized programs leverage hardware intrinsics on accelerators to boost tensor computation performance. With the rise of hardware customization, massive accelerators and intrinsics have emerged, posing engineering challenges for manually coding efficient tensorized programs. Tensorized program tuning in deep learning compilers (DLCs) is considered an effective approach. At the core of program tuning relies the design of the cost model to predict performance. However, there is currently a lack of cost models specifically for tensorized programs, which severely hinders the co-optimization of deep learning compilers and hardware accelerators. In this paper, we propose UniCoX, a unified cost model specifically for tensorized program tuning across ubiquitous accelerators. We systematically analyze the design challenges introduced by tensorized programs from the perspectives of feature representation and transfer prediction. For feature representation, we leverage attention mechanisms to mine key software schedule features and design corresponding aligned hardware features, resulting in a unified cross-accelerator feature representation. For transfer prediction, by integrating lifelong learning and transfer learning with data sampling strategies, we propose a unified transfer prediction strategy to keep pace with the rapid development of accelerators. To meet training and testing demands, we construct TensorizeSetX, a dataset dedicated to tensorized program tuning. Results show that UniCoX achieves the state-of-the-art accuracy while supporting low-cost and flexible transfer prediction. It can accelerate search time by 11.3x and improve inference speed by 1.9x within the state-of-the-art tensorized program tuning framework, TVM MetaSchedule.

Researchers use high-level synthesis (HLS) to develop large-scale accelerators, leveraging extensive resources to handle intricate computational tasks. Recent Multi-Die FPGAs feature several super logic regions (SLR) within one device to meet the high resource demands of advanced computation. However, implementing a large-scale accelerator on multi-die FPGAs presents three obstacles. The first is the considerable delay penalty caused by the interconnectivity of accelerator modules across multi-die boundaries. The second is the congestion caused by modules’ excessive use of local resources. The third is that the increasing number of accelerator modules expands the design space, making it difficult to find the optimal design point. All of these pose limitations on the maximum achievable frequency for developing large-scale accelerator designs on multi-die FPGAs. Therefore, this paper proposes a frequency-oriented design exploration with the congestion-aware module mapping algorithm, FrqBooster. It combines modules’ location and resource consumption to provide coarse-grained floorplanning optimization. Firstly, we analyze the connections between accelerator modules and build optimization objectives. Secondly, we analyze the resource cost of each module in accelerator design and propose a two-stage resource-balancing strategy, combined with resource consumption, to alleviate local congestion. Finally, we design a congestion-aware algorithm to complete the overall exploration for achieving higher potential performance. In the experiments, results show that FrqBooster achieves maximum frequency improvements of 73.15% and 81.22% on the U250 and U280 platforms, respectively, compared to the default implementation in Vivado. Compared to related works on accelerator DSE scenarios, FrqBooster achieved a maximum improvement of 25.84×, resulting in better overall performance.

High-level synthesis (HLS) improves FPGA design productivity by compiling high-level programs into synthesizable hardware, but searching for high-quality hardware directive configurations remains expensive. Recent GNN-based predictors estimate latency and resource utilization for design space exploration, yet they often treat HLS prediction data as a flat distribution, making it difficult to capture both kernel-level shared structure and pragma-induced configuration-level variation. In addition, joint prediction of latency and multiple resource metrics introduces gradient conflicts in multi-task learning. We propose a two-stage GNN-based framework that learns HLS design points in a coarse-to-fine manner: it first organizes a stable kernel-level representation distribution and then refines pragma-guided configuration-level differences within that distribution. The framework combines Indirect Intra-Kernel Distance Loss (IIKDL) for Stage-1 regularization, a Residual Fitting Module (RFM) for Stage-2 refinement, and Magnitude-Aware PCGrad (MA-PCGrad) for stable joint optimization. Experiments show that, compared with HARP and Hier-MoE, our method reduces mean squared error by 41 https://github.com/Pahdor/DS2-HLS .

Low-bitwidth multiply-accumulate (MAC) operations are fundamental to efficient hardware acceleration of recent neural networks. Existing approaches often fail to fully exploit the potential of digital signal processing (DSP) blocks in FPGAs. They struggle to balance the high-bitwidth computational capabilities of DSPs with the low-precision quantization requirements of neural networks. DSP packing consolidates multiple low-precision operations into a single DSP unit, significantly enhancing MAC efficiency. However, current DSP packing solutions suffer from limited support for continuous operations, poor adaptation to different neural network computation patterns, and complex software-hardware deployment workflows. These issues result in insufficient utilization of DSP resources, limiting the overall effectiveness of acceleration. To address these challenges, we propose UDP, a universal DSP packing framework that effectively tackles the aforementioned issues. UDP optimizes both mapping and packing phases to fully utilize DSP resources, achieving near-optimal computational parallelism at the theoretical level. We tightly integrate algorithmic and hardware design through parameterized Chisel-based templates, simplifying the hardware deployment process. Furthermore, we observe 2.28-8.33x better throughput per DSP block and 2.59-5.31x energy efficiency compared with state-of-the-art DSP packing schemes. UDP is particularly well-suited for scenarios with urgent demands for low-bitwidth quantization, such as large language models (LLMs), offering broad application prospects. The related codes are publicly available at https://github.com/wintermelon008/UDP.

The remarkable performance of large language models (LLMs) has driven their widespread deployment in data centers to support diverse user-facing applications. However, the rapidly growing computational and storage demands of these models have made single-device deployment increasingly impractical. Prior research on LLM inference has primarily addressed this challenge through algorithmic optimizations such as quantization or by integrating customized hardware acceleration frameworks. As model parameters continue to scale, multidevice deployment has become a necessary approach for enabling efficient LLM inference. Nevertheless, constructing low-latency multidevice platforms for LLMs inference using available field-programmable gate array (FPGA) or GPU accelerators remains constrained by inefficient synchronization schemes or limited compute intensity in current architectures. Furthermore, existing solutions often lack co-optimized designs that effectively integrate communication with computation. To address these limitations, this article proposes latency-oriented recurrent architecture (LORA), a low-latency end-to-end LLMs acceleration platform utilizing multiple FPGAs. First, we optimize the synchronization timing within the LLMs to minimize storage, computation, and block random-access memory (BRAM) overhead. Second, we tightly couple communication and computation through techniques such as pipeline overlapping and input data packing. Next, we deploy homogeneous accelerators on each FPGA device, leveraging a recurrent architecture to further reduce inference latency. Finally, we apply FPGA-specific optimizations and conduct performance modeling and analysis of the acceleration framework to select optimal deployment parameters for various computational tasks. Implemented on Xilinx Alveo U280 FPGAs, LORA-F and LORA-Q achieve average speedups of 14.4 & times; and 32.6 & times;, respectively, compared with NVIDIA V100 GPUs when running modern LLMs. Compared with existing multi-FPGA accelerator platforms, LORA-F and LORA-Q demonstrate average performance improvements of up to 2.6 & times; and 4.3 & times;, respectively.

Heterogeneous multi-chiplet accelerators allow chiplets to be configured independently to better match different operator characteristics and improve inference efficiency. However, heterogeneity makes simulator evaluation expensive, limiting the number of iterations affordable for hardware design space exploration (HW-DSE). Mainstream data-driven methods rely mainly on final metrics and a few predefined states, and require many search iterations to implicitly learn the relationships between input parameters and optimization objectives, making them less effective in this setting. In practice, evaluators also generate detailed reports on execution timelines, resource utilization, memory accesses, and communication behavior. Large language models (LLMs) can combine domain knowledge with these reports to explicitly identify bottleneck locations, degradation causes, and parameter adjustment directions, thereby improving each design decision under limited iteration budgets. Based on this observation, we propose Beacon, a report-driven LLM multi-agent framework for heterogeneous multi-chiplet HW-DSE. Beacon employs hierarchical agents for bottleneck localization, root-cause diagnosis, and hardware candidate generation, together with an Analysis Toolbox and RAG memory for closed-loop search. Under the same limited iteration budget, Beacon reduces the composite latency-energy-monetary-cost objective by 25.1%–93.5% compared with random search, Bayesian optimization, and reinforcement learning.

To tackle power management challenges in deep neural networks (DNNs), dynamic voltage and frequency scaling (DVFS) has gained attention for its ability to enhance energy efficiency without modifying DNN structures. However, current DVFS methods, which rely on historical data, such as processor utilization and task load, suffer from issues like frequency ping-pong, response lag, and limited generalizability. These challenges are exacerbated by real-world scenarios that prioritize time, energy, or energy efficiency differently, making it even harder for existing methods to effectively configure DVFS under such multiobjective constraints or tradeoffs. This article presents multilens (MTL), a multiobjective adaptive DVFS framework. First, we propose a power-sensitive feature extraction method along with multiobjective constraint modeling to characterize DNN inference behavior. Second, critical power blocks are then identified through clustering based on inference behavior similarity, enabling adaptive DVFS instrumentation point settings. Moreover, to enhance the adaptability of multiple platforms and the flexibility of multiple scenarios, MTL integrates a complete deployment process. Experimental results demonstrate the effectiveness of the MTL in optimizing energy efficiency across different hardware platforms and deployment scenarios.

Cause-effect chains, as a widely used modeling method in real-time embedded systems, are extensively applied in various safety-critical domains. End-to-end latency, as a key real-time attribute of cause-effect chains, is crucial in many applications. But the analysis of end-to-end latency for cause-effect chains on multicore platforms with shared caches still presents an unresolved issue. Traditional methods typically assume that the worst-case execution time (WCET) of each task in the cause-effect chain is known. However, in the absence of scheduling information, these methods often assume that all shared cache accesses result in misses, leading to an overestimation of WCET and, consequently, affecting the accuracy of end-to-end latency. However, effectively integrating scheduling information into the WCET analysis process of the chains may introduce two challenges: first, how to leverage the structural characteristics of the chains to optimize shared cache analysis, and second, how to improve analysis accuracy while avoiding state space explosion. To address these issues, this paper proposes a novel end-to-end latency analysis framework designed for multi-chain systems on multicore platforms with shared caches. This framework extracts scheduling information and structural characteristics of cause-effect chains, constructing fine-grained and scalable inter-core memory access contexts at the basic block level for time-sensitive shared cache analysis. This results in more accurate WCET (TSC-WCET) estimates, which are then used to derive the end-to-end latency. Finally, we conduct experiments on dual-core and quad-core systems with various cache configurations, which show that under certain settings, the average maximum end-to-end latency of cause-effect chains is reduced by up to 34% and 26%.

Owing to the inherent advantages of edge computing in latency, privacy, and always-on availability, Deep Neural Network (DNN) tasks are progressively shifting to edge devices. However, limited edge resources pose significant challenges for deploying large DNNs. Pipeline parallelism (PP) and post-training quantization (PTQ) are essential techniques for addressing these resource constraints, and mainstream frameworks (e.g., DeepSpeed, PyTorch) encapsulate them independently. Nonetheless, this decoupled treatment inevitably traps the optimization process into local optima and results in inferior throughput-accuracy trade-offs in heterogeneous edge environments, due to the natural circular dependency between the PP and PTQ optimization processes. To this end, we propose MP3, a novel framework for joint PP-PTQ optimization in heterogeneous edge environments. MP3 integrates three core components: (1) Task-Device Co-analyzer (TDC) utilizes a fine-grained modeling method to estimate computation and communication latencies precisely. (2) Information-Fused Accuracy Proxy (FAP) characterizes quantized DNN accuracy through a unified proxy combining intra-layer and inter-layer information, with a data-driven fusion function search strategy. (3) Pipeline Quantization Co-optimizer (PQC), leveraging latency estimates from TDC and accuracy characterizations from FAP, navigates the joint optimization space effectively by formulating it as a Constraint Programming (CP) problem. A comprehensive experimental evaluation on an edge testbed confirms the effectiveness of MP3, demonstrating that it lies on a superior Pareto curve (achieving up to 3.2× throughput speedup with accuracy drop within 1%) and exhibits robust adaptability across heterogeneous devices.

Heterogeneous systems commonly adopt dynamic scheduling algorithms to improve resource utilization and enhance scheduling flexibility. However, such flexibility may introduce timing anomalies, wherein locally reduced execution times can lead to an increase in the overall system execution time. This phenomenon significantly complicates the analysis of Worst-Case Response Time (WCRT), rendering conventional analysis either overly pessimistic or unsafe, and often necessitating exhaustive state-space exploration to ensure correctness. To address this challenge, this paper presents the first timing-anomaly-free dynamic scheduling algorithm for heterogeneous systems, referred to as Deterministic Dynamic Execution. It achieves a safe and tight WCRT estimate through a single offline simulation execution. The core idea is to apply deterministic execution constraints, which partially restrict the resource allocation and execution order of tasks at runtime. Based on a formally defined execution progress model for heterogeneous system scheduling, we prove the correctness of the proposed execution constraints and their ability to eliminate timing anomalies. Furthermore, we propose two methods to generate execution constraints. The first method derives execution constraints directly from the execution traces produced by existing scheduling algorithms. The second method is a heuristic-based approach that constructs execution constraints, enabling further reduction of the WCRT. Experimental results on synthetically generated DAG task sets under various system configurations demonstrate that, compared to traditional dynamic scheduling algorithms, our approach not only eliminates timing anomalies but also effectively reduces both the WCRT and response time jitter.

Fixed-point DNN inference on FPGAs remains dominated by multiply-accumulate (MAC) operations, while the binary representations of quantized operands often contain considerable bit-level redundancy. Directly exploiting such redundancy on FPGAs is difficult due to the high overhead of fine-grained bit-wise parsing, irregular communication, and complex control. To address this challenge, this paper presents a coarse-grained encoded MAC architecture with software-hardware co-design for FPGA-based bit-level redundancy-aware inference. The proposed method adopts coarse-grained encoding to convert irregular effective-bit patterns into structured group-level entries, enabling exact computation while improving hardware regularity. Building upon this abstraction, we develop an encoding-driven PE array with relaxed communication, lightweight distributed control, and RAM-based data organization to coordinate computation under irregular encoded workloads. Experiments on a Xilinx Zynq UltraScale+ FPGA show that the proposed design achieves up to 1.92× higher effective MAC throughput, with consistent positive gains across representative CNN and Transformer models. Per-PE logic cost is reduced from 18 CLBs to 7 CLBs at the finest encoding granularity while preserving exact inference behavior. These results demonstrate that coarse-grained encoding provides an effective path to bridging redundancy exploitation and practical FPGA realization.

Field-Programmable Gate Arrays (FPGAs) are increasingly used in cloud platforms to accelerate diverse workloads, thanks to their reconfigurability and high performance. However, in multi-tenant cloud environments, existing FPGA virtualization mechanisms fail to align with dynamic application demands due to their static partitioning methods, leading to significant internal fragmentation. To address these issues, we present Tetris , an FPGA virtualization framework that supports flexible and dynamic reconfigurable resource allocation to improve cloud platform deployment efficiency. Specifically, Tetris deploys nested dynamic reconfigurable regions on FPGAs and adopts a tree structure to manage reconfigurable resources, enabling fine-grained resource reallocation at run-time. Enabled by the proposed fat tree-based data transmission architecture between reconfigurable regions, Tetris can map dataflow-based applications onto these regions effectively. Additionally, Tetris offers dual-level resource optimization strategies to help system to balance the resource utilization and run-time compilation overhead. We evaluate Tetris with dataflow HLS benchmarks. Experimental results show that Tetris achieves a 1.16 \(\times\) improvement in resource utilization compared to advanced FPGA virtualization frameworks and delivers a 1.3 \(\times\) increase in runtime throughput, incurring less than 30 % additional compilation latency. Tetris enables scalable, on-demand cloud FPGA acceleration, improving adaptability across different workloads.

Diffusion language models (DLMs) generate text through iterative denoising, but at each denoising step the model still performs attention over the full sequence, including many tokens that remain masked, resulting in substantial redundant computation. Existing acceleration methods typically require retraining the DLM and imposing constrained update patterns to reduce inference cost, which limits their direct applicability to pretrained DLMs. Our token-level analysis reveals pronounced structural locality in DLM inference. In practice, only a small set of tokens near the undecoded prefix tends to drive decoding, while distant undecoded tokens have limited influence and the retained local context can be effectively reused. Motivated by these observations, we propose Window-Diffusion (Code repository: https://github.com/vhicrgit/Window-Diffusion ), a window-based token pruning and caching method for inference. We maintain a local computation window that slides rightward as denoising progresses, and partition undecoded tokens into: (i) active tokens that are computed online, (ii) buffer tokens whose KV states are cached and periodically refreshed, and (iii) far-field tokens that are pruned outside the window. Computation is restricted to active and buffer tokens within the window, while far-field tokens are omitted at each stage. Extensive experiments across diverse benchmarks and model scales demonstrate the effectiveness of our method. Our approach achieves up to 6.6 × speedup with negligible quality degradation.

Personalized Federated Learning (PFL) customizes models for each client to mitigate challenges from non-IID data, wherein a dominant strategy is model decoupling that partitions models into shared and personalized parts based on architectural priors (e.g., backbone vs. head). However, we reveal a critical flaw in this strategy: it induces "intrinsic drift," a performance degradation often more severe than the well-known client drift, which limits final accuracy. We trace this drift to a steep cliff of high loss emerging from the naive stitching of shared and personalized parts. To address this, we shift from architectural partitioning to a parameter behavior-driven paradigm. We introduce PPFL, an approach that employs a novel soft-fusion strategy guided by parameter-wise behavioral perception. PPFL dynamically infers each parameter's functional role—whether it behaves more like a 'personalist' or a 'generalist' in the current context—by synthesizing its multifaceted behavior observed during local training. Extensive experiments on image, text, and multimodal classification benchmarks show that PPFL outperforms eight state-of-the-art baselines by up to 5.3%. Moreover, it can function as a plug-in module, boosting the accuracy of vanilla FedAvg with a 16.82% absolute gain.

Vision Transformers (ViTs) achieve strong accuracy but incur high inference latency. Semi-structured N:M sparsity can reduce arithmetic cost, yet its theoretical savings often fail to translate into proportional end-to-end speedups on modern GPUs. This mismatch arises because deployment latency depends not only on arithmetic reduction but also on execution regularity and hardware scheduling under sparsity. Achieving practical acceleration, therefore, requires coordinated design across sparse execution and sparsity configuration. To this end, we propose a hardware-software co-design framework for N:M sparse ViT inference. On the hardware side, we design MD-SpMM, an N:M sparse CUDA kernel that reorganizes sparse GEMM into micro-dense, Tensor-Core-aligned dataflow and uses inference-aware adaptive parallelism to sustain utilization. On the software side, we perform layer-wise sparsity search under explicit end-to-end latency budgets using a three-stage heuristic search with constraint relaxation to avoid premature convergence and enable deployment-aware sparsity allocation. Experiments on multiple ViT/Swin models and GPU platforms show that the framework achieves over 2.2 × latency speedup while maintaining comparable accuracy and delivering superior accuracy under the same latency constraint. The code will be publicly released.

The growing scale and complexity of large language models (LLMs) have intensified the need for optimizing largescale tensor computations (e.g., self-attention and mixture-ofexperts) on hardware platforms. Existing solutions rely on either manual expert optimization or exploration-based autotuning methods. However, neither approach scales effectively for LLMs with hundreds, even thousands of operators and dynamic control flows, because of prohibitive optimization overheads or suboptimal performance. To address this problem, we present QiMeng-Tensify, the first framework that combines LLMs with sequential decision optimization for large-scale graph-level tensor computation. Our key insight is that: (1) tensor computation optimization can be formulated as a generalized sequential decision problem to enlarge the optimization space, and (2) LLMs inherently encode rich optimization knowledge and can reason about architectural characteristics, which can effectively guide this decision process. Concretely, we first model tensor computation optimization as a Markov Decision Process (MDP), enabling unconstrained graph transformations over pre-defined scheduling rules. To efficiently explore the vast transformation space, we introduce an architecture-aware LLM-guided Monte Carlo Tree Search (MCTS). The LLM shapes the prior probability distribution over candidate transformations, guiding the search direction toward promising program sketches and parameter configurations. To adapt to concrete hardware and workloads, we propose an architecture-aware prior adaptation mechanism that distills natural-language heuristics from a lightweight offline stage. We conducted comprehensive experiments for representative subgraphs and LLMs on NVIDIA A100 and H100. Regarding subgraphs, QiMeng-Tensify averagely outperforms PyTorch, TensorRT, TVM, Triton, FlashAttention, Welder, Mirage, and Reasoning Compiler by $6.49 \times, 2.86 \times, 1.68 \times, 2.64 \times, 1.27 \times, 13.49 \times 1.29 \times$, and $1.31 \times$, respectively. For LLM workloads, QiMeng-Tensify achieves average speedups of 1.56×, 1.22× and 1.30× over PyTorch, TensorRT-LLM and Mirage on the A100, and $1.78 \times, 1.29 \times$ and $1.30 \times$ on the H100, respectively. Results well demonstrate that QiMeng-Tensify provides a generalizable paradigm for optimizing large-scale tensor computation.

Chain-of-Thought (CoT) reasoning in large language models (LLMs) significantly improves accuracy on complex tasks, yet incurs excessive memory overhead due to the long think-stage sequences stored in the Key-Value (KV) cache. Unlike traditional generation tasks where all tokens are uniformly important, CoT emphasizes the final answer, rendering conventional KV compression strategies ineffective. In this paper, we present Crystal-KV, an efficient KV cache management framework tailored for CoT reasoning. Our key insight is the answer-first principle. By mapping answer preferences into think-stage attention map, we distinguish between SlipKV, which mainly maintains the reasoning flow but may occasionally introduce misleading context, and CrystalKV, which truly contributes to the correctness of the final answer. Next, we propose an attention-based Least Recently Frequently Used algorithm. It precisely identifies when a SlipKV entry's utility expires and evicts it, retaining CrystalKV without disrupting reasoning flow. Finally, we introduce an adaptive cache budget allocation algorithm. Based on the dynamic proportion of CrystalKV, it estimates the importance of each layer/head and adjusts the KV cache budget during inference, amplifying critical components to improve budget utilization. Results show that Crystal-KV achieves state-of-the-art KV cache compression, significantly improves throughput, and enables faster response time, while maintaining, or even improving, answer accuracy for CoT reasoning.

Real-world Federated Learning (FL) deployments face the coupled challenges of data heterogeneity and long-tailed distributions. This creates a dilemma: prioritizing global long-tailed optimization yields a robust generic model but fails to adapt to client-specific heterogeneous distributions; conversely, relying solely on personalization is ineffective because the misalignment between global and local tail definitions prevents clients from learning critical shared representations. Existing approaches typically isolate these challenges, addressing either global imbalance or personalization on balanced data, thus failing to reconcile the conflict between generic robustness and local specificity. To address this, we propose FedGAMA (Federated Group-wise Asymmetric Masked Aggregation), a novel framework that decouples representation learning from classifier re-balancing to simultaneously enhance global and personalized performance. First, to resolve the misalignment of tail definitions, it employs a frozen Global Anchor Classifier (GAC) for robust feature extraction and a Global Probing Classifier (GPC) to estimate global distributions. Based on the GPC weight norms, it partitions head and tail classes to establish alignment of tail definitions for subsequent targeted optimization. Second, to reconcile the conflict between global and local optimization, it introduces Group-wise Asymmetric Masked Aggregation, which updates head classes universally while restricting tail updates to relevant clients, shielding tail knowledge from gradient noise. Extensive experiments demonstrate that FedGAMA outperforms state-of-the-art Federated Long-Tailed and Personalized FL methods.