Compute Express Link (CXL) has emerged as a key enabler of memory disaggregation for future heterogeneous computing systems to expand memory on demand and improve resource utilization. However, CXL is still in its infancy stage and lacks commodity products on the market, thus necessitating a reliable system-level simulation tool for research and development. In this article, we propose CXL-DMSim (open-sourced at https://github.com/ferry-hhh/CXL-DMSim), an open-source full-system (FS) simulator to simulate CXL disaggregated memory systems with high fidelity at a gem5-comparable simulation speed. CXL-DMSim incorporates a flexible CXL memory expander model along with its associated device driver, and CXL protocol support with CXL.io and CXL.mem. It can operate in both the app-managed (AM) mode and the kernel-managed (KM) mode, with the latter using a dedicated NUMA-compatible mechanism. The simulator has been rigorously verified against a real hardware testbed with both FPGA- and ASIC-based CXL memory devices, which demonstrates the qualification of CXL-DMSim in simulating the characteristics of various CXL memory devices at an average simulation error of 3.4%. The experimental results using LMbench and STREAM benchmarks suggest that the CXL-FPGA memory exhibits a $\sim 2.88\times $ higher latency than local DDR, while the CXL-ASIC latency is $\sim 2.18\times $ ; CXL-FPGA achieves 45%-69% of local DDR memory bandwidth, whereas the number for CXL-ASIC is 82%-83%. The study also reveals that CXL memory can significantly enhance the performance of memory-intensive applications, improving by $23\times $ at most with limited local memory for Viper key-value database and approximately 60% in memory-bandwidth-sensitive scenarios such as MERCI. Moreover, the simulator's observability and expandability are showcased with detailed case studies, highlighting its great potential for research on future CXL-interconnected hybrid memory pools.

Value prediction is a microarchitectural technique that improves instruction-level parallelism by speculatively predicting the outcomes of instructions and eliminating data dependencies. While local value predictors have seen substantial progress by exploiting context-sensitive value locality, global value prediction remains relatively underexplored due to practical limitations such as inaccurate prediction, value delay constraint, and excessive hardware cost. This paper revisits global value prediction with a focus on addressing these long-standing challenges and presents EgDiff, a redesigned global predictor that introduces aggressive confidence mechanisms tailored for deep pipelines, performs non-speculative updates using committed values to preserve critical correlations, and employs deferred prediction to avoid premature decisions based on incomplete global history. To reduce hardware overhead, EgDiff uses a distance polling technique that compresses the predictor table by over 95% without sacrificing accuracy. Experimental results show that EgDiff achieves more than 99% accuracy and delivers a 4.37% average IPC speedup. When combined with EVES, a state-of-theart local predictor, a compact 19KB hybrid achieves a 6.16% speedup, already surpassing 32KB EVES alone (4.81%). With unlimited storage, the hybrid further reaches a $\mathbf{7. 0 2 \%}$ speedup. These findings demonstrate that global and local value prediction are complementary, and that revisiting global prediction with modern design principles offers practical and significant benefits in processor.

Over the past two years, the rapid evolution of Large Language Models (LLMs) and the relatively slower progress in hardware development have led to the emergence of many operator-level optimization techniques and theories. These software-based methods accelerate model inference by improving memory management and computational efficiency, and they have demonstrated empirical effectiveness. However, in current computer architecture research, hardware designers often focus on microarchitectural optimizations or the performance of individual operations. They tend to adopt existing software optimizations passively, without actively leveraging them as design principles. Constrained by this limited perspective, researchers may overlook opportunities to systematically integrate insights from software into hardware design, potentially hindering more efficient and flexible architectural innovations. To combine operator-level optimization methods with hardware design, we introduce a hardware evaluation platform called LLMSGHD ( L arge L anguage M odel S oftware- G uided H ardware D esign), which focuses on operator-level optimized LLM inference workloads. LLMSGHD simulates hardware inference behavior while aiming for broad applicability and high efficiency. LLMSGHD integrates advanced software optimization techniques to offer more insightful analysis for hardware design, particularly regarding computational density variations in inference and their interaction with software-level optimizations. Based on LLMSGHD, we develope a heterogeneous LLM inference platform targeting high throughput and low cost.

Deep-learning accelerators such as TPUs and GPUs now run ever larger models. Conventional fault-tolerant accelerators, designed for CNNs, are ineffective and cost-prohibitive for emerging large language models (LLMs) due to their exponentially higher computational demands. To address the impact of soft errors on computation, various specialized fault-tolerant DNN accelerators have been proposed, typically employing full-element protection. Yet, emerging LLMs exhibit exponentially higher computational demands compared to traditional CNN models, rendering conventional fault-tolerant accelerators both cost-prohibitive and ineffective in handling multi-point faults. To tackle these challenges, we conduct fault injection experiments on multiple representative LLMs, revealing the inherent parameter redundancy in Transformer-based models. Specifically, only 1%–2% of the elements significantly affect the output when perturbed—these critical elements are identified as outliers. Leveraging this insight, we propose OrCA, a hierarchically redundant fault-tolerant accelerator, which introduces the principle of selective protection for critical elements and optimizes the dataflow accordingly. Through extensive fault injection experiments and hardware simulations, we demonstrate that OrCA outperforms conventional fault-tolerant accelerators, achieving superior protection at equal or lower area overhead. Notably, OrCA delivers better performance under fault rates up to 10× higher and supports elastic protection against diverse hardware faults (e.g., transient and permanent faults), adapting to varying fault-tolerance requirements. Furthermore, OrCA breaks the limitation of traditional accelerators that require separate error detection and correction steps, enabling more efficient fault resilience.

Network-on-Chip (NoC) is a key technology for communication in chip multiprocessors and multiprocessor system-on-chips. However, traffic in Network-on-Chip is often unevenly distributed, which may lead to congestion on certain links while leaving others underutilized. Inserting long-range links into NoC topologies can help balance traffic by diverting packets away from congested regions. However, the effectiveness of this approach strongly depends on the position, length, and combination of these links, which together constitute a large and complex design space. To address this issue, this paper introduces LIRL-NoC, a reinforcement learning-based method for long-range link insertion. LIRL-NoC leverages a graph convolution network to encode the NoC architecture and a deep reinforcement learning agent to effectively explore the extensive design space. In addition, it performs buffer capacity allocation to balance performance against power consumption. Experimental results show that compared to the baseline mesh topology, LIRL-NoC reduces latency by 21.00% at the cost of a 2.25% increase in power consumption; compared to state-of-the-art work, it achieves a 5.59% reduction in latency with similar power consumption.

AI workloads increasingly demand high parallelism and efficient on-chip memory utilization on modern GPUs. However, the limited capacity of shared memory often constrains thread-level parallelism, while portions of the L2 cache remain underutilized. In this work, we propose SAI , a mechanism that virtualizes shared memory into the L2 cache to improve GPU performance for AI applications. SAI dynamically activates virtualization based on runtime resource usage, enabling additional CTAs with minimal architectural modifications. We further introduce an L2 cache management strategy that integrates associativity-based virtual page allocation and a replacement information table, reducing page-swapping overhead while preserving L2 cache performance. Experimental results demonstrate that SAI achieves an 18.6% performance improvement over the baseline design and outperforms SMILE-opt, our reproduced state-of-the-art baseline, by 8.8%. Moreover, SAI captures 60.2% of the performance gain delivered by the idealized Double SMEM design while reducing energy consumption to 87.7% of the baseline. These results highlight the effectiveness of SAI in enhancing thread-level parallelism and optimizing on-chip memory for AI workloads.

The victim cache was originally designed as a small-capacity secondary cache to hold recently evicted lines from the L1 data cache in CPUs. However, this design is often suboptimal for graphics processing units (GPUs) due to the following two specific features of GPUs. First, a significant number of GPU applications have low hit rates in the L1 data cache, while the probability of lines evicted from the L1 data cache being subsequently reused is high. This means that the number of memory requests that hit in the victim cache can approach or even exceed the number of requests that hit in the L1 data cache. Second, GPUs use a unified architecture for the L1 data cache and shared memory, or the L1 cache and texture cache, to save hardware overhead. However, this hardware architecture significantly increases the L1 data cache access latency. We observe that some applications are susceptible to access latency, and increasing latency can severely degrade performance. To this end, we introduce HIVE+, an enhanced high-priority victim cache designed to accelerate GPU memory accesses. The key idea of HIVE+ is to exploit idle registers as the data array of the victim cache and place the tag array of the victim cache at a higher level in the memory hierarchy than the L1 data cache. Our experimental results show that HIVE+ achieves an average performance improvement of 47.5% and 35.0% compared to the baseline (BL) and Linebacker architectures, respectively.

Modern GPUs rely on private per-SM L1 caches and a shared L2 cache, but this organization obscures cross-SM reuse: an L1 miss is typically forwarded to L2 even when the requested line already resides in a peer L1 cache, leading to redundant L2 access. Prior GPU L1-sharing designs attempt to recover such reuse through exact or broad remote-hit searches, which become increasingly difficult to scale and can interfere with the critical L1 miss path under high concurrency. Based on this insight, we propose C2P-Cache, a scalable GPU L1-sharing mechanism that transforms remote-hit discovery from a chip-wide exact search problem into a lightweight filtering-and-confirmation process. C2P-Cache maintains compact Bloom-filter-based snapshots of private L1 tags, performs parallel chip-wide candidate filtering, and selectively probes only a small number of likely peer caches. To sustain high concurrency, C2P-Cache organizes filtering as bit-sliced matching over a banked and replicated snapshot matrix, enabling efficient, parallel processing of many concurrent misses without interfering with normal L1 accesses. Across a wide range of GPU workloads, C2P-Cache improves instructions per cycle (IPC) by up to 49.7% and by 23.5% on average for applications with high remote-L1 reuse and strong sensitivity to L2 latency, demonstrating that lightweight, scalable filtering can effectively unlock cross-SM reuse with modest overhead.

The pursuit of high-performance PDE solvers rests on three fundamental challenges: (i) the curse of dimensionality in kinetic and financial equations, (ii) the poor extrapolation of purely data-driven surrogates, and (iii) the widening gap between algorithm design and hardware specialization. To overcome these challenges, we present NeuroPDE+, a scalable neuromorphic PDE solver design based on spintronic and ferroelectric devices for accelerating PDE solutions. NeuroPDE+ consists of two dedicated units: a diffusion tracking unit (DTU), which emulates random walks on Markov chains through activations between hardware neurons, and a scattering tracking unit (STU), which samples non-local jumps via a multi-level probability tree. System-level simulations suggest that NeuroPDE+ achieves a squared error below 1e-2 in steady-state heat equation and particle transport problems. Simulation results further indicate that the DTU achieves up to a 315x performance gain over previous neuromorphic processors, and that the STU achieves a 1000x speedup compared to a general-purpose CPU. Co-designing algorithm and hardware with intrinsic stochasticity and non-volatile in-memory computing, NeuroPDE+ preliminarily explores a new paradigm for efficient and scalable neuromorphic PDE solvers. This approach could pave the way for probabilistic computing architectures in large-scale scientific simulations.

The processing-in-memory (PIM) architecture becomes a promising candidate for deep learning accelerators by integrating computation and memory. Most PIM-based studies improve the performance and energy efficiency by using the weight stationary (WS) data flow due to its high parallelism. However, the WS data flow has some fundamental limitations. First, the WS data flow has huge activation movements between on-chip memory and off-chip memory due to the limited memory space of the resistive random-access memory (ReRAM) array. Second, the WS data flow needs to read the input activation repeatedly according to the convolution window. These data movements decrease the energy efficiency and performance of the PIM architecture. To address these issues, the input stationary (IS) data flow stores activations instead of weights to reduce data movements. But the IS data flow faces some challenges. First, the data dependency between adjacent layers limits the performance. Second, there are huge across-array computations due to the special mapping method. Third, the previous IS data flow cannot realize the high parallelism. Fourth, the IS data flow depends on the 3-D ReRAM structure. To address these issues, we propose a novel data flow for PIM architectures. We optimize the IS data flow to decrease the activation movement and propose a parallel computing method to realize high parallelism and reduce the across-array computations. We identify and analyze the fundamental limitations and impact of different interlayer data flows, including the WS-WS, IS-IS, WS-IS, and IS-WS. We also propose a method to build a hybrid data flow by combining these interlayer data flows to tradeoff performance and energy consumption. Our experimental results and analysis demonstrate the potential of our design. The performance and energy efficiency of our design reach 0.13-1.77 TFLOPS and 61-85 TOPS/J, respectively. Compared to the state-of-the-art design, the NEBULA, our design can improve performance by 1.4x,2.3x, and 3.5xfor deploying the MobileNet-V1, ResNet-18, andVGG-16, and also can improve energy efficiency by 3.3x,2x,and 2x, respectively.

Particle transport simulations using stochastic methods face significant challenges on conventional von Neumann architectures,particularly due to random branching events and irregular memory access patterns.These limitations stem from the fundamental mismatch between probabilistic algorithms and deterministic computing paradigms.To bridge the gap between architecture and algorithms,a probabilistically tunable true random number generator was developed based on spintronic and ferroelectric devices.The physical randomness of spintronic devices was leveraged to provide a physical random source for the architecture,and the throughput of random bits was enhanced through optimized control logic and writing mechanisms.Next,programmable synapses were designed based on the memristive properties of ferroelectric devices,enabling non-volatile continuous weight storage with tunable probabilities.The experimental results indicate that the proposed approach achieves performance improvements ranging from 171 to 1 028 times compared to a general-purpose CPU when solving a sample transport problem.Furthermore,compared to existing spin-transfer torque magnetic tunnel junction based true random number generators,the developed method not only enables tunable probability random sampling but also achieves a throughput of 303 Mbit/s when generating uniformly distributed random sequences.

Deadlock-free adaptive routing is extensively adopted in both on-chip and off-chip interconnection networks to improve communication bandwidth and reduce latency. Introducing virtual channels (VCs), also known as virtual lanes (VLs), is the mainstream technique to handle deadlocks incurred by adaptive routing, and provides VC preemption for higher priority traffic. However, existing deadlock-free flow control schemes either underutilize memory resources due to inefficient buffer management to simplify hardware implementation, or rely on complicated global coordination and synchronization with very high hardware complexity. Most hardware-friendly schemes use more VCs and memory resources to enable ease of implementation of deadlock-free flow control. In contrast, sophisticated schemes achieve deadlock freedom with minimum VC cost, even eliminating additional buffer requirement through the complicated control mechanisms. In this work, we rethink the root cause of the deadlock problem from a different perspective by considering it as a lack of credit, which makes us find an efficient solution to the deadlock problem. With minor modification of credit accumulation and return, our proposed bubble-swap flow control (BSFC) ensures atomic buffer swap between two adjacent routers only based on local credit status while making full use of the buffer space. BSFC achieves a better tradeoff between implementation complexity and memory overhead and can be easily integrated in the industrial router with no modification on buffer allocation or port arbitration. The simulation results demonstrate BSFC outperforms existing bubble-based deadlock-free methods by average 64% higher throughput. We further propose a credit reservation strategy to eliminate the escape virtual channel (VC) cost for fully adaptive routing implementation. The synthesizing results demonstrate that BSFC along with credit reservation (BSFC-CR) can reduce the area and power consumption by respectively 29% and 26% in contrast to the traditional critical bubble scheme (CBS).

Deep learning becomes increasingly popular, and its main workload is Sparse-Sparse Matrix Multiplication (SpMSpM). Most SpMSpM accelerators usually only support a single dataflow. Different dataflows have different performance in different computing environments. Therefore, the single-dataflow accelerator cannot maintain the highest performance in all environments. Compared with single-dataflow accelerators, multi-dataflow accelerators provide flexible options for different workloads and improve the overall performance. Flexagon, Sparm, and SPADA are state-of-the-art multi-dataflow accelerators. However, the computation process of Flexagon and Sparm is not fully pipelined, and SPADA cannot support inner product dataflow. Additionally, Flexagon, Sparm, and SPADA cannot switch dataflows quickly and accurately. Inspired by these observations, we present SpMARD, a SpMSpM accelerator with reconfigurable dataflow. The computation process of SpMARD is fully pipelined, and SpMARD can support six dataflow variants simultaneously. Through the design of a Two-stage Pipeline Adder Network (TPAN) and a Position-based Psum Array (PPA), SpMARD can execute element-level merging, which can hide the merging overhead. Through the quantitative analysis of dataflows, we implement a Dataflow Switcher (DSwitcher), which can switch dataflows more efficiently. For the SpMSpM workload, the performance (GOPS) of the SpMARD we proposed is 1.27 times that of Flexagon, 1.18 times that of Sparm, and 1.22 times that of SPADA.

Large Language Models (LLMs) inference have gained popularity over the past two years, driven by their high performance achieved through rapid increases in the number of parameters. The inference process of LLMs consists of two distinct stages: prefill and decode, each with unique computational characteristics. While existing neural network inference platforms, such as Google's TPU, perform well during the prefill stage, they often suffer from poor resource utilization during the decode stage. To address this challenge, we propose the scalable Headtile architecture, specifically designed to improve hardware resource utilization. By analyzing the inference behavior of LLMs, we examine how each layer executes on TPUv3 and introduce the Maarg paradigm in Headtile for inter-layer scheduling and mapping. Experimental results show that Headtile can achieve up to 24 × higher throughput in the decode stage compared to TPUv3. In addition, the Maarg paradigm reduces memory accesses by up to 60 % during the prefill stage.

The Processing-In-Memory (PIM) architecture becomes a promising candidate for realizing energy-efficient CNN acceleration by integrating computation and memory. For deploying a CNN, the PIM-based designs need a scheduling scheme to translate massive hardware resources into actual performance. The scheduling scheme includes the intra-layer scheduling scheme to map data of one layer and the inter-layer scheduling scheme to allocate resources for multiple layers. Currently, the research of scheduling schemes for PIM-based studies is in its early stages and faces the following limitations. First, the intra-layer scheduling scheme mainly uses the Weight Stationary (WS) data flow to avoid weight movements. However, the amount of activation movement is larger than that of the weight movement for some DNNs or layers, making the activation movement become the bottleneck for reducing latency and energy consumption. Second, the Input Stationary (IS) data flow provides new opportunities to further reduce activation movements. However, the traditional IS data flow introduces a huge amount of across-crossbar computations and relies on the 3D-ReRAM structure. Third, the layer-wise processing structure of the DNN introduces complex inter-layer data dependence and imbalance workload, decreasing pipeline efficiency. Fourth, there is little study on co-optimization of intra-layer and inter-layer scheduling schemes. Inspired by these observations, we propose the Hybrid data flow based Intra-layer and inter-layer Scheduling schemes Optimization framework for PIM-based architectures, named HiSo. Specifically, we propose a novel data flow, named PP-IS, that replaces the convolution unrolling with selecting activations according to convolution windows to reduce activation movements. We also define and optimize the partition methods to improve data reuse and computational parallelism for the WS and PP-IS data flows. Then, we trade off performance and energy efficiency by using the hybrid data flows. Finally, we further improve energy efficiency and performance by co-optimization of intra-layer and inter-layer scheduling schemes. Our experimental results and analysis demonstrate the potential of the HiSo. Compared to the state-of-the-art design, the NEBULA, the HiSo can significantly improve energy efficiency, performance, and power efficiency by 1.03× ∼ 17.98×, 1.6× ∼ 50.7×, and 1.2× ∼ 229×, respectively.

Biologically inspired computing models have made significant progress in recent years, but the conventional von Neumann architecture is inefficient for the large-scale matrix operations and massive parallelism required by these models. This paper presents Spin-NeuroMem, a low-power circuit design of a Hopfield network for the function of associative memory. Spin-NeuroMem is equipped with energy-efficient spintronic synapses which utilize magnetic tunnel junctions (MTJs) to store weight matrices of multiple associative memories. The proposed synapse design achieves as low as 17.4 × speedup with a comparable associative memory effect. By harnessing the potential of spintronic devices, this work paves the way for the development of energy-efficient and scalable neuromorphic computing systems.

In recent years, new methods for solving partial differential equations (PDEs) such as Monte Carlo random walk methods have gained considerable attention. However, due to the lack of hardware-intrinsic randomness in the conventional von Neumann architecture, the performance of PDE solvers is limited. In this paper, we introduce NeuroPDE, a hardware design for neuromorphic PDE solvers that utilizes emerging spintronic and ferroelectric devices. NeuroPDE incorporates spin neurons that are capable of probabilistic transmission to emulate random walks, along with ferroelectric synapses that store continuous weights non-volatilely. The proposed NeuroPDE achieves a variance of less than 1e-2 compared to analytical solutions when solving diffusion equations, demonstrating a performance advantage of 3.48x to 315x speedup in execution time and an energy consumption advantage of 2.7x to 29.8x over advanced CMOS-based neuromorphic chips. By leveraging the inherent physical stochasticity of emerging devices, this study paves the way for future probabilistic neuromorphic computing systems.

Resistive Random Access Memory (ReRAM) based architectures have shown great potential for realizing energy-efficient Deep Neural Network (DNN) acceleration. When deploying a DNN, the ReRAM-based designs need a scheduling scheme to translate massive hardware resources into actual performance. Different scheduling schemes would lead to different levels of data reuse and computational parallelism, resulting in different energy efficiency and performance. However, the ReRAM-based scheduling scheme faces the following limitations. First, current studies mainly focus on intra-layer scheduling scheme optimizations by using the Weight Stationary (WS) data flow. These studies ignore the difference between layers and limit optimization opportunities. Second, there is no systematic definition and analysis for inter-layer scheduling schemes. Third, there is no co-optimization study on intra- and inter-layer scheduling schemes. Fourth, the complex network structure leads to intricate inter-layer data dependency, making the optimization of the scheduling scheme more challenging. These limitations restrict the comprehensive understanding of the scheduling schemes. Inspired by these observations, we identify the fundamental impact of intra-layer scheduling schemes on ReRAM-based designs, including the WS and Input Stationary (IS) data flows. We also systematically define and analyze inter-layer scheduling schemes according to different combinations of data flows, including the WS-WS, IS-IS, WS-IS, and IS-WS data flows. We analyze and explore different resource allocation strategies for these schemes. We also propose the intra- and inter-layer co-optimization to further improve performance and energy efficiency. Then, we propose a method for building a hybrid scheduling scheme by flexibly combining these inter-layer scheduling schemes for complex networks. Finally, we seek the potential to improve performance and energy efficiency for hybrid scheduling schemes. For deploying the MobileNet-V1, ResNet-18, VGG-16, and AlexNet, the hybrid scheduling scheme improves performance by 10.2x similar to 130.7x,1.9x similar to 16.5x, 7.0x similar to 56x, and 1x similar to 153.1x than the WS-WS, IS-IS, IS-WS, and WS-IS based scheduling schemes, respectively. Similarly, the power efficiency can also be increased by 15x and 26x than the WS-WS and IS-IS based scheduling schemes, respectively.

The victim cache was originally designed as a secondary cache to handle misses in the L1 data (L1D) cache in CPUs. However, this design is often sub-optimal for GPUs. Accessing the high-latency L1D cache and its victim cache can lead to significant latency overhead, severely degrading the performance of certain applications. We introduce HIVE, a high-priority victim cache designed to accelerate GPU memory accesses. HIVE handles memory requests first, before they reach the L1D cache. Our experimental results show that HIVE achieves an average performance improvement of 77.1% and 21.7% compared to the baseline and the state-of-the-art architecture, respectively.

Value prediction is a microarchitectural technique that enhances processor performance by speculatively breaking true data dependencies. It has demonstrated improved performance in both single-threaded and multi-threaded workloads, rendering it an appealing microarchitectural approach. While high-performance value predictors can achieve impressive accuracy, they may also incur significant costs in terms of area, power consumption, and complexity. Therefore, there is a demand for lightweight value prediction techniques capable of striking a favorable balance between performance and overhead. However, designing value predictors with superior performance using limited resources presents an urgent challenge, as inappropriate parameter configurations may result in cost overruns and degraded processor performance. Consequently, this work proposes a design space exploration framework for the state-of-the-art EVES value predictor, aiming to efficiently configure the design parameters of the value predictor within constrained RAM resources. Additionally, the article evaluates the performance of the explored value predictor across a wide range of workloads. The explored value predictors exhibit high efficiency across RAM sizes ranging from 2KB to 16KB while maintaining acceptable computational complexity. Furthermore, the results indicate that the explored value predictor achieves optimal efficiency under the 2KB constraint, with the highest acceleration-to-cost ratio reaching 8.74% per KB, approximately three times greater than that of the current state-of-the-art value predictor.