In-memory computing (IMC) is considered as a promising technique to alleviate the memory wall bottleneck. Various specialized IMC-based accelerators have been proposed recently to accelerate applications in different fields. In this paper, we propose GPA, a general-purpose accelerator based on the IMC technique. GPA is a massively parallel architecture that works in single-instruction-multiple-data (SIMD) manner. It is composed of a three-level hierarchy of computation and storage units (PE-core-array), where the IMC arrays are elaborated to support various operations on both integers and floating-points. We propose four working phases to avoid the potential structural conflict problem in IMC accelerators. An instruction set is also developed for GPA as a programming interface. Compared with an NVIDIA V100 GPU, evaluation results show that GPA achieves 161.28×, 169.66×, 73.18×, and 579.16× speedups and 6.23×, 6.66×, 24.16×, and 412.37× energy savings in four different applications.

Sparse Matrix-Dense Vector Multiplication (SpMV) is an important computational primitive which is bounded by memory bandwidth. Computing-in-memory (CIM) is regarded as an effective approach to reduce data movement. Due to the lack of flexibility in architectural design, current CIM-based SpMV accelerators struggle to simultaneously support high-parallelism computations and the storage of irregular sparse data. We propose a flexible CIM-based accelerator named MACAM for high-precision SpMV. Each array of MACAM can be configured into sparse or dense modes according to the local-sparsity of the sparse matrix. We propose a unified data layout approach that enables MACAM to meet the data storage requirements of different modes. We also propose a sparse storage format and a workload-balancing approach to further improve the performance of MACAM. Experiments show that MACAM achieves 167.26× speedup and 286.04× energy saving over the GPU baseline. MACAM also achieves 97.41× and 6.56× speedup and 213.65× and 10.06× energy saving compared with two state-of-the-art CIM-based SpMV accelerators.

Large language model (LLM) inference is increasingly limited by the capacity of High-Bandwidth Memory (HBM) in GPUs, as model weights and KV cache grow rapidly. High-Bandwidth Flash (HBF) provides higher capacity than HBM while offering comparable bandwidth, making it a promising substrate for capacity-constrained LLM inference. However, its inherently high access latency, low bandwidth utilization, and lack of support for heterogeneous resource management make it difficult to integrate HBF into GPUs for LLM inference. We present FlashAccel, a co-designed system that enables efficient LLM inference using HBF. FlashAccel integrates HBF into HBM-based GPUs, providing architectural support to mitigate access latency. It improves bandwidth utilization through specialized data layouts for both model weights and KV cache, and introduces an HBF-aware storage management layer together with a programming model to organize persistent data in HBF and coordinate heterogeneous memory resources at the system level. Experimental results demonstrate that integrating six HBF stacks into the GPU enables FlashAccel to deliver an average improvement of 2.49× and 1.93× in throughput per GPU and energy efficiency over the HBM-only GPU under a 100ms latency constraint, respectively.

Large language models (LLMs) have achieved remarkable performance across a wide range of natural language processing tasks, yet their growing parameter counts and memory demands pose substantial challenges to efficient deployment. Block floating-point (BFP) quantization has emerged as a compelling solution, offering enhanced numerical precision over integer formats while reducing memory overhead via shared exponent encoding. However, a critical yet overlooked source of precision degradation originates from excessive mantissa right-shifting during shared exponent alignment, particularly under low-bitwidth settings. In this work, we propose Dual-Stage Mantissa Shifting (DSMS), a precision-enhancing BFP quantization framework that selectively decomposes the mantissa right-shifting process into two stages: shift encoding and post-multiplication (Post-M) alignment. In the shift encoding stage, small shifts are applied immediately, whereas large shifts are partially deferred to preserve low-order bits. During inference, the deferred shifts are performed after multiplication using the inherently widened bitwidth of multiplier output, thereby mitigating premature truncation and improving inference accuracy. Extensive evaluations show that DSMS improves inference accuracy while maintaining high energy efficiency compared with state-of-the-art baselines.

Resistive memory compute-in-memory accelerators provide energy efficient analogue matrix vector multiplication for neural network inference, but frequent reprogramming of analogue weights remains costly because of device variability and iterative write and verify operations. This limitation hinders their use in edge model adaptation, including approximate machine unlearning and continual learning, where model parameters may need to be updated repeatedly in response to data deletion requests or newly arriving tasks. Here we present a co-design approach across hardware and software that maps frozen pretrained weights to analogue resistive memory arrays while placing trainable low rank adaptation branches in SRAM connected digital compute. By using LoRA style parameter efficient updates, the proposed scheme confines adaptation to a small set of digital parameters and avoids repeated reprogramming of the analogue backbone. To our knowledge, this work provides the first experimental demonstration of approximate machine unlearning on a fabricated resistive memory CIM accelerator. We validate the framework on a 180 nm 128x128 1T1R resistive-memory macro for face recognition, and through circuit-accurate simulations for speaker authentication and stylized image generation tasks, owing to the substantial model sizes involved. Compared with a baseline that directly updates analog weights, our hybrid mapping reduces analog training/update cost by up to 148x, on-chip deployment overhead by up to 388x, and inference energy by up to 59x, while preserving competitive task performance. These results show that hybrid analogue-digital LoRA mapping can enable efficient post-deployment adaptation on RM-CIM hardware, although formal machine-unlearning guarantees and large-scale system integration remain open challenges.

The design of XPU-PIM integrated Domain-Specific Accelerators relies heavily on high-performance simulators. However, existing simulation tools often suffer from limited integration capability and insufficient simulation speed. To address these challenges, this paper proposes HOPESim, a lightweight and fast simulator specifically designed for XPU-PIM accelerators. We introduce a Register Synchronous Simulation Model (RSSM), which abstracts multiple Processing Elements (PEs) and their interconnections into parallel blocks, enabling seamless integration with third-party simulators through a modular structure. A non-global timestamp strategy is proposed to break cycle dependencies between components, while a decoupled lightweight coroutine library leveraging modern C++ features is adopted to enhance large-scale simulation performance. Experimental results demonstrate that HOPESim achieves over 27× speedup compared to RTL simulation while maintaining compatibility with third-party simulation tools.

Although processing-in-memory (PIM) techniques have widely been used for deep neural networks (DNNs) acceleration, the inference performance of aged PIM-based accelerators remains to be investigated. This paper makes the first attempt to study Hot Carrier Injection (HCI) and Negative Bias Temperature Instability (NBTI) aging impacts on SRAM-based DNN accelerators, which provides a novel and unified framework, termed AgeBalance for aging detection, analysis and mitigation. First, we discuss a convenient aging detection scheme. Then, we benchmark the inference accuracy drops of DNNs running on aged SRAM-based PIM accelerators. Finally, we propose a low-cost anti-aging training method without incurring additional hardware overhead on SRAM-based DNN accelerators. Extensive experimental results on MNIST, CIFAR10 and AG News datasets show that aging can cause the inference accuracy of shallow or deep DNNs to drop to about 10%, close to random guessing. The aging mitigation scheme proposed in this paper can largely restore the accuracy to the original. Moreover, the SRAM write overhead of our method is much reduced thanks to a score-based training approach, leading to a reduction of 5× to 10× writing energy compared to the traditional training method.

The integrated architecture that features both in-memory logic and host processors, or so-called "processing-in-memory" (PIM) architecture, is an emerging and promising solution to bridge the performance gap between the memory and host processors. In spite of the considerable potential of PIM, the workload offloading policy, which partitions the program and determines where code snippets are executed, is still a main challenge in PIM. In order to determine the best PIM offloading partitions, existing methods require in-depth program profiling to create the control flow graph (CFG) and then transform it into a graph-cut problem. These CFG-based solutions depend on detailed profiling of a crucial element, the execution time of basic blocks, to accurately assess the benefits of PIM offloading. The issue is that these execution times can change significantly in PIM, leading to inaccurate offloading decisions. To tackle this challenge, we present a novel PIM workload offloading framework called "RDPIM" for CPU-PIM graph processing accelerators, which systematically considers the variations in the execution time of basic blocks. By analyzing the relationship between data dependencies among workloads and the connectivity of input graphs, we identified three key features that can lead to variations in execution time. We developed a novel reuse distance (RD)-based model to predict the exact performance of basic blocks for optimal offloading decisions. We evaluate RDPIM using real-world graphs and compare it with some state-of-the-art PIM offloading approaches. Experiments have demonstrated that our method achieves an average speedup of 2x compared to CPU-only executions and up to 1.6x compared to state-of-the-art PIM offloading schemes.

Deploying deep neural networks (DNNs) on conventional digital edge devices faces significant challenges due to high energy consumption. A promising solution is the processing-in-memory (PIM) architecture with resistive random-access memory (RRAM), but RRAM-based systems suffer from imprecise weights due to programming stochasticity and cannot effectively utilize conventional weight encryption/decryption intellectual property (IP) protection schemes. To address these issues, we propose a novel software-hardware co-design Guarder. On the hardware side, we introduce 3T2R cells to achieve reliable multiply-accumulate (MAC) operations and use reconfigurable inverter operating voltages to encode keys for encrypting DNNs on RRAM. On the software side, we implement a contrastive training method that ensures high model accuracy on authorized chips while degrading performance on unauthorized ones. This approach protects DNN IP with minimal hardware overhead while significantly mitigating the effects of RRAM programming stochasticity. Extensive experiments on tasks such as image classification (using MLP, ResNet, and ViT), segmentation (using SegFormer), and image generation (using DiT) validate the effectiveness of our method. The proposed contrastive training ensures negligible performance degradation on authorized chips, while performance on unauthorized chips drops to random guessing or generation. Compared to traditional RRAM accelerators, the 3T2R-based accelerator achieves a 1.41x reduction in area overhead and a 2.28x reduction in energy consumption.

The widening performance gap between processors and memory has become a major bottleneck in modern computing systems, highlighting the importance of cache performance. Traditional cache replacement policies primarily exploit data locality but often neglect the critical impact of concurrency. In fact, modern caching techniques support concurrent data accesses by servicing multiple accesses concurrently. Due to concurrency, the cost of cache misses varies significantly, offering an opportunity to enhance cache replacement by prioritizing the reduction of costly misses. In this paper, we introduce a perceptron-based concurrencyaware miss cost predictor (CAMP) to enhance locality-based cache replacement decisions. CAMP predicts the actual cost of cache misses in environments with concurrent data accesses by analyzing multiple correlated program features. Perceptron learning is used due to its lightweight and adaptable nature, enabling accurate and generalizable predictions of cache miss costs across diverse workloads. By integrating CAMP with a locality-based cache replacement policy, we demonstrate that CAMP enhances cache management for data-intensive applications by introducing concurrency awareness. Evaluations show that integrating CAMP with SHiP++ outperforms LRU by 7.1% and 12.6% in 1-core and 4-core systems on SPEC workloads, and by 10.7% in 4-core GAP workloads, surpassing state-of-the-art policies with only modest overhead.

Processing-in-Memory (PIM) offers a promising architecture to alleviate the memory wall challenge in graph processing applications. The key aspect of PIM is to incorporate logic within the memory, thereby leveraging the near-data advantages. State-of-the-art PIM-based graph processing accelerators tend to offload more to the memory in order to maximize near-data benefits, causing significant load imbalance in PIM systems. In this paper, we demonstrate that this intention is not true and that host processors still play a vital role in heterogeneous CPU-PIM systems. For this purpose, we propose CAPLBS, an online contention-aware Processing-in-Memory load-balance scheduler for graph processing applications in CPU-PIM systems. The core concept of CAPLBS is to steal workload candidates back to host processors with minimal off-chip data synchronization overhead when some host processors are idle. To model data contentions among workloads and determine the stealing decision, a measurement structure called Locality Cohesive Subgraph is proposed by deeply exploring the connectivity of the input graph and the memory access patterns of deployed graph applications. Experimental results show that CAPLBS achieved an average speed-up of 4.8x and 1.3x (up to 9.1x and 1.9x) compared with CPU-only and the upper bound of locality-aware fine-grained in-memory atomics. Moreover, CAPLBS adds no hardware overhead and works well with existing CPU-PIM graph processing accelerators.

Memristive computing-in-memory and near-threshold computing are two unconventional computing paradigms that can potentially enhance the energy efficiency and real-time performance of edge devices. However, their scalability faces challenges, primarily due to process variation. Here, we report a 1-Mb, 16-macro near-threshold memristive computing-in-memory engine. The two-transistor-one-resistor cells provide strong cell current modulation capability with more than 120-times amplified resistance ratio. To mitigate variation issues, we compensate for transistor mismatches by leveraging the intrinsic variations in memristors. Additionally, we propose a charge stacking technique between multiple analog-to-digital converters to perform analog weight-and-combine operations with small energy and area overhead. Moreover, we introduce an inter-macro hybrid control scheme to reduce the task-level inference power. The fabricated chip can perform highly parallel analog computing over 256 input channels with a 2.4% relative standard deviation. It achieves a throughput up to 10.49 tera-operations per second and an energy efficiency up to 88.51 tera-operations per second per watt.

Integrating processing-in-memory (PIM) with GPUs accelerates large language model (LLM) inference, but existing GPU-PIM systems encounter several challenges. While GPUs excel in large general matrix-matrix multiplications (GEMM), they struggle with small-scale operations better suited for PIM, which currently cannot handle them independently. Additionally, the computational demands of activation operations exceed the capabilities of current PIM technologies, leading to excessive data movement between the GPU and memory. PIM's potential for general matrix-vector multiplications (GEMV) is also limited by insufficient support for fine-grained parallelism. To address these issues, we propose Pyramid, a novel GPU-PIM system that optimizes PIM for LLM inference by strategically allocating cross-level computational resources within PIM to meet diverse needs and leveraging the strengths of both technologies. Evaluation results demonstrate that Pyramid outperforms existing systems like NeuPIM, AiM, and AttAcc by factors of 2.31x, 1.91x, and 1.72x, respectively.

In the past decade, various processing-in-memory (PIM) accelerators based on various devices, micro-architectures, and interfaces have been proposed to accelerate deep neural networks (DNNs). How to deploy DNNs onto PIM-based accelerators is the key to explore PIM's high performance and energy efficiency. The scale of DNN models, the diversity of PIM accelerators, and the complexity of deployment are far beyond the human deployment capability. Hence, an automatic deployment methodology is indispensable. In this work, we propose PIMCOMP, an end-to-end DNN compiler tailored for PIM accelerators, achieving efficient deployment of DNN models on PIM hardware. PIMCOMP can adapt to various PIM architectures by using an abstract configurable PIM accelerator template with a set of pseudo instructions, which is a high-level abstraction of the hardware's fundamental functionalities. Through a generic multilevel optimization framework, PIMCOMP realizes an end-to-end conversion from a high-level DNN description to pseudo instructions, which can be further converted to specific hardware intrinsics/primitives. The compilation addresses two critical issues in PIM-accelerated inference from a system perspective: 1) resource utilization and 2) dataflow scheduling. PIMCOMP adopts a flexible unfolding format to reshape and partition convolutional layers, adopts a weight-layout guided computation-storage-mapping approach to enhance resource utilization, and balances the system's computation, memory access, and communication characteristics. For dataflow scheduling, we design two scheduling algorithms with different interlayer pipeline granularities to support varying application scenarios while ensuring high-computational parallelism. Experiments demonstrate that PIMCOMP improves throughput, latency, and energy efficiency across various architectures. PIMCOMP is open-sourced at https://github.com/sunxt99/PIMCOMP-NN.

Deploying large language models (LLMs) on edge devices has the potentials for low-latency inference and privacy protection. However, meeting the substantial bandwidth demands of latency-oriented edge devices is challenging due to the strict power constraints of edge devices. Resistive random-access memory (RRAM)-based processing-in-memory (PIM) is an ideal solution for this challenge, thanks to its low read power and high internal bandwidth. Moreover, applying quantization methods, which require different precisions for weights and activations, is a common practice in edge inference. But existing accelerators cannot fully leverage the benefits of quantization, as they lack multiply-accumulate (MAC) units optimized for mixed-precision operands. To achieve low-latency edge inference, we design an RRAM-based PIM die that integrates dedicated energy-efficient MAC units, providing both computation and storage capabilities. Coupled with a dynamic random-access memory (DRAM) die for storing the key-value (KV) cache, we propose Lyla, an accelerator for low-latency edge LLM inference. Experimental results show that Lyla achieves 3.8x, 2.4x, and 1.2x latency improvements over a GPU and two DRAM-based PIM accelerators, respectively.

The human brain is a complex spiking neural network (SNN) capable of learning multimodal signals in a zero-shot manner by generalizing existing knowledge. Remarkably, it maintains minimal power consumption through event-based signal propagation. However, replicating the human brain in neuromorphic hardware presents both hardware and software challenges. Hardware limitations, such as the slowdown of Moore’s law and Von Neumann bottleneck, hinder the efficiency of digital computers. In addition, SNNs are characterized by their software training complexities. Here, to this end, we propose a hardware–software co-design on a 40 nm 256 kB in-memory computing macro that physically integrates a fixed and random liquid state machine SNN encoder with trainable artificial neural network projections. We showcase the zero-shot learning of multimodal events on the N-MNIST and N-TIDIGITS datasets, including visual and audio data association, as well as neural and visual data alignment for brain–machine interfaces. Our co-design achieves classification accuracy comparable to fully optimized software models, resulting in a 152.83- and 393.07-fold reduction in training costs compared with state-of-the-art spiking recurrent neural network-based contrastive learning and prototypical networks, and a 23.34- and 160-fold improvement in energy efficiency compared with cutting-edge digital hardware, respectively. These proof-of-principle prototypes demonstrate zero-shot multimodal events learning capability for emerging efficient and compact neuromorphic hardware. This study presents a neuromorphic computing platform capable of learning cross-modal, event-driven signals for efficient real-time knowledge generalization. It also achieves zero-shot transfer learning for multimodal data.

Resistive random-access memory (ReRAM) based Physical Unclonable Functions (PUFs) have emerged as an attractive hardware security primitive due to their low energy consumption and compact footprint. However, the reliability of existing ReRAM-based PUFs is challenged by read noise and temperature variations, as well as their resistance to Deep Neural Network (DNN) modeling attacks and Side Channel Attacks (SCAs). In this paper, we propose a novel 3T2R ReRAM-based reconfigurable PUF to address these challenges. By adopting the digital 3T2R voltage division cell design, we improve its reliability against ReRAM read noise and temperature variations, while the adjustable analog supply voltage of inverters enables quick, low-cost reconfigurability without reprogramming ReRAMs, effectively mitigating DNN modeling and SCA vulnerabilities. Our Re4PUF chip has been experimentally validated, achieving a low Bit Error Rate (BER) of 1% at 85 degrees C, a 7.59-fold reduction compared to existing ReRAM-based PUFs. It also demonstrates robust resistance to both DNN modeling attacks (MLP and Transformer) and SCAs, with success rates of approximately 50% and less than 70%, respectively.

Graph mining, critical for discovering specific patterns within complex structures, is becoming increasingly important in our data-driven world. Due to their memory-bound nature, graph mining applications encounter significant limitations with conventional processor-centric systems, like central processing units (CPUs) and graphics processing units (GPUs), stemming from the costly data movement between memory and processing units. Memory-centric computing systems, such as processing-in-memory (PIM) where computation occurs directly within or near memory modules, have the potential to accelerate graph mining. However, accelerating graph mining applications with PIM presents three primary challenges: (1) the difficulty in utilizing locality, (2) the challenge of exploring parallelism, and (3) the complexity of workload offloading between PIM and CPU. Addressing these intricate challenges, we introduce ProMiner, a novel framework that integrates three key techniques through cohesive software and hardware co-design. First, we propose a partitioning method tailored for graph mining to enhance data locality. Second, we design a coarse-fine parallelism optimization scheme to explore parallelism across different levels of memory. Third, we introduce a concurrency-aware mechanism for performance estimation, aimed at identifying the optimal computing engine for workload offloading to maximize performance. Our experimental results demonstrate that ProMiner significantly advances the state-of-the-art in graph mining, achieving 48.8% and 29.9% execution time reduction over NDMiner and DIM- Mining, respectively.