JPEG-AI is a recent image compression standardization effort that employs advanced deep learning techniques to achieve superior compression performance. However, the high computational complexity of JPEG-AI poses challenges for real-time applications and the implementation on resource-constrained devices. In this paper, we propose an efficient hardware accelerator for JPEG-AI on FPGA. A dedicated hardware architecture is designed, which includes a processing element array for neural network computations, a vector unit for diverse operations, and an efficient entropy coding module for me-tANS coding. Additionally, we propose a grouped on-chip coding scheme that enables cross-layer kernel fusion, reducing memory bandwidth requirements and improving overall system efficiency. Experimental results demonstrate that our implementation achieves a decoding FPS of 74.18 for 720P images and 38.40 for 1080P images, enabling real-time decoding. It achieves a peak performance of 4.12/3.98 TOPS for en/de-coding. To the best of our knowledge, this is the first work to implement the inference of JPEG-AI models on FPGA in the open literature.

compute-in-memory (ACIM) architectures offer significant throughput and energy benefits by performing multiplication-and-accumulation (MAC) operations directly within memory arrays. However, their overall efficiency is fundamentally constrained by the high power consumption of the per-column high-resolution analog-to-digital converters (ADCs) required to support modern DNNs (e.g., transformers), many of which demand both high computational precision and large-column throughput. In conventional ADC designs, energy in the noise-limited regime scales near-exponentially, typically by 4 & times; per additional bit, making high-resolution ADCs on every column power-prohibitive. This article proposes a high-precision and power-efficient multicolumn residue accumulation (MCRA) ACIM architecture to efficiently support precision-demanding modern DNNs. Each column uses a low-resolution coarse ADC (cADC), while the per-column residuals are accumulated and further quantized by an energy-efficient time-domain multi-input incremental sigma-delta (Mi-Sigma Delta) fine ADC (fADC). This approach amortizes the near-exponential energy growth across columns, while exploiting the more favorable power-resolution scaling of the time-domain Mi-Sigma Delta quantization. Postlayout simulations demonstrate a 66.2-dB signal-to-noise-and-distortion ratio (SNDR) per column at only 1/21 the energy of the baseline with per-column high-resolution ADCs, and 0.405 & times; (1/2.47) the energy of an energy-saving ADC per column, which achieves a similar SNDR. Circuit-and system-level simulations demonstrate that our MCRA CIM architecture achieves negligible accuracy degradation in precision-demanding ViT tasks while delivering high energy and area efficiency.

The efficient deployment of Mixture-of-Experts (MoE) models on edge devices is hindered by the Top-K expert selection bottleneck, where latency can exceed matrix multiplication by up to 27× on specialized hardware. Software algorithms suffer from poor parallelism due to conditional branches and data dependencies, while hardware designs face trade-offs among flexibility, programmability, and compatibility. To address these challenges, this work proposes Top-V, a flexible and programmable Top-K acceleration framework based on the RISC-V Vector (RVV) Extension. Top-V introduces a Reconfigurable Reduced Bitonic Sorter (R-RBS) generator that automatically produces energy-efficient sorting structures for diverse configurations. A custom VTOPK instruction is proposed to provide an efficient software interface supporting INT4/8/16 and FP16/32 data types. A dedicated Top-K hardware unit is integrated into the open-source Ara RISC-V vector processor, realizing a hardware-software co-design solution. FPGA evaluations show up to 3.07× and 5.24× speedups over the min-heap and standard RVV-based sorting, respectively, offering both efficiency and programmability for edge MoE deployment.

Auto-regressive (AR) models, initially successful in language generation, have recently shown promise in visual generation tasks due to their superior sampling efficiency. Unlike image generation, video generation requires a substantially larger number of tokens to produce coherent temporal frames, resulting in significant overhead during decoding. We first make specific key observations: (i) MLP modules in the decode phase dominate the inference latency, and (ii) there exists high temporal redundancy in MLP outputs of adjacent frames. With the insights, we propose **FastCar** to accelerate the decode phase for the AR video generation by exploring the temporal redundancy. The Temporal Attention Score (TAS) is proposed to determine whether to apply the replay strategy (i.e., reusing cached MLP outputs from the previous frame to reduce redundant computations) with detailed theoretical analysis and justification. Furthermore, we develop a hardware accelerator on FPGA with Dynamic Resource Scheduling based on TAS to enable better resource utilization and faster inference. Experimental results demonstrate the effectiveness of our method, which outperforms traditional sparse attention approaches with more than 2.1x decoding speedup and higher energy efficiency on the edge. Furthermore, by combining FastCar and sparse attention, FastCar can boost the performance of sparse attention with alleviated drifting, demonstrating our unique advantages for high-resolution and long-duration video generation.

Consumer electronic devices increasingly require low-latency, privacy-preserving, and always-on sequence intelligence for applications such as voice interaction, text prediction, and wearable sensing. In this context, studies show that although xLSTM provides strong long-sequence modeling capability and favorable real-time processing characteristics, its heterogeneous operators and high memory overhead make on-device deployment challenging. These challenges are particularly pronounced under tight power and resource constraints. To overcome these limitations, this work presents a high-efficiency and reconfigurable xLSTM accelerator for edge inference. A row-balanced block pruning approach along with five location-based sparsity schemes is introduced for the mLSTM attention process, and these are integrated with hybrid quantization techniques and an enhanced CSR encoding format to compress weights. In addition, eleven operator-level instructions are incorporated into the RISC V instruction set to improve flexibility and configurability. Third, an optimized hardware and software architecture is developed. Based on these techniques, an optimized hardware-software co-design is implemented for efficient xLSTM execution. Experimental evaluations show that the proposed solution achieves 3.52× greater energy efficiency than a CPU and 2.11× greater energy efficiency than a GPU, while lowering power consumption by 83.1% relative to the GPU.

Monocular Depth Estimation (MDE) has emerged as a pivotal task in computer vision, supporting numerous real-world applications. However, deploying accurate depth estimation models on resource-limited edge devices, especially Application-Specific Integrated Circuits (ASICs), is challenging due to the high computational and memory demands. Recent advancements in foundational depth estimation deliver impressive results but further amplify the difficulty of deployment on ASICs. To address this, we propose QuartDepth which adopts post-training quantization to quantize MDE models with hardware accelerations for ASICs. Our approach involves quantizing both weights and activations to 4-bit precision, reducing the model size and computation cost. To mitigate the performance degradation, we introduce activation polishing and compensation algorithm applied before and after activation quantization, as well as a weight reconstruction method for minimizing errors in weight quantization. Furthermore, we design a flexible and programmable hardware accelerator by supporting kernel fusion and customized instruction programmability, enhancing throughput and efficiency. Experimental results demonstrate that our framework achieves competitive accuracy while enabling fast inference and higher energy efficiency on ASICs, bridging the gap between high-performance depth estimation and practical edge-device applicability. Code: https://github.com/shawnricecake/quart-depth

Existing transformer‐based image captioning methods face two primary limitations: first, they struggle to adequately represent visual features from multiple regions during the encoding phase, and second, the decoder fails to effectively utilize future semantic information during the inference phase. To address these challenges, an attention‐enhanced image captioning model is proposed. During the encoding phase, multigranular visual features are integrated by combining cross‐attention and self‐attention mechanisms, fully utilizing both grid and regional features. Additionally, a novel dense global self‐attention module is introduced to enhance model performance with minimal computational cost by fully leveraging the contextual information and fine‐grained details of the image. This model is particularly well‐suited for biomimetic wearable devices, where real‐time visual assistance plays a crucial role in enhancing the user experience. In the decoding phase, a bidirectional decoding structure with an adaptive masking module is designed to dynamically adjust the focus on past and future semantic information, enabling the model to combine historical and future context effectively for generating more accurate and relevant descriptions. Experimental results on the MSCOCO dataset show that the model outperforms the baseline, achieving a 2.1 percentage point improvement in the CIDEr metric. Comprehensive hardware evaluations on the wearable platform demonstrate real‐time efficiency with minimal memory footprint, significantly outperforming state‐of‐the‐art models in edge deployment scenarios.

This study presents a high-speed, 9-bit integrating-mode phase interpolator (IMPI) in a 12 nm FinFET process. The proposed slew-rate-tolerant design accepts bandwidth-limited inputs, relaxing the stringent need for high-slew-rate clocks found in prior research. This is primarily achieved through an optimized switch design that converts the sinusoidal voltage input into a quasi-square-wave current. A detailed theoretical model identifies asymmetrical clock feedthrough as the dominant nonlinearity, which is suppressed by a cancellation circuit. Furthermore, an adaptive biasing loop is employed to compensate for Process, Voltage, and Temperature (PVT)-induced P/N mismatch. This work is validated through comprehensive post-layout simulations; operating from a 0.8 V supply at 16 GHz, the PI achieves a peak-to-peak Integral Nonlinearity (INL) of 4.3 LSB (530 fs) while consuming 6.56 mW.

This paper presents a high-speed 8-bit asynchronous successive approximation register (SAR) analog-to-digital converter (ADC) featuring tailored bit intervals (TBI). The design employs built-in SAR logic to set delays autonomously, eliminating the need for digital assistance, and thereby reducing both power and area consumption. This approach also effectively shortens the waiting time before lower-bit comparisons, enabling faster conversions. The ADC is simulated in the 16 nm process, occupying only 0.0012 mm(2), with post-simulation conducted under various extreme process and temperature conditions. Compared to prior works, our design exhibits notable performance advantages, achieving an ENOB of 7.38 bits at TT 25 degrees C with a power consumption of 6.94 mW. Furthermore, the designed TBI-ADC attains a sampling rate of 1.6 GS/s at FF -40 degrees C, representing a 33% increase over the fastest previously reported single-channel, 1b/cycle, 8-bit SAR ADC.

While posit format offers superior dynamic range and accuracy for transprecision computing, its adoption in RISC-V processors is hindered by the lack of a unified solution for lightweight, precision-scalable, and IEEE-754 arithmetic compatible hardware implementation. To address these challenges, we enhance RISC-V processors by 1) integrating dedicated posit codecs into the original FPU for lightweight implementation, 2) incorporating multi/mixed-precision support with dynamic exponent size for precision-scalability, and 3) reusing and customizing ISA extensions for IEEE-754 compatible posit operations. Our comprehensive evaluation spans the modified FPU, RISC-V core, and SoC levels. It demonstrates that our implementation achieves 47.9 state-of-the-art posit-enabled RISC-V processors, while achieving up to 2.54× throughput improvement in various GEMM kernels.

This paper presents a reconfigurable accelerator that unifies seven distinct nonlinear functions into a single, flexible hardware architecture. Efficient approximation of nonlinear functions is achieved by dedicated brain floating point 16(BF16) compute units. A carefully designed parallel pipeline architecture is proposed to deliver high computational speed and intense throughput. A reconfigurable interconnect, which contains broadcasters and switches, controls the data flow in the pipeline, allowing the computation of multiple activation functions and the time-multiplexed reuse of computing units. Evaluations using challenging datasets demonstrate the advantage of our approach. The code is available at: https://github.com/FPGA-acceleration/Activation_Func_Accelerate.git

Diffusion models (DMs) have emerged as a rev-olutionary technology in Al-generated content. Despite their superior quality, DMs face significant deployment challenges due to two key limitations: 1) the substantial computational cost inherent in their iterative denoising process and 2) the complexities of optimizing the execution of their multiscale U-Net architecture, which exhibits diverse computational pat-terns and varying memory requirements across layers. Existing optimization approaches face limitations on both algorithmic and hardware fronts. At the algorithm level, current methods struggle with inellicient feature enhancement through uniform channel processing and inflexible feature reuse with static caching strategies. On the hardware side, current solutions lack the flexibility and efficiency to adapt to the varied computational and memory demands within the U-Net. To address these challenges, we propose DiffAccel, an algorithm-hardware co-optimized accel-erator designed to accelerate the inference of various DMs. First, two key algorithmic innovations are introduced: selective feature enhancement targeting only high-dimensional channels and an adaptive feature caching mechanism hased on identified evolution patterns. Second, at the hardware level, a reconfig-urable array (RA) is proposed to enable efficient execution across diverse U-Net layers by switching its dataflow between output-stationary (OS) and weight-stationary (WS) modes based on the specific computational patterns of each layer. Moreover, a data-volume-aware memory management unit (MMU) is implemented, employing adaptive input allocation and hierarchical output stor-age to efficiently manage the disparate layer memory footprints of the U-Net, thereby improving on-chip memory efficiency and reducing waste. Third, fine-grained pipeline fusion is employed for key U-Net operation sequences to minimize intermediate data movement and significantly reduce latency. Specifically, row-first processing is applied for linear-Softmax operations, and channel-priority output is utilized for convolution-groupnorm (GNorm) sequences. At the algorithm level, DiffAccel achieves 2.93x speedup and reduces computational complexity (from 105 to 69 TFLOPs) compared to the baseline stable diffusion (SD) v2.0. DiffAccel is implemented with a 28-nm technology. In terms of hardware efficiency, based on post place-and-route (P&R) implementation results, DiffAccel achieves 5.25x-5.79x higher throughput and maintains comparable area efficiency compared to state-of-the-art (SOTA) accelerators.

The row-wise product algorithm shows significant potential for sparse matrix-matrix multiplication (SpGEMM) on hardware accelerators. Recent studies have made notable progress in accelerating SpGEMM using this algorithm. However, several challenges remain in accelerating imbalanced SpGEMM, where the distribution of non-zero elements across different rows is imbalanced. These challenges include: (1) the fixed dataflow of the merger tree, which leads to lower PE utilization, and (2) highly imbalanced data distributions, such as single rows with numerous non-zero elements, which result in intensive computations. This imbalance significantly challenges SpGEMM acceleration, leading to time-consuming processes that dominate overall computation time. In this paper, we propose AiSpGEMM to accelerate imbalanced SpGEMM on FPGAs. First, we improved the (CSR)-S-2 format to adapt it for imbalanced SpGEMM acceleration based on the row-wise product algorithm. This reduces off-chip memory bank conflicts and increases data reuse of matrix B. Secondly, we design a reconfigurable merger (R-merger) with flexible interconnects to improve PE utilization. Additionally, we propose an intra-row parallel merging algorithm and its corresponding hardware architecture, the parallel merger (P-merger), to accelerate intensive operations. Experimental results demonstrate that AiSpGEMM achieves a geometric mean (geomean) speedup of 5.8x compared to the state-of-the-art FPGA-based SpGEMM accelerator. In Geomean, AiSpGEMM achieves a 3.0x speedup and a 9.8x improvement in energy efficiency compared to the NVIDIA cuSPARSE library running on an NVIDIA A6000 GPU. Moreover, AiSpGEMM-21 demonstrated a 4x increase in average throughput compared to the same GPU.

Pruning has emerged as an effective technique for compressing convolutional neural networks (CNNs) by eliminating redundant weights, achieving lightweight models with negligible loss in inference accuracy. To leverage the sparsity for acceleration, many accelerators built for sparse CNNs have been developed. Existing hardware accelerators can perform well with structured or customized sparsity patterns. However, when facing the unstructured sparsity which can achieve higher compression rates, the corresponding hardware always suffers from insufficient utilization of computational resources, severe load balance between process elements, and significant overhead in logic resources, which results in a reduction of throughput, making it difficult to leverage high ratio sparsity for acceleration effectively. An efficient CNN inference accelerator that can handle both structured and unstructured sparse networks is proposed to address these issues. By flexibly employing multiple parallel computation methods combined with carefully developed sorting algorithms, the proposed architecture mitigates the hardware utilization inefficiencies caused by unstructured sparsity. Through a hardware-software methodology, new sparsity rules are introduced, nearly eliminating the load imbalance issue. The proposed Processing Element (PE) architecture can effectively select inputs for sparse networks while reducing the overhead of logic resources. The proposed architecture is implemented on the XCVU9P FPGA, achieving a frequency of 200 MHz. It achieves a computational throughput of 350.49 GOPs and 326.01 GOPs on ResNet-50 and ResNet-152, respectively, demonstrating a 1.2-2.1x DSP efficiency improvement compared to previous works.

To address performance degradation from the domain shift and to support user-specific services while considering privacy, security, and communication overhead, there is an urgent need for efficient on-device training accelerators for deep neural networks (DNNs). Given limited computing resources and battery capacity constraints, implementing complex DNN training on edge devices is extremely challenging. To address these issues, we introduce a Winograd-Integrated Gradient Optimization Framework (WIGOF) for cross-phase operand sharing in the Winograd domain, which significantly reduces the number of multiplications and additions. Additionally, we develop WiFlow, an efficient, precision-scalable on-device training accelerator, minimizing area and power overheads of the dedicated Winograd transformation unit. The WiFlow supports 16-bit floating point (FP16), 16-bit brain floating point (BF16), and 8 and 4-bit fixed point (INT8 and INT4), demonstrating scalable improvements in both computational throughput (TOPS) and energy efficiency (TOPS/W) at low precision. A novel data rearrangement pattern, named channel augmentation, addresses the imperfect decomposition to enhance the utilization of processing element units. Furthermore, we propose a Winograd interleaved block-execution dataflow (WInBlock), along with Hierarchical Adaptive Reuse Memory Optimization (HARM) to improve data reuse and reduce both the amount of DRAM and SRAM access. The end-to-end training of WiFlow is achieved on Xilinx XCVU440 FPGA. WiFlow is also synthesized with a 28nm CMOS technology, achieving an area efficiency of 624 GOPS/mm2 and an energy efficiency of 4.4 TOPS/W at a supply voltage of 0.9V and an operating frequency of 500 MHz. WiFlow accomplishes 7.75x higher area efficiency and 2.91x higher energy efficiency in actual DNN training compared with the state-of-the-art on device training accelerators.

Channel estimation (CE) is a critical component in the massive multi-input multi-output (MIMO) communication systems. Compared with conventional CE algorithms, deep learning (DL)-based approach becomes a promising alternative, due to its capability of offering enhanced performance and robustness across diverse scenarios. However, efficient DL-based CE algorithms have two key properties that make them challenging for implementation in existing architectures at the edge side: the diversity of deep neural networks (DNNs) and CE strategies, and the involvements of multiple computation-intensive tasks that compass conventional signal processing, artificial intelligence (AI) inference, and online learning. To address these challenges, a domain-specific processor based on an extended RISC-V instruction set architecture (ISA) is proposed to perform these DL-based CE algorithms. First, a dedicated RISC-V ISA extension is developed to support all essential operations required by a DL-based CE algorithm, such as matrix inversion, in a flexible manner. Building on the customized ISA extension, a highly adaptable and scalable RISC-V processor is developed, featuring scalar and vector posit arithmetic units to alleviate high computational and memory demands of DNNs during both inference and training phase. Additionally, a coarse-grained matrix accelerator is integrated to expedite various matrix operations ensuring high throughput. In this way, both high flexibility and computational efficiency are achieved. Finally, our processor is implemented on a TSMC 28-nm technology. Implementation results show that the processor achieves a speedup of 5.16 similar to 6.80x for all matrix operations compared with the state-of-the-art work. Moreover, the proposed processor provides an area efficiency improvement of 1.61x and an energy efficiency enhancement of 6.6 similar to 15.4x compared to the open-source vector processor Ara. Notably, this work is the first RISC-V domain-specific processor tailored for diverse DL-based CE algorithms.

With the rapid development of autonomous driving technology, millimeter-wave (MMW) radar has emerged as a critical sensor modality due to its superior environmental adaptability compared to conventional LiDAR systems. Direction of Arrival (DOA) estimation serves as a fundamental component in MMW radar target detection, whose estimation accuracy directly impacts the quality of generated radar point clouds. While traditional DOA estimation algorithms achieve satisfactory performance in ideal conditions, their performance significantly degrades in challenging scenarios involving coherent signals or low signal-to-noise ratio (SNR) conditions. To address these challenges, this paper proposes Deep UCA-ESPRIT, a novel neural network enhanced DOA estimation method that combines the advantages of traditional DOA estimation algorithms with deep learning capabilities. The proposed architecture employs a covariance matrix reconstruction network to mitigate coherence effects, followed by a robust uniform circular array-based 2D-ESPRIT estimator for joint elevation and azimuth angle estimation. Experimental results demonstrate that Deep UCA-ESPRIT maintains superior estimation accuracy across various SNR conditions and coherent signal scenarios, outperforming conventional DOA estimation methods while preserving computational efficiency.

This paper presents a successive approximation register (SAR) and incremental sigma-delta modulator (ISDM) hybrid analog-to-digital converter (ADC) that operated at a minimum voltage supply of 0.575 V. A thorough analysis of the non-linearities caused by PVT variations and common-mode voltage (VCM) shifts in the ISDM stage is presented. The ADC employs an improved high-precision double-bootstrapped switch, and the synchronous clock is also double-bootstrapped to work under the low supply voltage. A modified merged capacitor switching (MCS) approach is presented to maintain a stable VCM at the differential input. The chip was fabricated using a 0.18 µm CMOS process, with a core area of 0.21 mm2. It consumed only 0.42 µW at a 0.6 V supply and a sampling rate of 10 kS/s, which achieved an effective number of bits (ENOB) of 11.03. The resulting figure of merit (FOMW) was 20.05 fJ/conversion-step, which is the lowest reported for ADCs of this architecture in a 0.18 µm process.

Convolutional neural networks (CNNs) are expected to bridge the domain shift between the training data and real-world tasks. Moreover, the efficient training of CNNs on resource-constrained platforms has become more important because of communication latency and privacy concerns. However, deploying CNN training on edge devices is challenging due to the intensive computation and diverse computational patterns. In this work, we firstly propose a hybrid decomposition Winograd (HDW) method that significantly reduces the number of multiplications and flexibly handles various convolution operations during training. Secondly, we design a reconfigurable CNN training accelerator, named WinTA, utilizing a set of unified transformation units to support various Winograd operations. Thirdly, we implement an efficient and flexible data access scheme using a hierarchical barrel shifter network (HBSN). Experimental results on the Xilinx Alveo U50 FPGA Card demonstrate that WinTA effectively accelerates CNN training. Compared to CPU and GPU implementations, WinTA achieves speedups of 7.1x and 1.65x, respectively, while improving energy efficiency by 26.6x and 10.4x , respectively. Additionally, our design provides 1.24x and 2.04x improvements in terms of throughput and resource efficiency compared to prior-art FPGA-based training accelerator.