Recently, vision-language models (VLMs) have made significant progress in 3D scene understanding, driving advances in applications such as embodied intelligence and robotic vision. However, existing approaches typically either rely directly on explicit 3D inputs (e.g., point clouds or RGB-D sequences), or introduce an additional 3D geometry encoder to derive 3D-aware visual tokens from 2D images. Such designs structurally decouple 3D geometric perception from the rich 2D semantics learned via vision-language pre-training, hindering the development of a unified 3D vision-language representation. In this work, we propose Occ-VLM, a novel framework for 3D scene understanding that operates purely on posed RGB images and employs a single 2D vision encoder. Specifically, Occ-VLM reconstructs 3D scene occupancy as an auxiliary geometric prior, which is utilized to spatially associate foreground 2D tokens with 3D space. These tokens are then decoded by a Large Language Model (LLM) for unified scene understanding. Extensive experiments demonstrate that Occ-VLM achieves both accurate geometric perception and robust vision-language reasoning: it attains state-of-the-art performance on multi-view occupancy prediction, while performing on par with 3D-input VLMs on 3D Visual Question Answering (VQA) and 3D dense captioning benchmarks.
To address the issues of large prediction deviation and high dataset cost in traditional frequency selective surface (FSS) design combined with neural networks, an agile FSS design framework based on improved random forest (RF) and multi-scale convolutional neural network (MS-CNN) was proposed. In the framework, the improved RF optimized the sampling strategy through electromagnetic characteristic splitting criteria and multi-feature interaction evaluation to construct a high-quality dataset—only 1 157 samples were needed to achieve a prediction mean squared error (MSE) < 2.0, reducing the sample size by 61% compared with traditional sampling. The MS-CNN used 3×1, 5×1, and 7×1 multi-scale convolution kernels to extract electromagnetic response features, and combined with a frequency gradient loss function; the prediction MSE of TE/TM dual-polarization S21 curves at 0°/70° incident angles was as low as 2.2.With MS-CNN as the prediction agent, reverse design combined with particle swarm optimization (PSO) was conducted to output FSS parameters meeting the requirements: S21 ≥ -1.5 dB in the 25~33 GHz band, stable response at 0°~70° incident angles, and dual-polarization adaptation. The parameters were verified to meet the standards via HFSS, and the model's generalization was validated in the 20~28 GHz band.This framework provides an efficient solution for the agile design of wideband, multi-polarization, and wide-angle FSS.
The widespread deployment of resource-constrained intelligence visual Internet of Things (IoT) edge devices has driven an urgent demand for low-power, high-fidelity image compression solutions. The conventional off-sensor compression approach incurs significant quantization energy overhead, digital memory burden, and data transform latency. To address this issue, this brief presents a 2-D discrete cosine transform (2D-DCT) processor suitable for a processing-in-sensor (PIS) architecture, utilizing a one-time-programmed (OTP) floating-gate transistor (FGT) array as the DCT computing unit (CU). The proposed 2D-DCT processor allows image data to be compressed before digitization, significantly reducing the analog-to-digital converter (ADC)’s quantization overhead and the bandwidth of digital data transmission. The prototype chip is fabricated in a 65 nm flash technology. The proposed 2D-DCT processor was integrated with a $32\times 32$ analog image sensor array to build a PIS system for evaluation. Compression results show a peak signal-to-noise ratio (PSNR) ranging from 29.52 to 32.14 dB and a structural similarity index (SSIM) ranging from 0.934 to 0.973, with a PSNR exceeding 24.3 dB at an $8\times $ compression ratio (CR), indicating high image compression quality. The constructed PIS system achieves a maximum processing frame rate of 398 fps, with a power consumption of $7.06~\mu $ W of the 2D-DCT processor, leading to an energy efficiency of 17.32 pJ/pixel.
Vision-Language-Action (VLA) models enable robotic systems to perform embodied tasks but face deployment challenges due to the high computational demands of the dense Large Language Models (LLMs), with existing early-exit-based sparsification methods often overlooking the critical semantic role of final layers in downstream tasks. Aligning with the recent breakthrough of the Shallow Brain Hypothesis (SBH) in neuroscience and the mixture of experts in model sparsification, we conceptualize each LLM layer as an expert and propose a Mixture-of-LayEr Vision Language Action model (MoLe-VLA or simply MoLe) architecture for dynamic LLM layer activation. Specifically, we introduce a Spatial-Temporal Aware Router (STAR) for MoLe to selectively activate only parts of the layers based on the robot’s current state, mimicking the brain's distinct signal pathways specialized for cognition and causal reasoning. Additionally, to compensate for the cognition ability of LLM lost during the layer-skipping, we devise a Cognitive self-Knowledge Distillation (CogKD) to enhance the understanding of task demands and generate task-relevant action sequences by leveraging cognition features. Extensive experiments in RLBench simulations and real-world environments demonstrate the superiority of MoLe-VLA in both efficiency and performance, improving the mean success rate by 9.7% across ten simulation tasks while accelerating inference by 36.8% over OpenVLA.
This work presents a fully integrated $138-\text{GHz}$ Dielectric Waveguide (DWG) communication link in a $\mathbf{2 8}-\mathbf{n m}$ CMOS process, addressing the critical challenges of high cost and excessive power consumption in DWG interconnects. The proposed low-cost packaging scheme, combining wire bonding, SIW transition, and transverse aperture coupling, achieves a low coupling loss of 5.2 dB. The dynamic bias adjustment of the PA and LNA enables power consumption optimization for different link lengths. Measurement results demonstrate that the link delivers a state-of-the-art energy efficiency of $4.87 \text{pJ} / \text{bit}$ at 1.5 m with a data rate of 17 Gbps. Although the data rate declines at 3 m, it still reaches the theoretical maximum rate predicted by the dispersion factor model, validating the effectiveness of the system design.
With the ever-increasing demand for highly energy-efficient Internet of Things (IoT) edge devices, nonvolatile SRAM (NVSRAM) architectures are proposed to reduce leakage power during sleep mode, backup latency and power consumption, in which emerging nonvolatile memory (NVM) devices are embedded into SRAM. However, previous NVSRAM architectures introduce significant area overhead due to the additional, larger-sized transistors required, as well as extra masks resulting from the incompatibility of embedded NVM devices with standard CMOS technology. Therefore, in this paper, we propose a fully logic-compatible NVSRAM architecture based on the Charge-Trap Programming (CTP) method. By utilizing minimum-sized transistors, the proposed fully logic-compatible 8T NVSRAM bitcell achieves a 15.3% reduction in area overhead compared to existing designs. A 1Kb NVSRAM macro is designed to support full-array Boolean logic operations, in-situ storage of computation results without loss of weight data, row-wise in-bitline copy, and write/read operations by a logic configuration driver. The chip is fabricated using 22nm standard CMOS technology. It achieves a bit error rate (BER) of 0.11% by employing a data-aware error correction (DAEC) circuit, representing a 4.69% improvement over operation without error correction. It also demonstrates an energy efficiency of 336.93 TOPS/W for Boolean logic operations and achieves a 66.1% reduction in energy consumption for in-bitline copy operations compared to read-write-based (RWB) copy operations.
Edge deployment of low-batch large language models (LLMs) faces critical memory bandwidth bottlenecks when executing memory-intensive general matrix-vector multiplications (GEMV) operations. While digital processing-in-memory (PIM) architectures promise to accelerate GEMV operations, existing PIM-equipped edge devices still suffer from three key limitations: limited bandwidth improvement, component under-utilization in mixed workloads, and low compute capacity of computing units (CUs). In this paper, we propose CD-PIM to address these challenges through three key innovations. First, we introduce a high-bandwidth compute-efficient mode (HBCEM) that enhances bandwidth by dividing each bank into four pseudo-banks through segmented global bitlines. Second, we propose a low-batch interleaving mode (LBIM) to improve component utilization by overlapping GEMV operations with GEMM operations. Third, we design a compute-efficient CU that performs enhanced GEMV operations in a pipelined manner by serially feeding weight data into the computing core. Forth, we adopt a column-wise mapping for the key-cache matrix and row-wise mapping for the value-cache matrix, which fully utilizes CU resources. Our evaluation shows that compared to a GPU-only baseline and state-of-the-art PIM designs, our CD-PIM achieves 11.42x and 4.25x speedup on average within a single batch in HBCEM mode, respectively. Moreover, for low-batch sizes, the CD-PIM achieves an average speedup of 1.12x in LBIM compared to HBCEM.
In this paper, the Charge-trap transistor (CTT) family is extended to include Ultra-Low Voltage Threshold (ULVT), Standard Voltage Threshold (SVT), and High Voltage Threshold (HVT) transistors in standard 28nm bulk-Si CMOS technology. Measurements across 45 transistor types and channel widths (W) under various bias stresses indicate that W and transistor type have a minimal effect on the maximum threshold voltage shift (∆Vth,max). After the programming (PRG) operation, CTTs exhibit reduced low-frequency noise (LFN) possibly due to charge filling of interface traps, stable on-resistance (Ron) under identical VGS − Vth bias conditions, and enhanced anti-aging characteristics. These results highlight the potential of the CTT family as fundamental components for nonvolatile storage and analog circuits.
This paper presents a 30Gb/s/pin three-level pulse amplitude modulation (PAM3) single-ended transceiver with a crosstalk cancellation coding scheme for memory interfaces. Designed for PAM3 signaling, the proposed 4D-transition-pattern-free (4DTPF) coding avoids data transition patterns with high crosstalk-induced jitter (CIJ) and thus reduces the severe crosstalk noise in high-density memory links. Impedance and ratio level mismatch (RLM) calibration as well as C-peaking equalizer are employed to further improve the signal integrity. The prototype chip demonstrates an energy efficiency of 1.35pJ/b and an active area of 0.042mm2/lane. Post-layout simulation shows a 46.2% jitter reduction at 30Gb/s/pin for the 5cm printed circuit board (PCB) channels via the 4DTPF coding. And the instrumental measurement also exhibits a jitter reduction around 43% for various data rates.
As deep learning scales, single-die inference is bounded by reticle limits, motivating heterogeneous multi-chiplet systems with AI domain-specific accelerators (DSAs). However, inter-chiplet communication degrades performance–energy efficiency and complicates deployment. To address this issue, a compilation framework is introduced to automatically optimize deployment on such systems. Unlike prior work, both activation and weight traffic are modeled, including repeated weight transfers under limited on-chip memory. The deployment is cast as combinatorial optimization with throughput- or energy- prioritized objectives. A double-layer genetic algorithm is employed to decouple task segmentation from chiplet mapping, shrinking the search space and accelerating convergence while retaining global optimality considerations. A fast event-driven evaluator is integrated to estimate throughput and energy. Given a model and hardware description, deployment strategies are produced automatically. On an 8-chiplet system, up to 30.83% higher throughput and 24.65% lower energy were achieved; on average, throughput improved by 12.26% and energy decreased by 10.87% relative to state of the art work.
The rapid adoption of data-driven methods in biomedicine has intensified concerns over privacy, governance, and regulation, limiting raw data sharing and hindering the assembly of representative cohorts for clinically relevant AI. This landscape necessitates practical, efficient privacy solutions, as cryptographic defenses often impose heavy overhead and differential privacy can degrade performance, leading to sub-optimal outcomes in real-world settings. Here, we present a lightweight federated learning method, INFL, based on Implicit Neural Representations that addresses these challenges. Our approach integrates plug-and-play, coordinate-conditioned modules into client models, embeds a secret key directly into the architecture, and supports seamless aggregation across heterogeneous sites. Across diverse biomedical omics tasks, including cohort-scale classification in bulk proteomics, regression for perturbation prediction in single-cell transcriptomics, and clustering in spatial transcriptomics and multi-omics with both public and private data, we demonstrate that INFL achieves strong, controllable privacy while maintaining utility, preserving the performance necessary for downstream scientific and clinical applications.
Continual test-time adaptation adapts a source-pretrained model to non-stationary, unlabeled target streams while retaining past competence, yet texture-biased backbones risk error accumulation and catastrophic forgetting. Drawing inspiration from the process of decoupling shape and texture in the human visual system, we introduce MoASE, a plug-in mixture-of-experts that disentangles domain-agnostic structure from domain-specific texture using Activation Sparsity Experts with Spatial Differentiable Dropout, forming complementary high- and low-activation pathways, while high- and low-rank bottlenecks diversify representations. The Activation Sparsity Gate produces input-adaptive SDD thresholds for precise token selection, and the Domain-Aware Router assigns per-sample expert weights using texture-sensitive cues. To curb confirmation bias on unlabeled streams and stabilize supervision, we then introduce Domain-Adaptive On-Policy Distillation to constitute MoASE++, with an EMA-anchored on-policy reverse KL distillation and an augmentation policy conditioned on entropy and confidence that aligns predictions across the same views and improves the robustness-plasticity balance. Extensive experiments on classification (CIFAR-10/100-C, ImageNet-C) and semantic segmentation (Cityscapes->ACDC) demonstrate consistent state-of-the-art performance, offering a principled, controllable approach to continual adaptation in dynamic visual environments.
Quantization techniques vary considerably in their requirements for computational precision. Although prior works have introduced various multi-precision multiply-accumulate (MAC) units, most offer limited precision coverage. Moreover, most designs focus on standard arithmetic and lack optimizations for inference in large language models (LLMs). To overcome these limitations, this paper proposes UniMAC, a multi-precision MAC architecture that unifies integer, floating-point, and FP-INT operations within a single computing datapath by employing a new General-Purpose Data Partitioning (GPDP) method and an Iterative Neighbor Grouping Scheduling (INGS) strategy. In addition, we implement two acceleration modes for FP16 guided by an on-the-fly activation-to-activation (act-to-act) search (OAS) algorithm to improve throughput and efficiency in act-to-act computations, a dominant contributor to inference latency in LLMs with long contexts. Compared with prior works, the proposed design supports a wider range of precision formats while delivering up to 3.9× higher energy efficiency in FP8 and 2× for act-to-act computations in FP16.
Existing quantization methods are fundamentally limited by rigid, integer-based bit-widths (e.g., 2, 3-bit), creating a "deployment gap" where LLMs cannot be optimally fitted to specific memory budgets. To bridge this gap, we introduce LiftQuant, a novel framework that enables continuous bit-width control for true Pareto-optimal deployment. The core innovation is a "lift-then-project" mechanism: we represent d-dimensional weight vectors by projecting a simple 1-bit lattice from a tunable D-dimensional "lifted" space. By adjusting the lifted dimension D, LiftQuant naturally yields an effective bit-width of D/d, allowing for seamless, continuous resolution adjustment rather than discrete steps. This projection generates a structured yet non-uniform codebook, capturing the expressive power of Vector Quantization. Crucially, its decoding path relies solely on linear transformations and 1-bit uniform quantizers, retaining hardware-friendly efficiency. This flexibility is transformative: LiftQuant enables a 70B LLM to be compressed to 2.4 bits to precisely fit a 24GB GPU, where its performance significantly surpasses state-of-the-art 2-bit models. With a decoding throughput up to 6.7x faster than FP16, LiftQuant redefines compression as a continuous optimization problem, paving the way for a new generation of hardware-aware LLM deployment.
In-network computing techniques, exemplified by NVLink Sharp (NVLS), offer a promising approach to addressing the communication bottlenecks in LLM inference by offloading collective operations, such as All-Reduce, to switches. However, the accelerator-centric architecture of NVLS suffers from two fundamental limitations: 1) it relies on GPU load instructions to trigger reduction operations, which means that the data reduced in the switch must be additionally transferred back to the initiating GPU rather than being broadcast directly, thereby introducing unnecessary communication overhead; 2) due to its architectural constraints, NVLS cannot offload operators that are not decomposable into memory-semantic instructions, such as the in-network quantization (INQ) proposed in this work. As a result, All-Reduce in NVLS must operate at FP16/BF16 precision, leading to substantial bandwidth waste.To address these limitations, we propose SCIN, the first switch-centric in-network architecture for shared-memory networks of AI accelerators, enabling both low-latency and high-bandwidth All-Reduce. Specifically, we introduce an in-switch accelerator (ISA) capable of initiating memory-semantic operations for in-network processing, together with a co-designed communication fabric that incurs negligible protocol overhead. By eliminating redundant data movement, SCIN delivers lower All-Reduce latency than NVLS. Moreover, by integrating a quantization module into the ISA, SCIN enables INQ for All-Reduce, reducing its precision to 8 bits and nearly doubling bandwidth with negligible accuracy loss. We also present a prototype of SCIN on a multi-FPGA system to demonstrate its feasibility and effectiveness. Experimental results show that our design accelerates All-Reduce by up to 8.7x for small messages and 3.8x for large messages, leading up to 1.74x faster TTFT and 1.34x faster TPOT on LLaMA-2 models.
Polynomial multiplication across variable degrees has been a dominant bottleneck in cryptographic applications, including post-quantum cryptography (PQC) to fully homomorphic encryption (FHE). Yet, none of the existing approaches can achieve both high performance and scalability across variable degrees. Number theoretic transform (NTT) excels at large degrees but suffers from performance degradation when the polynomial is decomposed into small degrees, while the iterative convolution algorithm (ICA) scales poorly to large degrees. Reconciling two disparate approaches has been hindered by a mathematical barrier: NTT decomposition produces heterogeneous polynomial rings that defy uniform ICA processing. This work overcomes this barrier by establishing a new algorithm, based on isomorphic mapping, to transform heterogeneous rings into a fixed-size cyclic ring. Leveraging this algorithm, we propose a scalable architecture called MG-PM, a mixed-granularity polynomial multiplier that, for the first time, enables efficient acceleration across variable degrees. Implemented on a Xilinx VCU118 FPGA, MG-PM supports degrees from 256 to 65536, achieving up to 9.1x latency reduction and 2.2x area reduction under different configurations, as well as 92.3% improvement in area-time product (ATP) compared with other state-of-the-art designs, establishing MG-PM as a highly efficient and scalable solution for accelerating polynomial multiplication across variable degrees.
Existing quantization methods are fundamentally limited by rigid, integer-based bit-widths (e.g., 2, 3-bit), resulting in a “deployment gap" where Large Language Models cannot be optimally fitted to specific memory budgets. To bridge this gap, we introduce LiftQuant, a novel framework that enables continuous bit-width control for true Pareto-optimal deployment. The core innovation is a “lift-then-project" mechanism which approximates low-dimensional weight vectors by projecting a simple 1-bit lattice from a higher-dimensional “lifted" space. Crucially, the effective bit-width is determined simply by the ratio of the lifted dimension to the original dimension, which allows the bit-width to be tuned quasi-continuous as the dimension is a flexible structural parameter. This projection generates a structured yet non-uniform codebook, capturing the expressive power of Vector Quantization (VQ). While beneficial over VQ, LiftQuant's decoding path relies solely on linear transformations and 1-bit uniform quantizers, retaining hardware-friendly nature. This flexibility is transformative: LiftQuant enables a 70B LLM to be compressed to 2.4 bits to precisely fit a 24GB GPU, where its performance significantly surpasses state-of-the-art 2-bit models fitted on the same device. Our code and ckpt is available at https://github.com/Heliulu/LiftQuant.
Deep neural networks (DNNs) improve the accuracy of remote photoplethysmography (rPPG) signal extraction but challenge hardware implementation by requiring support for two distinct computational modes: convolution and spectral calculation. This paper proposes a convolution and variable-size Fast Fourier Transform (FFT) fusion accelerator named CVFFA, aiming to support both operations in deep learning-based rPPG algorithms simultaneously. The CVFFA integrates an incomplete decomposition algorithm for FFT with a fixed butterfly operation mapping mechanism, enabling concurrent support for convolution computation and variable-size FFT operation. To apply the incomplete decomposition algorithm in CVFFA, a dataflow is proposed to prevent on-chip memory conflicts during continuous DMA writes under the FFT operation. A precision-aware training method is developed to address low-precision fixed-point FFT effects, which reduces the mean absolute error (MAE) and root mean square error (RMSE) in heart rate prediction through retraining both the supervised learning model PhysNet and the unsupervised learning model SiNC. Compared with dedicated FFT hardware implementations, CVFFA achieves a reduction of up to 68% in computation time. Implemented in TSMC 28 nm technology, CVFFA operates at 400 MHz clock frequency, while delivering 2.63 TOPS/W energy efficiency for rPPG-based non-contact heart rate detection applications.
Neural networks (NNs) have emerged as powerful tools for accelerating the design of frequency-selective surfaces (FSSs) due to their fast prediction of electromagnetic (EM) responses. However, the intrinsic gap between low-dimensional geometric parameters and high-dimensional broadband spectral outputs often causes prediction inaccuracies, leading to optimization illusion-a phenomenon where designs appear optimal under surrogate models but fail under full-wave validation. To address this issue, we propose a performance-oriented end-to-end FSS inverse design framework that simultaneously enhances broadband prediction accuracy and optimization robustness. The multibranch autoencoder (MBAE) achieves high-fidelity EM response modeling by integrating a pretrained variational autoencoder (VAE), Fourier layers, and a rule-guided loss function. It supports broadband prediction with up to 400 frequency points, reducing the prediction error to 0.43 within the 0-40-GHz frequency range, which represents a 79.3% improvement over standard MLPs. The consensus-guided multimodel performance evaluation (CG-MPE) mechanism increases the inverse design success rate from 40% to 90% by aggregating predictions from multiple surrogate models. This framework provides a reliable solution for neural network-assisted broadband FSS design and has the potential to be extended to the inverse design of other broadband passive structures.