Deploying Vision Transformers on edge devices demands extreme compression without sacrificing accuracy. Encoder–decoder compression methods can cut model size dramatically, but they consistently leave LayerNorm parameters in full floating-point precision and ignore redundancy in the encoded representations themselves. We revisit both oversights. Our framework combines channel-wise LayerNorm quantization, designed to match LayerNorm’s per-channel affine structure, with magnitude-based pruning of encoded weights stabilized by gradient freezing during fine-tuning. On DeiT-S (Data-efficient Image Transformer-Small), channel-wise LayerNorm quantization alone drops top-1 accuracy by just 1.14% at 15.26× compression (5.51 MB, 8.88 GFLOPs). Adding pruning and knowledge distillation pushes compression to 16.2× (5.20 MB) with a 1.34% accuracy loss and 8.18 GFLOPs. Treating LayerNorm carefully and exploiting latent-space sparsity together push ViT compression noticeably past what current encoder–decoder methods achieve. Git link: Layernorm aware compression of vision transformers
Cross-component prediction (CCP) is a promising coding tool that provides bitrate reductions by exploiting inter-channel correlations between luma (Y) and chroma (UV) components for next-generation video coding (NGVC). In fact, CCP is utilized in the exploration test model for NGVC, namely Enhanced Compression Model (ECM), developed by the Joint Video Experts Team (JVET). However, existing CCP methods assume fixed chroma sample positions, and this assumption does not hold for diverse YUV 4:2:0 video sequences in which the chroma components are subsampled with a different phase relative to luma. We propose an adaptive CCP framework that estimates suitable chroma sample positions from the first frame of each sequences and signals them using existing indices to avoid additional signaling overhead. We introduce the proposed method into seven CCP variants in ECM by modifying downsampling filters, coordinates, and gradients. Experiments on ECM-18 show significant bitrate reductions with negligible complexity impact for diverse test sequences of JVET’s common test conditions and call for evidence, especially for gaming content with -2.84% (U) and -2.09% (V) reductions. The proposed method has been adopted in ECM-19 due to its favorable coding gains.
Preserving subtle spectral variations and temporal consistency is critical for reliable endoscopic video denoising. However, most existing methods process RGB channels independently and emphasize spatial noise suppression, often causing color distortion and temporal incoherence. In this paper, we propose STQFormer, a spatio-temporal denoising framework that models color channels as a unified vector entity and captures long-range temporal dependencies via quaternion-based representations. STQFormer integrates Spatio-Temporal Quaternion Transformer blocks with a frame-wise Swin encoder and is trained in an unsupervised manner without paired clean–noisy data. Experimental results on endoscopic video datasets demonstrate that STQ-Former achieves state-of-the-art denoising performance while preserving diagnostic color fidelity and temporal coherence.
Video anomaly detection (VAD) in nighttime traffic scenarios remains challenging due to low illumination, overexposure, motion blur, and reflection noise. To address these issues, we propose a Hybrid CNN–Swin Temporal Attention Network (HSTA) for unsupervised nighttime video anomaly detection based on future-frame prediction. HSTA integrates convolutional and transformer-based branches to jointly capture local texture details and global contextual dependencies. A Random Temporal Shift (RTS) module enhances temporal robustness, while a Temporal-Aware Channel Attention Block (TACAB) emphasizes motion-sensitive channels for refined spatial-temporal modeling. Experimental results on real nighttime traffic data demonstrate the effectiveness of HSTA, achieving AUC scores of 76.4%, which is 1.5% higher than the state-of-the-art method. Furthermore, additional evaluations on three daytime benchmarks indicate that HSTA exhibits promising generalization and robustness across different lighting conditions.
Recent advances in 3D Gaussian Splatting (3DGS) have demonstrated impressive performance in 3D reconstruction. However, artifacts such as spurious geometry and floaters remain annoying in practical 3DGS. While recent approaches increasingly leverage diffusion priors to improve visual fidelity, they often suffer from geometric hallucination due to their exclusive reliance on 2D image supervision. In this paper, we propose DepthFix3D, a geometry-aware diffusion framework designed for high-quality artifact removal in 3DGS renderings. The core of our approach introduces depth constraints in diffusion-based artifact removal, which consist of multi-source geometric cues extracted from both 3DGS and point cloud. The depth images from 3DGS and point cloud exhibit distinct artifacts-related patterns, providing complementary geometric cues that help the model locate artifact-prone regions. Moreover, we find that introducing point cloud representations provides complementary scene information, enabling more 3D-consistent and higher-quality content generation. Multi-source geometric cues are conditioned into the diffusion process via a ControlNet, and a progressive view synthesis mechanism is further utilized to enhance 3DGS quality. Experiments on diverse datasets demonstrate that DepthFix3D significantly outperforms existing baselines in both reconstruction fidelity (1.87 dB in RGB-averaged PSNR) and geometric consistency.
Multi-Object Tracking (MOT) in complex dynamic scenes remains challenging due to frequent occlusions, non-linear motion, and appearance drift, which make motion and appearance cues highly non-stationary in reliability. Most existing trackers rely on fixed heuristics or static cue fusion, and standard temporal modeling often over-smooths short-term discriminative dynamics, leading to identity switches under sudden maneuvers. To address these issues, we propose CG-Track, a robust MOT framework that adaptively regulates information flow along both temporal and modality dimensions. Specifically, a Temporal Feature Enhancement Network (TFEN) captures multi-scale temporal context while selectively emphasizing salient short-term fragments to suppress noisy history. In addition, a Cue-Gated Attention Fusion (CGAF) module dynamically re-weights appearance and motion features based on their instantaneous reliability under local spatial context, mitigating failures caused by occlusion or aggressive motion. Extensive experiments on DanceTrack and SportsMOT demonstrate consistent improvements on association-oriented metrics—achieving 67.0 and 81.2 Identification F1 scores (IDF1), respectively—as well as enhanced robustness in crowded, fast-changing scenarios.
Continual learning (CL) aims to learn tasks sequentially, reflecting real-world applications such as intelligent surveillance systems. Such practical situations are often resource-constrained, making sparse continual learning (sparse CL) particularly attractive as it ensures computational and memory efficiency by updating only a small subset of parameters without model expansion. However, existing sparse CL methods still struggle with the core challenge of CL: the stability–plasticity dilemma—retaining prior knowledge (stability) while acquiring new knowledge (plasticity). This paper proposes adaptive continual sparse learning (ACSL), a novel sparse CL method that adaptively controls both how many and which parameters are updated for each task. ACSL estimates the task familiarity of the model and dynamically allocates the number of parameters to update, thereby tailoring the stability–plasticity balance to each task. Furthermore, ACSL periodically reselects parameters to update in a stochastic manner, controlling plasticity at the parameter level. Experiments on multiple image classification benchmarks show that ACSL achieves higher accuracy than existing sparse CL methods across most datasets.
Diffusion-based visual navigation can produce temporally consistent trajectories from onboard camera observations. However, camera-only policies cannot reliably estimate obstacle distances, which becomes safety-critical under perception drift. To address this limitation, we propose Distance-Field Safety Plug-in (DFSI), a plug-in execution-time safety module that augments a pre-trained diffusion navigator with LiDAR-based distance-to-obstacle safeguarding, without retraining or modifying the backbone policy. The framework consists of two components: Reliability-aware Distance-Field Mapper (R-DFM) and Dual-Stage Safety Gate (DSG). R-DFM constructs a robot-centric local distance field using Euclidean Signed Distance Field (ESDF) mapping and adaptively refreshes the map to maintain reliability. Based on this geometric representation, DSG introduces a hierarchical safeguarding strategy that uses ESDF-derived distances as indicators to correct diffusion-sampled trajectories during execution. We evaluate the proposed system on real-robot indoor and outdoor experiments, including scenarios with dynamic obstacles. Results show that DFSI reduces safety interventions and increases the minimum observed obstacle clearance, with only a minimal increase in on-device latency.
Nighttime image dehazing must cope with two co-occurring yet physically distinct degradations: achromatic haze from atmospheric scattering, which attenuates luminance, and chromatic glow from artificial light sources, whose spatially varying colors corrupt the scene. Existing methods process both degradations in a single color space, causing mutual interference and incomplete restoration; supervised approaches further suffer from color shifts induced by the synthetic-to-real domain gap. We observe that these two degradations possess distinct chromatic properties and should therefore be separated in their respective optimal color spaces. Based on this insight, we present LDFNet, a fully unsupervised framework that disentangles achromatic haze in the YUV luminance channel and chromatic glow in the RGB space. A dual-branch Deep Image Prior architecture performs per-image decomposition without any training data, and a subsequent fusion stage with adaptive enhancement restores textures weakened during decomposition. Extensive experiments on both a synthetic benchmark with full-reference metrics and real-world nighttime images with five no-reference image quality metrics demonstrate leading performance across multiple metrics, validating color-space-aligned decomposition as an effective paradigm for nighttime image restoration.
Reconstructing relightable 3D scenes from in-the-wild photo collections is highly ill-posed due to uncontrolled illumination and inconsistent capture conditions. Existing inverse rendering-based frameworks capture complex illumination with environment maps, which remains fundamentally under-determined and unstable for in-the-wild setting. To this end, we introduce Multi-Light Relightable Gaussian Splatting (MR-GS) - a physically grounded relighting framework from in-the-wild images - that approximates illumination with multiple light sources and represents scene attributes with the simple yet effective Phong Reflectance model. Additional physics-aware regularizations are introduced to ensure correct and realistic material-lighting decomposition. Experiments on the NeRF-OSR benchmark demonstrate that MR-GS outperforms state-of-the-art methods in both quantitative and qualitative evaluations, while achieving a significantly higher rendering speed.
In recent years, image compression that supports both machine and human vision has garnered growing attention. Existing approaches typically rely on scalable or collaborative coding paradigms, which often introduce redundancy, architectural complexity, or difficulties in balancing machine analysis performance and human visual quality. In this paper, we propose a preprocessor-enhanced image compression framework for machine and human vision. A Mask-Guided Latent Attention (MGLA) module is introduced into a learning-based image compression system to improve the suitability of compressed representations for machine vision with minimal perceptual degradation. Furthermore, a plug-and-play preprocessor is introduced before the encoder to preserve task-relevant semantic information while suppressing redundant content, achieving improved machine vision performance with controlled bitrate overhead. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art preprocessing-based methods, achieving BD-rate reductions of 53.38% and 47.87% in terms of mAP on the COCO dataset for object detection and instance segmentation tasks, respectively.
Industrial defect detection is a crucial task in manufacturing, associated with many critical applications. Recent approaches adopt the Zero-Shot Anomaly Detection (ZSAD) paradigm to address the task, utilizing an auxiliary training dataset to overcome the scarcity of labeled defective data in real-world applications, with the vast majority of them employing CLIP-based models. In this paper, we propose a novel method for improving the performance of a downstream CLIP-based model for ZSAD, leveraging LMMs as inverse reasoners. Particularly, we first prompt an LMM specialized for anomaly detection, in order to extract counterfactual defect explanations for the normal images of the auxiliary dataset. Subsequently, we introduce this knowledge to the CLIP-based model for ZSAD, in order to improve its defect discrimination ability. To do so, we extract the text embeddings from the LMM-generated counterfactual explanations using the text encoder of CLIP, and introduce an auxiliary objective that minimizes the cosine similarity between the CLIP’s image embeddings of normal images and their corresponding text embeddings that convey defect semantics. We apply the proposed method on a state-of-the-art model for ZSAD, considerably improving its performance. The proposed method can be extended to generic detection and classification tasks for improving the baseline performance of CLIP-based models.
This paper introduces CLOE, a confidence-based local-to-global fully convolutional framework for multispectral illuminant estimation, which jointly captures spatial structure and spectral correlations through adaptive confidence-weighted pooling. The architecture integrates a Spatial Feature Extractor and a Spectral Branch through a confidence-based fusion mechanism that produces robust global illuminant predictions. Evaluations on the KAUST and BeyondRGB datasets demonstrate that CLOE consistently outperforms both classical statistical methods and recent learning-based approaches, achieving state-of-the-art performance in terms of angular error between the estimated and ground-truth illuminant spectra (mean-∆AMS and std-∆AMS). Qualitative analyses further show that CLOE provides interpretable intermediate representations, enabling visual inspection of spatial–spectral cues. These results highlight the effectiveness and robustness of the proposed method for real-world multispectral imaging.
As autonomous driving systems and mobile robots transition from offline perception to online closed-loop decision-making, embodied agents must continuously maintain 3D spatial representations for interaction and planning under strict latency constraints and limited compute budgets. However, most existing monocular 3D reconstruction methods produce mesh outputs with fixed topology and a fixed number of vertices, making it difficult to achieve an adaptive accuracy–efficiency trade-off under varying scene workloads. In this paper, we propose MR-Mono3D, a multi-resolution monocular 3D reconstruction framework tailored for embodied spatial perception. Specifically, MR-Mono3D incorporates three shape reconstruction branches at different resolutions and enables on-demand switching of reconstruction resolution without modifying the backbone network, thereby providing a controllable accuracy–efficiency trade-off: the high-resolution branch recovers finer geometric details, whereas the low-resolution branch significantly reduces visualization latency. In addition, we introduce a cross-resolution shape consistency loss for joint training, allowing the low-and mid-resolution branches to transfer geometric structure information from the high-resolution branch and alleviating the performance degradation at lower resolutions. Experimental results demonstrate that MR-Mono3D achieves pose and shape accuracy comparable to state-of-the-art baselines, while offering resolution switching and a controllable accuracy–efficiency trade-off, providing a flexible perception output mechanism for real-time deployment in embodied systems.
Intra prediction exploits spatial correlations within the same picture and is a core component of modern video compression systems. Compared to AV1, the AOMedia Video 2 (AV2) video coding format adopts a substantially enhanced intra prediction and intra mode coding framework to improve prediction accuracy and coding efficiency. This paper provides an overview of the intra prediction tools in AV2, including multiple reference line prediction, intra bi-prediction, data-driven intra prediction, implicit chroma-from-luma prediction, multi-hypothesis cross-component prediction, adaptive intra mode coding, and a 4-tap interpolation filter, along with key modifications to the existing AV1 intra prediction tools. The motivation, design rationale, and coding impact of these tools are discussed to highlight how AV2 advances intra prediction beyond previous generations.
Recent modifications to the Efficient Learned Image Compression (ELIC) framework have shown state-of-the-art rate-distortion performance for synthetic aperture radar (SAR) image compression. However, the entropy coding stage can be computationally expensive with minimal compression gains when the images’ latent features contain a high proportion of repetitive values, which is prevalent in SAR data preprocessed via Discrete Cosine Transform (DCT). We propose a simple method for accelerating SAR compression with a configurable bypass of ELIC’s asymmetric numeral systems (ANS) encoding in favor of traditional run-length encoding (RLE) for specific latent channel groups. This approach provides a level of control over the bitrate-speed trade-off with no reduction in image reconstruction quality. Experiments on 256×256 patches from the National Geospatial-Intelligence Agency (NGA) SAR dataset show that partial bypass configurations can achieve anywhere from 17% runtime savings at 2% bitrate cost to 83% runtime savings at 50% bitrate cost. Our analysis highlights how different latent feature characteristics and encoding algorithms contribute to the system’s bitrate and computational complexity, offering insights for future learned image compression model designs.
Video Object Segmentation (VOS) has recently benefited from foundation models such as SAM2. However, these models still struggle in scenarios where objects undergo drastic physical or topological transformations. In this work, we show that the causes of this performance decrease are largely due to the progressive degradation of internal representations caused by erroneous mask propagation over time. Motivated by this insight, we propose SAM2TC, a lightweight extension of SAM2 that introduces a Transformation Compensation (TC) module operating directly on the model’s internal representations. The TC is trained using privileged information derived from an oracle configuration, learning to correct degraded features and bring them closer to transformation-consistent representations. Experiments on popular benchmarks demonstrate that SAM2TC significantly outperforms SAM2 and recent state-of-the-art methods, achieving improved segmentation accuracy to extreme object deformations and state changes.
The rapid rise of AI-generated images (AIGIs) from text-to-image (T2I) models calls for effective quality assessment. Traditional IQA methods lack semantic alignment with text prompts, while existing AIGIQA approaches, especially CLIP-based ones, overemphasize semantics and ignore perceptual quality, relying on fixed prompts and static scoring. To bridge these gaps, we propose a dual-attribute prompt learning framework with progressive tuning based on CLIP that jointly models semantic consistency and perceptual quality. Our method employs dual-attribute learnable prompts and a dynamic patch-level regression head, trained through a three-stage strategy for improved stability and generalization. Experiments on multiple AIGIQA datasets show that our approach outperforms or matches state-of-the-art methods. The code of the proposed method is available at: https://github.com/Handvery/PPTCLIP.
Single image dehazing remains a challenging problem due to spatially varying atmospheric scattering and the degradation of structural details. While recent deep learning methods have achieved notable progress, most operate purely in the spatial domain and often struggle to preserve global geometric consistency under real-world haze. In this paper, we propose a frequency-aware dual-branch framework for image de-hazing that explicitly leverages spectral representations. Our key contribution is the Frequency-Adaptive Depth-Haze Consensus (FADHC) block, which operates in the Frequency domain to synchronize phase information between depth and dehazing branches, enforcing geometry-consistent structure recovery while adaptively restoring high-frequency components suppressed by haze. In addition, a CLIP-based semantic encoder is incorporated to provide robust semantic priors under dense and non-homogeneous haze. Experiments demonstrate that the proposed method achieves strong performance on both synthetic and real-world benchmarks, particularly on challenging datasets such as O-Haze and Dense-Haze. Project Code is available on https://github.com/AhmedSakr54/FADHC
Reliable out-of-distribution detection is pivotal for deploying deep learning in open-world scenarios. However, existing methods often rely on a single feature modality, either semantic direction (confidence) or activation intensity (norm), limiting their robustness against diverse shifts. To bridge this gap, we propose a unified framework that harmonizes representation quality with decision certainty. We introduce the angular-magnitude likelihood, which jointly evaluates semantic alignment and signal typicality. To ensure the reliability of the magnitude component, we integrate convolutional block attention modules into discriminative layers. This approach purifies feature norms without diluting low-level texture information. Experiments demonstrate that our framework achieves state-of-the-art performance.
