Real-time 4K video super-resolution (VSR) requires the effective reuse of temporal information under strict latency constraints, typically relying on temporal alignment and real-time reconstruction. However, imperfect alignment introduces perturbations into the temporal recursion, which can accumulate over time and degrade reconstruction quality—an effect widely observed in recurrent VSR pipelines but rarely analyzed explicitly from a dynamical perspective under real-time constraints. In this work, we formulate real-time VSR inference as a recurrent dynamical process, embedding alignment within the state update. This formulation enables explicit analysis of how alignment-induced perturbations are introduced and propagated during inference. Motivated by this analysis, we propose StableFlow, a stability-guided real-time VSR framework. Specifically, StableFlow introduces an Alignment Gain-aware Alignment Module (AGAM) to enhance the utility of temporally aligned features, a State-aware Perturbation Control Filter (SPCF) to suppress unreliable propagated information, and a Temporal Propagation Control (TPC) loss to regulate long-term recurrent state evolution. Our method combines efficient temporal alignment with state-aware perturbation control and a temporal propagation control loss to stabilize long-term recurrent behavior. Experiments demonstrate that StableFlow achieves real-time 4K performance (over 40 FPS on an NVIDIA RTX 3090) for 4×upscaling from 960×540 inputs to 3840×2160 outputs, with only 321K parameters and 77.45G FLOPs, while maintaining competitive reconstruction quality. The source code will be made public after the peer review process.
Accurately identifying priority attention regions in complex driving scenarios is critical yet challenging, due to the coupling of heterogeneous factors such as motion, structured semantics, and human experience. However, existing methods often isolate these factors without systematically modeling comprehensive reasoning based on their relations, and rely on supervision from biased gaze data or indiscriminate semantic overlays, failing to highlight truly critical regions. To address these limitations, we propose the Driving Attention Guidance Network (DAG-Net) based on a specially designed Chain-of-Attention-Thought (CoAT) strategy that hierarchically decouples factors and models cognition-guided reasoning. DAG-Net comprises three key modules: (1) The Entropy-Modulated Motion Module captures continuous multi-order motion features with emphasizing abrupt events modulated by entropy-guided attention; (2) the Distilled Semantic Structuring Module enhances contextual representations by cross-level harmonization for hierarchical distillation of global priors, and models local spatial-structured relationships via graph convolution; (3) the novel Cognitive Alignment Reasoning Module achieves attention consensus through stepwise reasoning, where Interactive Modulation enhances cross-modal consistency of decoupled perception and Experience-Perception Synergism emulates cognitive loops of iterative refinement between perception and experience. Furthermore, we propose the first automatic annotation strategy for generating the Driving Attention Guidance Map as supervision for risk-aware attention prioritization. It quantifies object importance via collision-cost modeling with multi-modal information integration, emphasizing safety-critical regions. Experiments show our method achieves accurate and comprehensive attention guidance, enhancing contextual risk awareness and supporting safer driver assistance and autonomous driving.
Point cloud completion aims to recover missing regions in 3D point clouds caused by sensor limitations and environmental occlusions. Although recent methods have achieved remarkable progress in recovering the overall shape of incomplete objects, explicitly controlling the spatial distribution of reconstructed points remains challenging, often leading to outliers and non-uniform point distributions. In this paper, we propose a novel framework, VFC-Net, which generates a uniformly distributed coarse point cloud to effectively guide dense reconstruction. Specifically, VFC-Net adopts a generation-upsampling paradigm. In the generation stage, termed VoxGen, the input point cloud is first converted into a voxel-based representation, from which a coarse yet complete point cloud is generated via voxel occupancy classification, naturally encouraging a uniform spatial distribution. In the upsampling stage, termed VoxPu, voxel occupancy predictions are fused with voxel features to guide dense reconstruction, thereby preserving the distribution consistency of the final output. Furthermore, we introduce a lightweight VoxAttn module in both stages to efficiently capture missing geometric structures. By decomposing the voxel grid into a set of 2D slices and performing attention-based feature aggregation, VoxAttn significantly reduces the computational complexity. To compensate for the information loss introduced by slicing, we further employ alternating slicing axes together with cross-slice feature aggregation. Extensive experiments on multiple challenging benchmarks demonstrate that VFC-Net achieves state-of-the-art performance in point cloud completion.
Hyperspectral image classification (HSIC) has witnessed remarkable progress with the rise of deep learning. Nevertheless, its real-world deployment remains substantially constrained by two fundamental challenges: substantial domain discrepancies across heterogeneous sensors and the scarcity of labeled samples in the real scene (target domain). These issues motivate the task of cross-domain zero-shot HSIC (CDZS HSIC), where the objective is to recognize unseen target categories without any labeled supervision. To address this challenge, we develop a Hilbert-Schmidt independence learning guided Kolmogorov-Arnold Network (HSIL-KAN). Specifically, we first construct a Cross-Domain Kolmogorov-Arnold Network (CD-KAN) as the core backbone, enabling expressive spectral-spatial representation learning with compact parameterization. Building upon this backbone, we design a Hilbert-Schmidt Independence Criterion guided Domain Feature Decomposition (H-DFD) module to explicitly factorize latent features into domain-invariant and domain-specific subspaces, thereby substantially enhancing cross-domain transferability. Furthermore, a Prototype-Regularized Equiangular Tight Frame (PR-ETF) classifier is proposed to maintain geometric class separability and alleviate class imbalance in the zero-shot condition. Comprehensive experiments on three benchmark hyperspectral datasets demonstrate that HSIL-KAN consistently achieves state-of-the-art performance while also exhibiting notable parameter efficiency and computational scalability. The code will be available online at https://github.com/jojolee6513.
Occluded by rain streaks and haze, rainy images suffer from information loss, which necessitates precise recovery of missing content. However, visually appealing yet spurious image details generated by some deraining methods may reduce image fidelity and induce misjudgments in subsequent detection and recognition tasks. Invertible Neural Networks (INNs) are inherently well-suited for the image deraining task. Benefiting from their invertible mapping property, INNs guarantee that all newly generated details can be traced back to the observed input images, thereby improving the fidelity of recovered images. However, it is challenging for INNs to directly map a rainy image to its clean version. Because the invertible mapping requires that the input and the output contain equivalent information, a rainy image inherently loses some details which may lead to information loss in the restored image. In this paper, we propose an Asymmetric Invertible Disentanglement (AInvDis) Network based on a dual-path INN to decompose a rainy image into a rain-free one and a rain map. AInvDis additionally introduces compensated image details generated by a CNN-based module, Prior Compensation Network (PCN), to assist the dual-path INN in restoring background details obscured by rain streaks. Although PCN makes the overall framework no longer strictly invertible, it adaptively extracts relevant details from the observed image based on the feedback of the INN. Moreover, AInvDis’s architecture constrains PCN to focus on the generation of missing details, which effectively prevents AInvDis from over-relying on the CNN model that has superior mapping capabilities to restore the entire image, thereby ensuring that the restored image details are maximally preserved through the INN transformation while the missing details are compensated by PCN. Experimental results on both synthetic and real-world datasets demonstrate that our proposed AInvDis not only outperforms recent deraining methods in traditional image assessment metrics but also achieves better perceptual quality on metrics PI and BRISQUE.
Few-shot defect segmentation (FSDS) has emerged as an effective approach to address the challenge of limited annotated samples in industrial quality inspection. However, existing methods often rely on features extracted from a single semantic level, without effectively suppressing noise interference, which leads to suboptimal performance when segmenting complex and diverse defect patterns. Motivated by these limitations, we propose a novel framework named Hierarchical Dense Aggregation Network (HDANet), which systematically integrates multilevel complementary information through three targeted components: (1) The Prior Disentanglement Module (PDM) explicitly mitigates semantic confusion between defective and defect-free regions by introducing dual training-free prior masks that provide robust structural guidance. (2) The Activation Enhancement Module (AEM) performs dense feature activation to capture fine-grained multiscale details while suppressing class-irrelevant noise. (3) The Progressive Aggregation Module (PAM) models hierarchical long-range dependencies among discrete defects and aggregates contextual relationships across multiple semantic levels. To alleviate the scarcity of annotated metal surface defect data, we further introduce MSD-Seg, a comprehensive benchmark dataset comprising 1,109 high-quality pixel-level annotated images covering 24 representative defect categories from five types of industrial materials. Extensive experiments on MSD-Seg, FSSD-12, and Defect-4i demonstrate that HDANet achieves superior defect segmentation performance. The dataset and code will be made publicly available at https://github.com/VDT-2048/HDANet.
Organic light-emitting diode (OLED)-based augmented and virtual reality (AR/VR) technologies are becoming increasingly integral to everyday life. However, these devices often face significant challenges, including high energy consumption and insufficient charging capabilities under typical usage conditions. Moreover, ensuring stereo consistency remains a significant issue in AR/VR devices. While stereo consistency has been extensively studied, most existing research has focused on stereo image restoration, with limited attention to power-constrained contrast enhancement (PCCE) in practical applications. To address these challenges, we propose an adaptive parallax-based contrast enhancement method for stereoscopic OLED displays aimed at reducing power consumption while enhancing visual contrast in the context of stereo consistency challenges. Specifically, our approach introduces a residual parallax attention module (RPAM), which dynamically adapts to different levels of parallax complexity and efficiently extracts multi-scale features, enhancing the overall consistency and quality of the stereo image pairs. Meanwhile, dilated convolution and channel attention mechanisms are incorporated within a power-aware attention module. Local enhancement techniques are integrated to adaptively regulate feature brightness in stereo pairs, enabling dynamic self-view and cross-view feature brightness adjustment. We compared our method with previous deep learning-based single-view PCCE methods and also conducted ablation studies on the parallax attention module (PAM). Image quality assessment metrics were used to evaluate image similarity and contrast. Comprehensive experiments conducted on three benchmark datasets confirm the effectiveness of our proposed approach in stereo PCCE methods.
Hyperspectral video data provide complementary contextual cues across spectral, spatial, and temporal dimensions for modeling object dynamics under challenging conditions. Many existing hyperspectral video object tracking (HVOT) approaches organize spatial-spectral and temporal modeling in successive stages, leaving room for closer interaction among video-level contextual cues. To address this, we propose HucrTrack, a unified contextual reasoning framework for HVOT trained by parameter-efficient fine-tuning (PEFT). HucrTrack forms synchronized hyperspectral and false-color representations from each hyperspectral cube and enhances spatial-spectral features through a weight-shared dual-representation backbone with unified contextual cue modeling. To effectively leverage contextual dynamics, we design a unified contextual reasoning module (UCRM) composed of three key components: memory dynamics unit (MDU), contextual injection unit (CIU), and selective retrieval unit (SRU). Specifically, MDU maintains a frame-wise dynamic memory via Mamba's hidden states; CIU hierarchically integrates this memory into the spectral-spatial backbone features; and SRU selectively retrieves relevant contextual information to reinforce the tracking representation. In contrast to representative stepwise designs, HucrTrack enables concurrent, unified reasoning over all three dimensions within one recurrent process. Extensive experiments on ten benchmarks demonstrate that HucrTrack compares favorably with existing trackers in both robustness and generalization.
Benign prostatic hyperplasia (BPH) is a prostate lesion, which is widely prevalent among middle-aged and elderly male individuals. Accurate segmentation of prostate structures in MRI imaging plays a critical role in aiding clinicians to assess the condition of the prostate gland. However, the boundary of the prostate gland often exhibits semantic features similar to those of adjacent connective tissues, leading to semantic confusion during prostate MRI segmentation in current methodologies. This phenomenon makes semantic ambiguity, which compromises clinicians’ judgment of glandular conditions and may result in misdiagnosis of the patient’s disease progression. This challenge imposes significant demands on existing prostate MRI segmentation models. Current models based on CNNs or Transformer architectures struggle to effectively capture long-range dependencies, rendering them prone to semantic confusion when processing images with indistinct boundaries. This limitation further exacerbates semantic ambiguities in the prostate MRI segmentation results. To mitigate the semantic ambiguity in prostate MRI segmentation in BPH, we propose the EScAPe, it contains Semantic Link Gates and a Cross-Layer Retro Connection. The Semantic Link Gate explicitly extracts semantic features at each point, mitigating information loss and semantic ambiguity. The Cross-Layer Retro Connection connects the initial and final layers, enabling the output to revisit the original features and preventing the loss of critical local features during sampling. We evaluate EScAPe on the MSD, NCI-ISBI 2013 and PICAI datasets compare its performance with existing Sota methods, and it achieved IoU scores of 61.66±4.99%, 80.18±%5.22, and 70.26±2.43% and Dice scores of 70.90±5.08%, 85.40±5.50%, and 81.15±2.18% on these three datasets respectively. Across most evaluation metrics, experiment shows that the EScAPe we proposed can solve the above-mentioned semantic ambiguity problem.
Spike camera is a kind of bio-inspired neuromorphic camera which is designed for capturing dynamic scenes of ultra-high speed motion at extremely high temporal resolution. It adopts an “integrate-and-fire” mechanism to convert the dynamic arrivals of photons at each sensor pixel to a stream of asynchronously fired spikes. The occurrence of spike firing may be disturbed by multiple factors, including the Poisson nature of photon arrivals and the quantization effect in spike readout. Therefore, recovering high-quality visual images from the recorded spike stream is an important yet challenging problem. This paper presents a reconstruction scheme for spike camera based on Bayesian imaging framework. We discuss the probability model of photon arrival in the imaging process and derive the likelihood model of the Bayesian framework. To fully exploit the dynamic information of spike streams, temporal correlation is used to model the image prior of reconstructed scenes. To be specific, we propose an auto-regressive model based on a non-stationary Laplacian distribution to model the temporal correlation. An efficient way to solve the optimization problem of the proposed Bayesian framework is further given. Experimental results show that the proposed method improves the reconstruction quality on both real-captured and synthesized spike datasets.
Multi-phase contrast-enhanced CT (CECT) is widely employed to capture the dynamic enhancement patterns and temporal evolution of organs and lesions. However, acquiring multiple phases increases radiation exposure and is inevitably accompanied by inter-phase misalignment and inconsistencies due to patient motion and the temporal variations in contrast uptake. Further dose reduction exacerbates noise and streak artifacts, severely degrading image quality and diagnostic reliability. In this work, we propose a novel reconstruction framework for multi-phase low-dose CECT that is guided by a routinely acquired non-contrast CT scan under weakly paired conditions. Specifically, the reconstruction model was formulated that explicitly separates common anatomical structures from phase-specific contrast variations and noise by deep dictionary representations. Then we employ a proximal gradient optimization method, analytically deriving its iterative procedure and unfolding it into an end-to-end trainable architecture, which preserves the theoretical interpretability of the model and facilitates efficient inference. To enhance structural alignment, we integrate local optimal transport to establish anatomically meaningful correspondences across phases, thereby enforcing structural fidelity and radio-density consistency. Extensive experiments on real clinical multi-phase datasets demonstrate that our method effectively suppresses noise and streak artifacts while recovering fine contrast-enhanced details. Both quantitative evaluation and expert clinical assessment confirm its superior performance compared with existing approaches. Moreover, downstream evaluation using the TotalSegmentator liver-lesion model shows substantial gains in hepatic tumor detectability under reduced-dose settings, enabling reliable lesion identification while significantly lowering radiation exposure. The codes and models are available at https://github.com/lixing0810/LOT-NCIRecon.
Deep Neural Networks (DNNs) remain vulnerable to adversarial perturbations, raising significant concerns in image processing applications, particularly in high-stakes domains such as medical imaging and security-critical systems. Most existing defense strategies are limited by domain specificity, architectural dependence, or the need for extensive retraining, making them impractical for real-world deployment. In this work, we propose ADEx, the first framework to integrate low-rank image approximation with explainability-driven analysis for the detection of adversarial samples. ADEx works by extracting a low-rank representation of the input image using Singular Value Thresholding (SVT), and identifying important image regions by computing class-specific gradient maps from the final layers of the classifier. These maps are then compared using Rank-Biased Overlap (RBO) to quantify the degree of attention drift induced by adversarial perturbations. ADEx is designed for adversarial detection in image classification systems, where class-specific gradient-based explanations are well defined. The framework operates without retraining or architectural modification and can be applied to a wide range of differentiable classifiers, provided gradient access is available for explanation generation. Extensive experiments across multiple datasets, architectures, and attack types demonstrate consistent performance, robustness to hyperparameter choices, and low sensitivity to calibration size. The method provides an interpretable and lightweight solution suitable for practical deployment.
In semi-supervised object detection (SSOD), due to the limited availability of labeled data, the quality and quantity of pseudo labels generated from unlabeled images are crucial for model training. Our study reveals that in the early stages of training, the number of usable pseudo labels is very low, which hampers the model’s ability to converge quickly. Furthermore, throughout the training process, even though pseudo-labels are generated with high confidence, a significant number of labeling errors still occur. To address these challenges, we propose a novel framework, Generation-to-Optimization of Pseudo Labels (GO-PL), which comprehensively improves the model’s performance in both the pseudo labels generation stage and the pseudo labels optimization stage. Specifically, we design a CLIP-based Pseudo Label Generation (CPLG) module, which matches proposals from the teacher model with CLIP text embeddings to generate semantically consistent high-confidence pseudo labels. These labels are further integrated into student training via an auxiliary loss, enabling early-stage guidance with stable and informative supervision signals. We also introduce Pixel-guided Instance Weighting (PIW) module that utilizes the SAM to assess the pixel-level quality of pseudo labels generated by the teacher model. Combined with a local voting mechanism, PIW dynamically assigns weights to each pseudo-label, thereby optimizing the impact of labels with varying quality on gradient updates. GO-PL demonstrates strong performance on standard SSOD benchmarks, including the MS-COCO and PASCAL VOC datasets. Notably, it achieves 26.7% mAP with only 1% labeled data on the MS-COCO, surpassing the previous best approach by 1.7%.
The minimization of ℓ2,p norm is a powerful regularization for robustness feature extraction, sparse representation and data denoising. However, its potential for clusters distribution modeling remains largely unexplored. In this paper, we present theoretical evidence that the ℓ2,p norm can effectively promote clusters balance with the maximization (0 < p < 2). Based on this theoretical foundation, we present a discrete regression clustering framework which incorporates the proposed regularization. Compared with existed anchor-graph multi-view clustering methods, our proposed method explicitly leverages the probabilistic nature of anchor graphs and realizes collaborative clustering for anchor points and sample points. It is important that our method can guarantee the balanced anchor points distribution which helps improve the robustness. Experimental results validate the effectiveness and superiority of the proposed approach.
Multimodal human activity recognition (HAR) benefits from complementary skeleton, inertial, and visual observations. However, many learning-based models still treat relationships among modalities, joints, and sensor variables as symmetric associations. This limits their ability to represent asymmetric information flow and can weaken the preservation of modality-specific cues during feature fusion. We propose Causality-Inspired Structure Representation Learning (CSRL), a multimodal HAR framework that uses directional dependency modeling as a structural prior for representation learning. CSRL first estimates transfer-entropy-based graphs from temporal entities, including skeleton joints and IMU sensor variables. These graphs provide asymmetric priors that guide recognition-oriented graph learning in the representation space. CSRL further combines hybrid contrastive learning with an encoder-decoder architecture to learn modality-invariant, modality-specific, and structure-aware representations in a unified framework. This design encourages cross-modal alignment while retaining local motion cues that are important for fine-grained action discrimination. Experiments on five public HAR benchmarks, including UTD-MHAD, MMAct, CZU-MHAD, NTU RGB+D, and NTU RGB+D 120, show that CSRL consistently improves accuracy, F1 score, and recall over competitive supervised and contrastive baselines. These results support TE-guided directional structure modeling as a practical and interpretable prior for multimodal action recognition.
Mamba, a global context modeling paradigm with a selective scanning mechanism, has recently attracted increasing attention in multimodal image fusion. Multimodal fusion aims to preserve and enhance cross-modal complementary information to produce high-quality fused images. Despite its suitability, existing Mamba-based fusion methods often blur modality-specific feature boundaries, mine complementary cues insufficiently, and lack explicit modal-specific modeling. To address these limitations, inspired by cross-attention, we extend Mamba to a multimodal setting and propose a Cross-Mamba architecture. As a plug-and-play module, Cross-Mamba adaptively mines inter-modal complementary information via cross-modal screening, thereby strengthening cross-modal interactions. Moreover, to emphasize complementary cues, reduce redundancy, and lower aggregation complexity, we impose rank constraints to preserve the spatial low-rank structure of features, retaining more informative components and improving fusion quality. Finally, we introduce a masked semantic guidance mechanism to narrow the semantic gap between fusion and downstream tasks, enhancing downstream adaptability. Extensive experiments on multiple fusion datasets, including qualitative, quantitative, and ablation studies, demonstrate that our method achieves state-of-the-art performance.
RGBE tracking aims to integrate modality-aligned RGB and event features to perform temporal identity association and spatial localization of targets. Existing methods do not effectively mitigate the modality gap between RGB frames and event data. In response, we develop a hierarchical spatio-temporal alignment framework tailored to RGBE tracking. Spatially, our framework introduces Global Indirect Alignment (GIA) and Local Direct Alignment (LDA). GIA establishes modality-level alignment by projecting RGB and event backbones into a unified CLIP-based semantic space. In parallel, LDA enforces target-region matching between template and search regions to ensure precise spatial alignment. By coupling the coarse modality-level semantic correspondence from GIA with the fine-grained target-region correspondence from LDA, we establish a spatial alignment hierarchy from global cross-modal semantics to local target discrimination under CLIP’s textual guidance. TIMF further extends this hierarchy from spatial alignment to temporal-invariance alignment. By aligning predicted invariant features with features sampled from the same target sequence, it preserves identity consistency across frames and provides a temporal prior that is adaptively fused with current observations. Extensive evaluations on three benchmarks show state-of-the-art success rates: COESOT 67.0%, VisEvent 60.6%, and FELT 46.9%, with consistent improvements across both standard and challenging scenarios. Our source code is available at https: //github.com/RiverUp/TextBridge-Track.
Generalizable vehicle re-identification (ReID) seeks to develop models capable of adapting to previously unseen domains without additional fine-tuning or retraining. Most existing approaches attempt to learn domain-invariant representations by aligning data distributions across source domains. However, they often neglect the inherent domain-related redundancy within source images, which suppresses the learning of complementary features characterized by lower occurrence probabilities and weaker activations. To overcome this limitation, we introduce Unity in Diversity (UID), a framework of multi-expert knowledge adversarial learning and collaboration. UID incorporates a training-free mechanism to filter out domain-related redundancy in source images, thereby promoting the learning of complementary feature representations. Specifically, we design a Spectrum-based Transformation for Redundancy Elimination and Augmentation Module (STREAM), which generates two distinct types of image inputs for a two-stage complementary feature learning process. In the multi-expert knowledge adversarial learning phase, STREAM enables the model to acquire a diversified identity-oriented prompt set that captures subtle but discriminative visual cues critical for distinguishing highly similar vehicles. This multi-expert prompt set is progressively integrated into complementary feature representations through the proposed knowledge confrontation and collaboration mechanism, which substantially enhances the model’s ability to extract fine-grained and complementary information. Extensive experiments conducted on multiple benchmarks demonstrate that UID achieves state-of-the-art performance, validating its effectiveness and generalizability. Our code is available at https://github.com/KZYYYY/UID.
As a paradigm in continual learning, class incremental learning (CIL) aims to assimilate tasks with mutually exclusive label spaces in sequence while preserving previously established knowledge. Mitigating forgetting in CIL fundamentally relies on transferring knowledge across tasks. A straightforward exemplar-based approach promotes balanced knowledge transfer by replaying an equal number of samples from each old class. However, in the more challenging exemplar-free setting, this balance cannot be ensured because distillation-based cross-task knowledge transfer tends to focus more heavily on the knowledge acquired from the most recent tasks. To address the unfairness in knowledge transfer, we analyze the mechanisms underlying dark knowledge and introduce a Semantic Enhanced Knowledge Transfer (SEKT) method for exemplar-free CIL. Specifically, SEKT adopts a bi-flow framework. The first flow is the Semantic Guidance Flow (SGF), which is inspired by knowledge distillation and produces latent semantic distributions from the outputs of the old model to guide the new model toward generating similar distributions. The second flow is the Semantic Propagation Flow (SPF), which propagates latent early knowledge to the current task in order to mitigate the unfairness in knowledge transfer. SPF constructs a cross-task semantic similarity graph using aligned intermediate representations to enable semantic propagation. It employs an expert network to learn the pattern of semantic propagation, enabling real-time and stable semantic recovery during training. In contrast to the SGF that is more effective for transferring recent knowledge, the SPF learns complementary early knowledge through a semantic complementarity constraint. Moreover, the SPF is robust to noisy semantics, as the learned semantic distribution is regularized with an $\ell _{2,1}$ norm. Extensive experiments conducted on six datasets demonstrate the superiority of the proposed SEKT over existing exemplar-free CIL approaches.
