Underwater images play a vital role in marine exploration, but are often severely degraded due to complex imaging conditions, including color distortion, haze effects, and non-uniform illumination. Existing deep learning-based enhancement methods predominantly rely on conventional RGB sensors, which struggle to distinguish between scattered and reflected light, thereby limiting enhancement performance. Polarization imaging, with its capability to capture directional light information, offers promising potential for underwater image enhancement. In this paper, we propose a lightweight yet effective polarization feature extractor that captures global spatial cues from polarization images. Additionally, we design a polarization-guided feature integration module that adaptively enhances the representational capacity of RGB features. Notably, the proposed module is plug-in and can be seamlessly integrated into existing RGB-based enhancement networks. Extensive experiments across multiple datasets demonstrate that incorporating polarization information significantly improves enhancement performance, highlighting its effectiveness as a valuable cue for underwater image enhancement. The code and pretrained models are at https://github.com/jgy0/UPGD.
Zero-shot graph learning (ZGL) is essential in many real-world applications, where models are expected to generalize to unseen graph structures and label distributions without task-specific fine-tuning, enabling knowledge transfer across diverse and evolving graph domains. Despite progress in graph representation learning, extracting universal task-agnostic features remains challenging, hindering plug-and-play zero-shot generalization across graph domains. To address this gap, we present ScaleGraph, a scalable self-supervised framework that alleviates feature misalignment, structural heterogeneity, and domain bias through three key innovations: (1) a parameter-free graph tokenizer that aligns heterogeneous feature spaces across domains via nonlinear kernel mapping, (2) a hierarchical transformer with linear attention that captures multi-scale structural dependencies, and (3) a lightweight classifier scheduler that achieves meta-learned domain adaptation through optimal transport. Extensive experiments demonstrate that ScaleGraph achieves state-of-the-art (SOTA) performance on cross-domain node classification and link prediction, highlighting its strong adaptability and effectiveness for zero-shot graph learning.
Cooperating intensity/RGB frames with event signals offers a practical and effective solution for high-speed object detection, while existing methods rely on frequent RGB frame updates, typically every 0.2 seconds, to preserve scene context. This leads to high computational costs, redundancy across frames, and latency issues, especially on edge devices. In this work, we propose the Delta Perception Network (DPNet), a novel learning-based detector that requires only a single RGB frame at the beginning of a sequence to detect high-speed objects over a 5-second duration, 25 times longer than prior methods. The key insight is that the initial frame can serve as a durable scene prior, while the continuous event stream supplies sufficient motion information to update this prior without needing further intensity frames. This design allows DPNet to track dynamic scene changes efficiently and accurately over extended periods. At the core of DPNet is a Delta Perception module that emphasizes temporal changes, and a Grid Sampling GRU that incrementally maintains and refines the scene representation. Extensive experiments show that DPNet outperforms state-of-the-art frame-based, event-based, and fusion-based detectors. Our results demonstrate that long-duration, high-speed object detection is not only feasible but also efficient using a single intensity/RGB frame and continuous event input, enabling real-time performance with reduced computational demands.
Existing multimodal image fusion methods mainly emphasize network design and fusion rules to combine complementary information from different modalities. However, the supervision information adopted during training is often not aligned with the evaluation metrics employed to assess fused images. As a result, this misalignment biases optimization toward pixel similarity rather than perceptual quality and detail preservation, which causes blurred edges, texture loss, and modality imbalance. In addition, when task labels are unavailable, many approaches fail to fully exploit semantic relations between modalities. This weak alignment of cross-modal features leads to missing salient regions. To address these issues, we propose DPSFusion, a novel fusion framework that integrates a dynamic pseudo-supervision (DP) mechanism and a semantic guidance (SG) module. Specifically, the DP mechanism constructs pseudo-supervision images from diverse candidate fused results and adaptively identifies high-quality references using a multimetric Fusion Quality Score (FQS). The pseudo-supervision set is subsequently iteratively updated during training, which ensures that the optimization objective gradually aligns with the evaluation metrics employed to assess fusion quality. In parallel, SG module is designed to exploit semantics without task-specific labels in order to produce fused images with sharper structures and higher target saliency. It extracts multi-level embeddings from infrared, visible, and pseudo-supervision images using a Contrastive Language-Image Pre-training (CLIP) model and converts them into feature modulation inside the network to highlight modality strengths and suppress redundancy. Finally, experiments on MSRS, RoadScene, TNO, and M3FD show state-of-the-art results on six standard fusion metrics and improved downstream performance in semantic segmentation and object detection, demonstrating the effectiveness of DPSFusion.
The Vision Transformer has gained popularity as a neural network architecture for event-based vision tasks. However, its use on resource-constrained devices is limited due to high computational and memory costs. This paper presents an Efficient Vision Transformer, a novel backbone for object tracking with event cameras. Specifically, we propose two adaptive token sparsification strategies based on the inherent characteristics of event data and tracking tasks, thereby reducing the computation while maintaining comparable performance. Firstly, since events are spatially sparse at pixel locations, we adaptively predict the sparsification ratio based on statistical entropy analysis and subsequently remove search tokens exhibiting dissimilarity to template tokens. Besides, we further eliminate redundant tokens by estimating the importance score of each token given the tracking targets. By hierarchically pruning more than 60
Emotion recognition is essential for improving user experience and interaction quality in human-centered applications. While recent studies have leveraged both event and traditional cameras to enhance eye-based emotion recognition, their practical deployment is hindered by the scarcity of event cameras and the complexity of dual-modality frameworks. Personalization, which is critical for handling individual differences in emotional expression, is also affected by these factors, resulting in reduced performance and adaptation efficiency. To address these challenges, we propose a lightweight and personalized single-eye emotion recognition network, called LPSEER. LPSEER introduces a novel hybrid neural architecture that integrates a convolutional neural network (CNN) and a spiking neural network (SNN) to capture spatiotemporal features from video frames and events, respectively. Additionally, we design a memorybased event feature inference (MEFI) module that recalls event features from video frames, eliminating the reliance on event cameras during inference and personalization while retaining the discriminative advantages of event-based representations. Experimental results demonstrate that LPSEER achieves state-of- the-art recognition accuracy while maintaining the smallest model size and lowest computational cost. Further experiments confirm the strong generalization capabilities and the ability to achieve faster, more accurate personalization. These advantages collectively enable lightweight, accurate, and efficient emotion recognition for real-world human-centered applications.
Answer grounding in visual question answering aims to locate the region from a given natural language question associated with the visual content of an image, which has garnered significant attention due to its practical applications. In this paper, we introduce the Dynamic Dual-level Vision Transformer Fusion Network (DDVT) for answer grounding in visual question answering. Specifically, we propose a question-guided dynamic regional-level module (QGDR) that combines complementary image context through ROI Align and text content, enabling precise localization of text-related visual content. Moreover, we present a cross-modal multi-scale aggregation module (CMA) that enhances feature fusion between pixel-level and region-level features, facilitating the effective localization of visual content associated with grounded answers. Furthermore, we fuse the located visual content with text features to locate the region and provide answers to questions posed about the image. Experimental results demonstrate that our DDVT outperforms state-of-the-art methods on several widely-used benchmarks.
The limited visual information in low-resolution (LR) fine-grained images reduces the sensitivity of the model to local distinct details, which leads to the models being more susceptible to non-discriminative feature interference and degrades the recognition accuracy. To address this issue, we propose a Part-aware attention Calibration and dynamic Enhancement Network (PCEN) to explore the fine-grained visual distinctions in LR inputs by adjusting the attention to patch features in the middle layer of the model and dynamically enhancing local key features. Specifically, a Part-aware attention Calibration (PC) module is developed to mine diverse and discriminative part representations from LR images by using learnable part query embeddings to perceive and link local fine-grained features. To stimulate the potential of part query embeddings to focus on more discriminative local features, we design an attention calibration loss to adjust the attention activation intensities of part query embeddings on patch embeddings in the middle encoder layer. Next, a Dynamic Enhancement (DE) module with dynamic reconstruction loss is introduced to make the local fine-grained features of crucial LR patches as close as possible to those of HR patches, thus capturing more accurate fine-grained clues. Extensive comparative experiments and ablation analyses are performed on five LR fine-grained datasets, including synthetic LR datasets (CUB-S, Nabird-S, Dog-S, and Car-S) and a real LR dataset (RP-281), to validate the effectiveness of the proposed PCEN and its components.
Most existing RGB-Event trackers rely on strictly aligned datasets, overlooking the asynchronous spatio-temporal resolutions common in real-world scenarios. This methodological limitation impedes effective RGB-Event feature alignment and ultimately degrades tracking performance. To overcome this limitation, we propose AlignTrack, a novel tracking framework built upon a Top-Down Alignment (TDA) strategy inspired by the human visual system. Our TDA framework follows an encode-decode-align paradigm: it first encodes multimodal features to generate target-related priors, which are then progressively decoded to guide a subsequent feature alignment pass. Within this framework, we introduce two key innovations: (1) a Cross-Prior Attention (CPA) module that effectively generates and integrates cross-modal priors, and (2) a Cross-Modal Semantic Alignment (CSA) loss that maximizes mutual information to enforce semantic consistency between modalities. Extensive experiments show that AlignTrack achieves state-of-the-art performance on four challenging RGB-Event tracking benchmarks, demonstrating its robustness in both aligned and unaligned scenarios. Ablation studies further validate the significant contribution of each proposed component.
Eye tracking is critical for VR/AR applications, requiring low-latency, high-frequency systems due to rapid eye movements. Event cameras, with high temporal resolution and dynamic range, are well suited for this task. However, most existing event-based eye tracking (EET) methods focus on discrete event features while ignoring eye structural information, limiting accuracy and practicality. Therefore, this paper presents a global-local spatiotemporal modeling scheme that leverages both eye structure and event characteristics. Specifically, we first propose a multi-scale information extraction module to derive rich local structural features such as eyelid morphology, which facilitates adaptation to scale variations of the eye. Given events' intrinsic discreteness, we further propose a module for modeling long-range feature dependencies to mitigate the challenges posed by sparse spatial information. These two modules can be seamlessly integrated with LSTM to extract spatiotemporal features. Extensive experiments on different event-based datasets validate that the proposed approach surpasses existing state-of-the-art methods.
Emotion recognition based on electroencephalogram (EEG) signals has significant potential for applications in mental health and human-computer interaction. However, existing methods often fail to fully exploit the intricate temporal dynamics and spatial dependencies inherent in EEG data. To address these challenges, we propose ST-SNN-DSFE, a novel framework that integrates temporal and spatial feature extraction to improve recognition performance. From the temporal perspective, Spiking Neural Networks (SNNs) are employed to capture fine-grained temporal dynamics, leveraging their biological plausibility and efficiency in processing temporal spikes. Additionally, the inherent characteristics of SNNs provide a slight denoising effect, which is advantageous for handling the noisy nature of EEG signals and enhancing the robustness of temporal feature extraction. From the spatial perspective, multi-scale convolutions are utilized to extract global physical information and positional encodings from EEG signals comprehensively, while prior knowledge is incorporated to define two distinct subgraph partitioning strategies based on brain regions and hemispherical divisions to extract local functional information and functional virtual positional encodings, enabling robust modeling of functional connectivity. Finally, Graph Convolutional Networks (GCNs) are employed to fuse the temporal and spatial features by integrating feature embeddings and positional relationships, improving emotion recognition performance. Extensive experiments on two benchmark datasets (SEED and SEED-IV) demonstrate the effectiveness of the proposed method, achieving state-of-the-art performance in both subject-dependent and subject-independent scenarios.
Mirror detection in dynamic scenes plays a crucial role in ensuring safety for various applications, such as drone tracking and robot navigation. However, current mirror detection models often fail in areas with mirrors that have a similar visual and color appearance to their surrounding objects. They also struggle to generalize well in complex cases, primarily due to limited annotated datasets. In this work, we propose a novel temporal prompt learning network with depth memory (TPD-Net) to address these critical challenges. Our approach includes several key components. First, we introduce a Temporal Prompt Generator (TPG) to learn temporal prompt features. Then, we devise Multi-layer Depth-aware Adaptor (MDA) modules to progressively adapt prompt features from the TPG, thereby learning mirror-related features by embedding temporal depth information as guidance. Moreover, we further refine these mirror-related features by constructing a depth memory and a Depth Memory Read module to read the temporal depths stored in the memory, boosting video mirror detection. Experimental results on a benchmark dataset show that our TPD-Net significantly outperforms 22 state-of-the-art methods in video mirror detection tasks.
Aiming at the industrial challenges in rail surface defect detection, such as online requirements, high missed detection rate of small targets, low distinguishability of defect features, and unbalanced sample distribution, this paper proposes a multi-module collaborative optimization real-time detection method based on the lightweight YOLOv11. Firstly, a C3k2-Sobel feature extraction module is designed, which integrates Sobel gradient prior information into convolution operations to effectively enhance the model's ability to perceive and extract defect edge features. Following this, a novel patch-level convolutional block attention mechanism is proposed innovatively to address the bottleneck of small target detection. By focusing on tiny defect regions through a block pooling strategy, it significantly suppresses background noise interference and thereby reduces the missed detection rate of small targets. In the final stage, to optimize the model training process, a dynamically weighted Focal Loss function is customized to specifically solve the dual imbalance problems of positive/negative samples and easy/hard samples in the dataset. Experimental validation was performed on a dataset comprising 1062 field-collected surface images of steel rails, capturing five representative defect categories under actual operational conditions. The proposed method demonstrates superior detection performance: it achieves a 13.3 %age-point improvement in mAP50 over the baseline YOLOv11 and obtains 57.3% Precision, 61.1% Recall, and 60.5% mAP50. Among the compared detectors, it achieves the highest Recall and mAP50 while maintaining a real-time inference speed of 38 FPS. In addition to offline testing, the proposed system was evaluated for six months on an actual steel rail production line before the rails were laid or put into service. The production-line results show an overall defect detection rate of 85.3% in terms of recall while maintaining real-time inference capability. These results demonstrate that the proposed framework provides an effective, real-time, and industrially applicable solution for online steel rail surface defect screening, with the potential to reduce missed detections and support manual re-inspection under production-line conditions.
Panorama has a full FoV (360$^\circ\times$180$^\circ$), offering a more complete visual description than perspective images. Thanks to this characteristic, panoramic depth estimation is gaining increasing traction in 3D vision. However, due to the scarcity of panoramic data, previous methods are often restricted to in-domain settings, leading to poor zero-shot generalization. Furthermore, due to the spherical distortions inherent in panoramas, many approaches rely on perspective splitting (\textit{e.g.}, cubemaps), which leads to suboptimal efficiency. To address these challenges, we propose $\textbf{DA}$$^{\textbf{2}}$: $\textbf{D}$epth $\textbf{A}$nything in $\textbf{A}$ny $\textbf{D}$irection, an accurate, zero-shot generalizable, and fully end-to-end panoramic depth estimator. Specifically, for scaling up panoramic data, we introduce a data curation engine for generating high-quality panoramic depth data from perspective, and create $\sim$543K panoramic RGB-depth pairs, bringing the total to $\sim$607K. To further mitigate the spherical distortions, we present SphereViT, which explicitly leverages spherical coordinates to enforce the spherical geometric consistency in panoramic image features, yielding improved performance. A comprehensive benchmark on multiple datasets clearly demonstrates DA$^{2}$'s SoTA performance, with an average 38\% improvement on AbsRel over the strongest zero-shot baseline. Surprisingly, DA$^{2}$ even outperforms prior in-domain methods, highlighting its superior zero-shot generalization. Moreover, as an end-to-end solution, DA$^{2}$ exhibits much higher efficiency over fusion-based approaches. Both the code and the curated panoramic data will be released.
Homeostatic mechanisms play a crucial role in maintaining optimal functionality within the neural circuits of the brain. By regulating physiological and biochemical processes, these mechanisms ensure the stability of an organism's internal environment, enabling it to better adapt to external changes. Among these mechanisms, the Bienenstock, Cooper, and Munro (BCM) theory has been extensively studied as a key principle for maintaining the balance of synaptic strengths in biological systems. Despite the extensive development of spiking neural networks (SNNs) as a model for bionic neural networks, no prior work in the machine learning community has integrated biologically plausible BCM formulations into SNNs to provide homeostasis. In this study, we propose a Dynamic Weight Adaptation Mechanism (DWAM) for SNNs, inspired by the BCM theory. DWAM can be integrated into the host SNN, dynamically adjusting network weights in real time to regulate neuronal activity, providing homeostasis to the host SNN without any fine-tuning. We validated our method through dynamic obstacle avoidance and continuous control tasks under both normal and specifically designed degraded conditions. Experimental results demonstrate that DWAM not only enhances the performance of SNNs without existing homeostatic mechanisms under various degraded conditions but also further improves the performance of SNNs that already incorporate homeostatic mechanisms.
Most existing fine-grained classification methods focus on looking for informative patches and extracting discriminative features. However, when processing low-resolution (LR) images, sparse image information weakens the model’s capacity to perceive subtle fine-grained features. To address this limitation, we propose a novel network named PATP, comprising a Probe search Aggregation (PA) module and an attribute Text Prompt (TP) module. The PA module adaptively searches for, aggregates, and refines potential discriminative features in the visual space. Furthermore, the TP module delivers attribute-aware guidance customized for each sub-category, thereby supporting the PA module in handling challenging samples. The synergistic optimization between the above two modules can reduce the model’s reliance on specific informative patches during discriminative feature extraction, thus capturing more reliable inter-class differences. Extensive experiments and ablation studies on four LR fine-grained benchmarks validate the effectiveness of PATP.
Surgical Visual Question Localized Answering (Surgical-VQLA) is an emerging task that supports surgical education by generating accurate answers and localizing relevant anatomical regions based on visual content and textual queries. This task requires precise spatial reasoning and tight semantic alignment across modalities, which remain challenging for current models due to limited spatial sensitivity and insufficient semantic integration. Mitigating these limitations, we propose EndoVisLoc, a dedicated framework that enhances visual-textual interaction through structured attention and gated fusion. Specifically, we design a Parallel Multi Attention Module (PMAM) to capture different visual features, improving the perception of anatomical structures. We further develop a Dynamic Gated Fusion Module (DGFM) to adaptively inject semantic priors into visual features via gated control, facilitating robust cross-modal fusion. Finally, we introduce a Hierarchical Classifier Head (HCH) to refine the fused representations and jointly optimize answer prediction and spatial localization. Extensive experiments on the EndoVis-18-VQLA and EndoVis-17-VQLA datasets demonstrate the superior performance of EndoVisLoc, surpassing the state-of-the-art OTAS model by +5.72% ACC, +5.73% F-score, and +1.82% mIoU on EndoVis-18-VQLA and by +1.59% ACC, +2.48% F-score, and +0.33% mIoU on EndoVis-17-VQLA. These results confirm the consistent advantage of EndoVisLoc in both answer accuracy and precise anatomical localization.
Images captured by consumer electronic devices, such as mobile phones and digital cameras, often suffer from low-light degradation due to sensor limitations and imaging pipelines, which degrades visual quality and affects downstream vision tasks. Existing methods based on Convolutional Neural Networks (CNNs) and Transformers have dominated current low-light image enhancement (LIE) due to their excellent ability to model hierarchical features. However, CNNs operate in local receptive fields that cannot model long-range dependencies, while Transformers overcome this problem but incur substantial computational costs. To address these challenges, we propose MambaLIE, a Scene Light Intensity-Boosted Low-Light Image Enhancement method based on a State Space Model (SSM). We first introduce scene light intensity to improve the structural distribution of illumination, which is then gated with the low-light input to guide enhancement. To better model the illumination while maintaining computational efficiency, we propose the Locally Enhanced State Space Model (LESSM) for efficient light enhancement. Our LESSM contains two branches: an SSM branch and a Local Enhanced branch, where the former is used to model the long-range dependencies with linear time complexity, while the latter is used to enhance local feature representations. Extensive experiments demonstrate that MambaLIE outperforms state-of-the-art CNN-based and Transformer-based LIE methods on four widely used synthetic benchmarks and five publicly available real-world benchmarks in terms of accuracy, speed, and model size, making it suitable for practical deployment on resource-constrained devices.
Time series anomaly detection in real-world Industrial Internet of Things (IIoT) systems is pivotal for identifying unsafe conditions and implementing timely preventive measures. While diffusion models are popular for capturing complex patterns, they often struggle to balance diversity and fidelity across scenarios due to limited exploration of context-window logical inference relationships and trend-pattern cues. To address these challenges, we propose ICAD, a novel method that rethinks the role of inference and cues in IIoT time series anomaly detection. ICAD defines trend patterns and uncertainties in textual form and utilizes a fine-tuned large language model to encode these descriptions as conditions for the diffusion model, thereby enhancing its generalization across diverse data distributions. Additionally, a “reasoning network for contextual window” mechanism is designed to capture temporal dependencies between adjacent windows, complemented by multi-scale and spatial feature adaptive fusion modules to further enhance the predictive performance. Empirical evaluations across four benchmark datasets and a large-scale ethylene oxide production process demonstrate that ICAD consistently outperforms state-of-the-art baselines, confirming its effectiveness and practicality in overcoming current anomaly detection model limitations.
Camouflaged Object Detection (COD) aims to segment objects that blend seamlessly into complex backgrounds, with growing interest in exploiting additional visual modalities to enhance robustness through complementary information. However, most existing approaches generally rely on modality-specific architectures or customized fusion strategies, which limit scalability and cross-modal generalization. To address this, we propose a novel framework that generates modality-agnostic multi-modal prompts for the Segment Anything Model (SAM), enabling parameter-efficient adaptation to arbitrary auxiliary modalities and significantly improving overall performance on COD tasks. Specifically, we model multi-modal learning through interactions between a data-driven content domain and a knowledge-driven prompt domain, distilling task-relevant cues into unified prompts for SAM decoding. We further introduce a lightweight Mask Refine Module to calibrate coarse predictions by incorporating fine-grained prompt cues, leading to more accurate camouflaged object boundaries. Extensive experiments on RGB-Depth, RGB-Thermal, and RGB-Polarization benchmarks validate the effectiveness and generalization of our modality-agnostic framework.
Rynson W. H. Lau (劉永雄)合作论文数Department of Computer Science, College of Engineering, City University of Hong Kong;Swansea University17