Multimodal semantic segmentation has emerged as a powerful paradigm for enhancing scene understanding by leveraging complementary information from multiple sensing modalities (e.g., RGB, depth, and thermal). However, existing cross-modal fusion methods often implicitly assume that all modalities are equally reliable, which can lead to feature degradation when auxiliary modalities are noisy, misaligned, or incomplete. In this paper, we revisit cross-modal fusion from the perspective of modality reliability and propose a novel framework termed the Reliability-aware Self-Gated State Space Model (RSGMamba). At the core of our method is the Reliability-aware Self-Gated Mamba Block (RSGMB), which explicitly models modality reliability and dynamically regulates cross-modal interactions through a self-gating mechanism. Unlike conventional fusion strategies that indiscriminately exchange information across modalities, RSGMB enables reliability-aware feature selection and enhancing informative feature aggregation. In addition, a lightweight Local Cross-Gated Modulation (LCGM) is incorporated to refine fine-grained spatial details, complementing the global modeling capability of RSGMB. Extensive experiments demonstrate that RSGMamba achieves state-of-the-art performance on both RGB-D and RGB-T semantic segmentation benchmarks, resulting 58.8

Real-time semantic segmentation is a core component of environment perception in autonomous driving systems. Models based on convolutional neural networks (CNNs) and Transformers have achieved significant progress in this task. However, both paradigms face difficulty in efficiently capturing multi-scale features under strict lightweight computational constraints. To address these limitations, this paper proposes a Multi-scale Hybrid Mamba Network (MHMNet) by enhancing a state space model with linear computational complexity and integrating it with the Multi-branch Feature Aggregation Block (MFAB). The proposed hybrid architecture adopts an asymmetric encoder-decoder design. In the encoder, MFAB enables cross-scale and multi-dimensional feature extraction, while the decoder employs a Hybrid Mamba module to collaboratively model local details and global contextual information. Subsequently, the Gated Feature Selection Head (GFSH) is introduced to suppress noisy channel interference and enhance the contribution of discriminative features, producing the final segmentation output. This design significantly reduces computational complexity while preserving segmentation accuracy. The proposed MHMNet contains only 0.77M parameters and achieves mean Intersection-over-Union (mIoU) scores of 71.48% and 69.5% on the Cityscapes and CamVid benchmarks, respectively. Experimental results demonstrate that the proposed model maintains competitive segmentation accuracy under a substantially reduced parameter budget, effectively alleviating the trade-off between accuracy and efficiency in lightweight segmentation models and achieving a favorable balance between performance and computational complexity.

Class imbalance presents a persistent bottleneck in machine learning, biasing models toward majority classes and degrading performance on rare events. Active learning partially alleviates this by focusing annotation efforts on underrepresented samples. However, this issue becomes more pronounced in multi-domain learning settings, where both intra-domain imbalances and shifts across domains complicate model training. Current multi-domain active learning techniques typically neglect the interaction of these dual imbalances, resulting in biased selection strategies and limited ability to generalize across domains. To tackle this, we introduce a comprehensive framework featuring a dynamic category-scaling weight mechanism that hierarchically addresses imbalance at the sample and domain levels. This mechanism assesses the relative difficulty of each class to guide effective sampling, while concurrently constructing a progressively refined "Ideal Domain" through the adaptive integration of well-performing domains. This ideal representation further supports a minimax optimization process to allocate annotation budgets both fairly and robustly. Extensive evaluations on various standard datasets confirm that our approach consistently surpasses leading methods in managing complex imbalanced scenarios across multiple domains.

Lightweight image super-resolution (SR) aims to reconstruct high-resolution images from low-resolution images under limited computational costs. We find existing frequency-based SR methods cannot balance the reconstruction of overall structures and high-frequency parts. Meanwhile, these methods are inefficient for handling frequency features and unsuitable for lightweight SR. In this paper, we show introducing both wavelet and Fourier information allows our model to consider both high-frequency features and overall SR structure reconstruction while reducing costs. Specifically, we propose a Dual-domain Modulation Network that integrates both wavelet and Fourier information for enhanced frequency modeling. Unlike existing methods that rely on a single frequency representation, our design combines wavelet-domain modulation via a Wavelet-domain Modulation Transformer (WMT) with global Fourier supervision, enabling complementary spectral learning well-suited for lightweight SR. Experimental results show that our method achieves a comparable PSNR of SRFormer (Zhou et al., 2023) and MambaIR (Guo et al., 2024) while with less than 50% and 60% of their FLOPs and achieving inference speeds 15.4 & times; and 5.4 & times; faster, respectively, demonstrating the effectiveness of our method on SR quality and lightweight.

Low-light image enhancement (LLIE) remains two fundamental challenges: accurately recovering fine details and effectively suppressing noise. Existing methods often struggle to strike a balance between these two objectives, resulting in results that are either overly smoothed or contaminated by excessive noise artifacts. Our motivation stems from the fact that during LLIE, noise predominantly manifests in the low-frequency components of the image and tends to propagate uniformly across the image. Leveraging this insight, we propose the Diffusion-based Laplacian Frequency-aware Network (DLFN), which decomposes images into multi-scale high- and low-frequency components using Laplacian decomposition. For low-frequency noise suppression, we exploit the denoising capability of diffusion models to eliminate noise introduced during the enhancement process effectively. To prevent over-smoothing and preserve fine details, we introduce the High-Frequency Enhancement Module (HFEM), which enhances high-frequency details across multiple Laplacian subbands, optimizing texture restoration. Additionally, we design the Multi-Scale Bilateral Fusion Module (MBFM) to facilitate the progressive fusion of high- and low-frequency features. Extensive experiments show that our method outperforms existing LLIE approaches.

Diffusion-based face restoration that adjusts the sampling trajectory of pre-trained diffusion models has achieved remarkable progress. However, existing approaches provide insufficient constraints during reverse diffusion, causing identity-related structural drift and degraded fidelity under severe degradations. To address this, we propose WaveFreqAnchor, a training-free framework based on Wave-Structural Anchoring and Frequency Correction Diffusion. Specifically, Anchor-Space Wave-Structural Guidance (ASWG) constrains facial structures through anisotropic wave-response consistency, while Multi-scale Wavelet-Fourier Injection (MWFI) aligns the predicted low-frequency subband with the observation by replacing its phase, correcting inconsistencies accumulated during reverse diffusion. For real-world scenes, we further introduce Subband High-Frequency Enhancement (SHE), which performs bounded, spatially masked refinement on the predicted high-frequency subbands to recover fine facial details under unknown compound degradations. Together, these designs effectively preserve facial identity while restoring sharp and realistic facial details. Extensive experiments show that our method consistently outperforms existing methods, achieving high-quality and high-fidelity face restoration.

Videos captured in the wild often suffer from rain streaks, blur, and noise. In addition, even slight changes in camera pose can amplify cross-frame mismatches and temporal artifacts. Existing methods rely on optical flow or heuristic alignment, which are computationally expensive and less robust. To address these challenges, Lie groups provide a principled way to represent continuous geometric transformations, making them well-suited for enforcing spatial and temporal consistency in video modeling. Building on this insight, we propose DeLiVR, an efficient video deraining method that injects spatiotemporal Lie-group differential biases directly into attention scores of the network. Specifically, the method introduces two complementary components. First, a rotation-bounded Lie relative bias predicts the in-plane angle of each frame using a compact prediction module, which normalized coordinates are rotated and compared with base coordinates to achieve geometry-consistent alignment before feature aggregation. Second, a differential group displacement computes angular differences between adjacent frames to estimate a velocity. These biases are combined with temporal decay and a banded attention mask to emphasize short-range reliable relations while suppressing long-range noise. DeLiVR achieves sharper details, fewer rain remnants, and stronger temporal coherence on both synthetic and real rainy benchmarks.

Multi-modal fusion serves as a cornerstone for successful depth map super-resolution. However, commonly used fusion strategies, such as addition and concatenation, fall short of effectively bridging the modal gap. As a result, guided image filtering methods have been introduced to mitigate this issue. Nevertheless, it is observed that their filter kernels usually encounter significant texture interference and edge inaccuracy. To tackle these two challenges, we introduce a Scene Prior Filtering network, SPFNet, which utilizes the priors’ surface normal and semantic map from large-scale models. Specifically, we propose an All-in-one Prior Propagation that computes similarity between multi-modal scene priors, i.e., RGB, normal, semantic, and depth, to reduce the texture interference. Besides, we design a One-to-one Prior Embedding that continuously embeds every single modal prior into depth using Mutual Guided Filtering, further alleviating texture interference while enhancing edge representations. Our SPFNet has been extensively evaluated on both real-world and synthetic datasets, achieving state-of-the-art performance. Project page: https://yanzq95.github.io/projectpage/SPFNet/index.html .

Previous RGB-D semantic segmentation methods commonly employ dual encoders to separately process RGB and depth inputs, followed by dedicated modules for cross-modal feature fusion. However, such designs often inadequately capture depth representations and consequently limit effective cross-modal interaction, while the additional encoder branch introduces redundant computation that hinders lightweight execution. To tackle these challenges, we propose URNet, a Unified Reparameterized RGB-D Network that performs simultaneous multi-modal feature extraction and cross-modal fusion within a single encoder. Specifically, we adopt a reparameterization strategy to compact the network architecture and facilitate fast inference. Within each Reparameterized Block (RepBlock), a Linear Gated Attention (LGA) module is introduced to fully exploit complementary RGB and depth cues across different feature scales. Furthermore, considering that decoder design has been relatively underexplored in existing RGB-D segmentation models, we develop a concise yet effective universal decoder, termed the Pyramid Merging Decoder (PMD). Extensive experiments on multiple RGB-D segmentation benchmarks demonstrate that URNet achieves state-of-the-art performance while maintaining high efficiency. Code will be available at https://github.com/Wild-Stephen/URNet.

Federated learning (FL) allows multiple parties to collectively train deep learning models without the need to disclose their local data. The data distributions among various parties are usually non-independently and identically distributed (non-IID), and simultaneously the class imbalance problem often exits locally and globally, which is the main challenge of FL. Although some FL works have been presented aiming to solve this issue, there still exist much room to enhance the image classification effect by using deep learning models. In addition, under the non-IID setting, how to ensure the security of FL methods against the attack of malicious clients or central servers has not been well researched. We develop a novel decentralized FL approach in this paper, namely Blockchain-based Federated learning with Metric and Imbalanced Learning (BFMIL). The triplet loss is introduced to promote the consistency of feature representations between the client model and server model. To address the class imbalance problem, a cost-sensitive semantic discrimination loss is designed to fully explore the discriminative information, and data in each party is divided into the majority classes and the minority classes for unequal training. To reduce malicious attack, we utilize the blockchain to store the local update and the global model, and a novel voting mechanism is used to select parties with better model parameters for aggregation in each round of FL. The effectiveness of BFMIL is demonstrated by experiments conducted on four imbalanced datasets.

Face restoration (FR) is a specialized field within image restoration that aims to recover low-quality (LQ) face images into high-quality (HQ) face images. Recent advances in deep learning technology have led to significant progress in FR methods. In this paper, we begin by examining the prevalent factors responsible for real-world LQ images and introduce degradation techniques used to synthesize LQ images. We also discuss notable benchmarks commonly utilized in the field. Next, we categorize FR methods based on different tasks and explain their evolution over time. Furthermore, we explore the various facial priors commonly utilized in the restoration process and discuss strategies to enhance their effectiveness. In the experimental section, we thoroughly evaluate the performance of state-of-the-art FR methods across various tasks using a unified benchmark. We analyze their performance from different perspectives. Finally, we discuss the challenges faced in the field of FR and propose potential directions for future advancements. The open-source repository corresponding to this work can be found at https:// github.com/ 24wenjie-li/ Awesome-Face-Restoration.

Real-world video arrives hazy, rainy, dark, or noisy, and a deployable restorer faces three demands at once: no degradation label, native 4K output, and stability in playback. Existing methods answer them separately and break on the joint problem, because per-frame degradation readings flip between frames, downsampled proxies erase the rain and noise they are meant to remove, and dense temporal alignment does not fit 4K memory. No paired benchmark even poses that problem, so we build one. UHV-4K-AIO renders physically modeled haze, rain, sensor noise, and low light over the same 100 clean 4K clips with shared depth and motion, and its construction exposes the split MoCRA is built on: haze and low light survive aggressive downsampling, while rain and noise exist only at native scale. Band-matched compositional conditioning follows, spending conditioning capacity, computation, and supervision in the band where each degradation lives. One dictionary of rank-1 atoms, recomposed sparsely per frame, conditions both a once-per-clip coarse branch and a shallow native-resolution refiner, in 3.6M parameters and with no optical flow. Trained once for all four tasks, MoCRA takes the best task-mean PSNR of eleven retrained image and video baselines, holds warping error at the level of the flow-based video models while never estimating motion, and restores native 4K in under half a second, against 1.7 seconds for the fastest baseline.

In plenty of real-world applications, data are generated/collected in a streaming way, and it is hard to obtain their accurate labels of known (seen) classes. Moreover, there are several unknown (unseen/novel) classes would emerge with evolving stream data. In the literatures, existing approaches suffer from three limitations: (1) a gap in intra-class variance arises when seen classes are learned more faster than novel classes; (2) a significant issue arises regarding the imbalance in feature weighting among the learning procedures for both new and old classes; (3) a catastrophic forgetting can occur if we exclusively update the model with new data, resulting in the loss of knowledge acquired from known classes when integrating information related to the current novel classes. This paper investigates the problem of learning with unseen classes detection over a non-stationary data stream. Particularly, we introduce uncertainty adaptive margin mechanism from open-world semi-supervised learning to address the bias stemming from the faster learning of discriminative features for seen classes compared to novel classes. We also develop adaptive weighting scheme to dynamically balance the usage of seen and novel classes data by updating their aggregation weights. In addition, we propose a model updating scheme to gradually incorporated the stored memory and novel class information, thereby reducing the risk of forgetting distinctive attributes associated with known classes. Finally, we formulate the objective in a bi-level optimization that enables our model to maintain consistent performance under class distribution shifts, detect unseen classes with minimal supervision, and achieve robust continual learning in open-world streaming scenarios. Our empirical evaluation of this framework using real-world datasets highlights its superior performance when compared to existing methods.

The Vision Transformer (ViT) has achieved notable success in computer vision, with its variants widely validated across various downstream tasks, including semantic segmentation. However, as general-purpose visual encoders, ViT backbones often do not fully address the specific requirements of task decoders, highlighting opportunities for designing decoders optimized for efficient semantic segmentation. This paper proposes Strip Cross-Attention (SCASeg), an innovative decoder head specifically designed for semantic segmentation. Instead of relying on the conventional skip connections, we utilize lateral connections between encoder and decoder stages, leveraging encoder features as Queries in cross-attention modules. Additionally, we introduce a Cross-Layer Block (CLB) that integrates hierarchical feature maps from various encoder and decoder stages to form a unified representation for Keys and Values. The CLB also incorporates the local perceptual strengths of convolution, enabling SCASeg to capture both global and local context dependencies across multiple layers, thus enhancing feature interaction at different scales and improving overall efficiency. To further optimize computational efficiency, SCASeg compresses the channels of queries and keys into one dimension, creating strip-like patterns that reduce memory usage and increase inference speed compared to traditional vanilla cross-attention. Experiments show that SCASeg's adaptable decoder delivers competitive performance across various setups, outperforming leading segmentation architectures on benchmark datasets, including ADE20K, Cityscapes, COCO-Stuff 164k, and Pascal VOC2012, even under diverse computational constraints.

Image Super-Resolution (SR) aims to recover high-resolution (HR) images from their low-resolution (LR) counterparts. However, existing SR methods suffer from insufficient multi-level feature interaction, leading to increased computational complexity. To address this limitation, we propose an Interactive Multi-Feature Residual Network (IMFRN) for lightweight SR. To facilitate feature exchange across different levels, we propose the Interactive Distillation Feature Refinement Module (IDFRM), which refines hierarchical features through cross-stage distillation and residual aggregation. IDFRM includes the Multi-Branch Feature Attention Block (MFAB) to integrate spatial and channel information from multiple branches adaptively. Additionally, the Dual Attention Fusion Module (DAFM) dynamically enhances feature representations using complementary attention mechanisms. To strengthen the global context, we integrate a Transformer-based module. Our IMFRN effectively facilitates interaction between features at different levels, achieving state-of-the-art performance with reduced parameters and computational cost.

Recently, Mamba-based super-resolution (SR) methods have demonstrated the ability to capture global receptive fields with linear complexity, addressing the quadratic computational cost of Transformer-based SR approaches. However, existing Mamba-based methods lack fine-grained transitions across different modeling scales, which limits the efficiency of feature representation. In this paper, we propose T-PMambaSR, a lightweight SR framework that integrates window-based self-attention with Progressive Mamba. By enabling interactions among receptive fields of different scales, our method establishes a fine-grained modeling paradigm that progressively enhances feature representation with linear complexity. Furthermore, we introduce an Adaptive High-Frequency Refinement Module (AHFRM) to recover high-frequency details lost during Transformer and Mamba processing. Extensive experiments demonstrate that T-PMambaSR progressively enhances the model's receptive field and expressiveness, yielding better performance than recent Transformer- or Mamba-based methods while incurring lower computational cost. Our codes will be released after acceptance.

Object detection from drones in hazy conditions is particularly challenging under limited compute due to three bottlenecks: tiny target scales, severe degradation, and dense target distributions. Existing methods based on high-level semantic detectors often fail here because haze erodes low-level visual cues crucial for localization and fine-grained discrimination. We observe that depth provides spatial priors for small-object localization and scene disentanglement in crowded scenarios, while frequency cues preserve high-frequency details and aid robust feature separation under haze. Building on this complementarity, we propose HazeRes-DFDet, a Haze-Resilient Depth–Frequency Detector comprising: (1) an Attention-Convolution Mix (ACmix) to extract multi-scale pyramid features, (2) a Cross-Domain Hierarchical Attention (CDHA) to capture and refine depth-aware features, and (3) a Frequency-Aware Depth Fusion Module (FDFM) that integrates depth and frequency information across scales. Extensive experiments show that HazeRes-DFDet improves mAP by 1.3% on synthetic datasets and 0.3% on real-world datasets, while reducing parameters by over 15% compared to state-of-the-art methods.

Face images captured in real-world low light suffer multiple degradations-low illumination, blur, noise, and low visibility, etc. Existing cascaded solutions often suffer from severe error accumulation, while generic joint models lack explicit facial priors and struggle to resolve clear face structures. In this paper, we propose PASDiff, a Physics-Aware Semantic Diffusion with a training-free manner. To achieve a plausible illumination and color distribution, we leverage inverse intensity weighting and Retinex theory to introduce photometric constraints, thereby reliably recovering visibility and natural chromaticity. To faithfully reconstruct facial details, our Style-Agnostic Structural Injection (SASI) extracts structures from an off-the-shelf facial prior while filtering out its intrinsic photometric biases, seamlessly harmonizing identity features with physical constraints. Furthermore, we construct WildDark-Face, a real-world benchmark of 700 low-light facial images with complex degradations. Extensive experiments demonstrate that PASDiff significantly outperforms existing methods, achieving a superior balance among natural illumination, color recovery, and identity consistency. Code and dataset will be available at https://github.com/IVIPLab/PASDiff.

Autonomous aerial vehicles (AAVs) have garnered significant attention due to their operational flexibility, enabling expanded application scenarios across diverse fields. The Text-Based Pedestrian Retrieval (TBPR) task aims to identify corresponding images from textual descriptions, yet existing research has primarily focused on ground-level views. To broaden the applicability of TBPR systems, we introduce aerial-view analysis and propose a novel Text-Based Aerial Pedestrian Retrieval (TBAPR) task. This task introduces unique challenges, particularly the dual gaps in cross-view (aerial vs. ground) and cross-modal (text vs. image) matching, which are more complex than traditional TBPR or aerial-ground pedestrian understanding tasks. To address these challenges, we propose an Adaptive Elastic Alignment Network with FIne-Grained Representation Mining (AEA-FIRM). Our framework tackles the cross-view gap through an AEA loss that adaptively prioritizes critical semantic features while dynamically aligning textual and aerial semantics under challenging conditions. Concurrently, the FIRM module refines visual-linguistic representations by mining fine-grained pedestrian attributes and explicitly textualizing them for cross-modal matching verification. Extensive experiments demonstrate that AEA-FIRM achieves state-of-the-art performance, outperforming existing TBPR methods by 4.87% in Rank-1 accuracy.

Exploring the query strategies based on deep learning shows promising results in terms of designing the criteria for active learning. However, the labels provided by the oracles might be noisy (inaccurate) due to similarities across several classes causing ambiguity, leading to unreliable results. To address this issue, we propose a noise-tolerant deep active learning method. Specifically, we design a consistency regularization for deep attention network as explicit regularizer, which is used to measure the uncertainty of examples. Besides, we develop the robust model for dealing with the noisy oracles, which first take the associations that make from embeddings of labeled data to those of unlabeled data and back, then we employ the association probability as a weighting fusion schema into angular margin based loss. Moreover, we design the submodular maximization function for reducing the redundancy of selected batch examples. Finally, the formulation is encapsulated into the multitask framework that helps to adaptive learning towards more generalizable performance. Experimentally, we conduct extensive experiments on classification and segmentation tasks, and the results clearly demonstrate the superiority of the proposed method to the existing state-of-the-art deep active learning approaches.