Stereo matching plays a crucial role in depth perception for autonomous vehicles. However, existing iterative approaches often incur high computational costs owing to redundant iterations, which fundamentally restricts their real-time applicability. To address this issue, we propose Context-Stereo-I, a lightweight iterative framework that reduces the number of iterations while enriching the hidden state with enhanced contextual and geometric information. Central to our approach is a Context-Guided Geometry Volume (CGV), constructed through gated interaction between a context-enhanced encoder and the decoder during cost aggregation. The CGV provides accurate initial disparity estimates and reliable matching cues for iterative refinement. Inside CGV, we propose a Gated Skip Connection (GSC) mechanism to improve information propagation from the encoder to decoder, effectively delivering more informative low-level features compared to standard skip connections. These low-level features are enhanced with contextual information via our proposed Enhanced Context Fusion (ECF) module. By integrating context-aware geometric cues into the iterative updater, Context-Stereo-I achieves outstanding performance in challenging regions and exhibits strong generalization in unseen scenarios. It ranks first on the KITTI 2012 and 2015 benchmarks among all published real-time methods, achieving a 3px-all error of 1.66% and a D1-all error of 1.73%, while running at 41 ms on an RTX 3080 GPU. Moreover, it demonstrates superior cross-domain generalization on both public datasets and real-world data compared to other real-time stereo matching approaches. Code is available at https://github.com/shidifen12/Context-Stereo.
Blind image deblurring remains a fundamental yet highly ill-posed problem in computer vision. Existing methods that rely on composite regularizations often lack a unified physical formulation, and thus struggle to balance structural preservation with fine-scale texture smoothing. Moreover, insufficient prior constraints on the kernel render these algorithms vulnerable to the larger-kernel effect, thus severely degrading performance when the kernel size is over-specified. To address these limitations, we propose novel image and kernel priors that derive from the inherent physical properties of clear images and ideal kernels. First, we introduce a flexible bimodal graph prior (FBP) that is based on graph signal processing. The FBP not only provides strong discrimination between sharp and blurred images but also flexibly promotes salient edge selection by enforcing a bimodal distribution of edge weights, which accurately recovers intermediate skeleton images. Subsequently, we reveal the inherent anisotropy in the patch distributions of ideal kernels via directional patch matching. On the basis of this crucial observation, we propose a discriminative kernel prior (DKP) with a clear physical interpretation. The DKP explicitly suppresses isotropic noise during kernel estimation, thereby robustly mitigating the larger-kernel effect and significantly increasing the tolerance of the algorithm to kernel size. Furthermore, we prove that the FBP can be well-approximated by the reweighted graph Laplacian (RGL) and derive an efficient alternating optimization scheme with closed-form solutions. The results of extensive experiments demonstrate that our method outperforms state-of-the-art algorithms on both synthetic and real-world datasets and exhibits superior efficacy in handling large-scale motion blur and defocus blur.
Contrastive Language-Image Pre-training (CLIP) has recently been explored for image-level weakly supervised semantic segmentation (WSSS) due to its semantic localization priors. However, CLIP is trained with global image-text alignment and lacks pixel-level discriminative supervision, making its category responses vulnerable to co-occurring non-target activations and confusion among semantically similar classes. Instead of merely refining visual response maps, this paper revisits CLIP-based WSSS from the perspective of text-prototype bias. We find that class text prototypes used for vision-text matching are jointly affected by instance-level co-occurrence semantics and category-level semantic coupling, which degrades response purity and inter-class separability. To this end, we propose TPD-CLIP, a text-prototype debiasing framework that calibrates biased class prototypes before vision-text matching to generate cleaner and more discriminative localization responses. TPD-CLIP consists of a Confusion-aware Semantic Decoupling Module (CSDM) and a Pairwise Semantic Confusion Matrix (PCM). CSDM leverages foreground and non-target region semantics to enhance target-semantic alignment while suppressing non-target interference. PCM further estimates asymmetric inter-class interference and adaptively modulates the separation strength for highly confusing negative categories. Experiments on PASCAL VOC 2012 and MS COCO 2014 showfinal that TPD-CLIP improves response quality, pseudo-label reliability, and final segmentation performance. Our code is available at https://github.com/jocka-weilen/TPD-CLIP.
The talking head generation task synthesizes videos from a single portrait image and audio input, animating the portrait to deliver the speech content. Non-autoregressive (NAR) approaches for talking head generation have demonstrated impressive quality and generation speeds by producing video frames in parallel, thereby overcoming the error accumulation problems inherent in frame-wise autoregressive (AR) methods. However, NAR methods have limited practical applicability due to prohibitive VRAM requirements, especially when generating long sequences (≥ 1000 frames) at high resolution (512 × 512). This paper proposes a novel framework that enables high-quality, non-autoregressive talking head generation while significantly reducing computational resource demands for both training and inference. We enhance efficiency through our Multi-stage Dynamics Compression Framework (MDCF), a two-stage pipeline that achieves a 768× compression ratio for pixel-level dynamics representations. Additionally, we demonstrate that this two-stage architecture cannot be effectively optimized via standard end-to-end training. We therefore introduce a Two-Phase Cascaded (TPC) training strategy to stably optimize the MDCF while effectively alleviating error accumulation during multi-stage compression. Experimental results demonstrate that our framework can non-autoregressively generate talking head videos with over 1600 frames at up to 512 × 512 resolution on a 16GB GPU, with state-of-the-art quality and inference speed. Our approach represents a significant advancement toward practical, resource-efficient talking head synthesis for real-world applications. The source code in the supplementary material will be publicly available.
Cross domain semantic segmentation faces significant challenges due to domain shift and the high cost of manual annotation. To address this problem, transfer learning has emerged as a key solution and it helps cross domain semantic segmentation by two manners. The first one is feature-level alignment by self-training or extracting domain invariant features, but the performance is limited by noisy pseudo label. The other is image translation-assisted approaches, which suffer from unsatisfactory translation results and the ignorance of potential semantic hidden features. Moreover, the synergistic relationship between transfer learning and semantic segmentation remains underexplored in existing methods, while the complementarity of feature-level and image-level alignment has received scarce attention. Hence, a novel end-to-end progressive co-training network (PCN) is proposed to overcome these shortcomings. Firstly, feature-level alignment branch and image translation branch both generate pseudo predictions and ensembled results are acquired, self-supervised learning and adversarial learning are utilized respectively. In order to ensure high-quality translation, semantic decoder and class center alignment module effectively consider class-wise visual consistency, while discrete wavelet transform further ensures sound structural details. Moreover, to facilitate the co-training of high-quality segmentation and nice transfer learning, a pseudo label refinement module is proposed by considering probability, information entropy and prediction consistency. Exhaustive experiments on ISPRS 2D Semantic Labeling Contest Dataset demonstrate the superiority of the PCN. The code will be available at https://github.com/lixinghua5540/PCN.
Text-to-image (T2I) generation has emerged as a crucial component of AI content generation, enabling the automatic creation of visual content from natural language descriptions. Within this rapidly evolving domain, evaluating whether a generated image and the associated text prompt are semantically aligned remains a significant challenge. The existing methods typically adopt implicit and holistic representations derived from pre-trained vision-language models, which fails to adequately explore semantic entities and their relationships in both visual and textual modalities. Motivated by the above facts, this paper presents Graph-Align, a novel graph-based T2I alignment evaluation model, which transforms T2I alignment into a graph matching problem. Specifically, a generated image and the corresponding text prompt are first represented as graph structures comprising nodes (entities) and edges (relations). Based upon the graph representations, two modules are designed to achieve fine-grained graph matching: Graph Fusion (GF) and Subgraph Search (SS). GF establishes latent semantic connections between visual and textual graphs. Then, SS searches the visual graph space to identify prompt-relevant subgraph regions. Finally, a T2I alignment score is obtained by calculating the similarity between the identified visual subgraphs and textual graphs. Extensive experiments and comparisons on three benchmark T2I alignment databases demonstrate the advantage of the proposed model. Codes are available at https://github.com/yzc-ippl/GraphAlign.
Reversible data hiding in shared images (RDHSI) has attracted increasing attention since it can improve the fault-tolerance and meanwhile provide conveniences for the shared image management. The existing RDHSI frameworks can be classified into internal embedding and external embedding. The former conducts image sharing with data embedding, which can provide the fault-tolerance for them. The latter only shares the original image and embeds the additional data into the shared images. However, there has not been a feasible method combining the advantages above two so far. To address this issue, we propose a novel RDHSI method, which is compatible with internal and external embedding, and possesses the advantages of those two frameworks. Thus, the proposed method has extensive applications. Specifically, a pixel is decomposed into two values whose 7 LSBs are both Hamming codes, yielding three crucial properties. The decomposed values are regarded as the coefficients to construct polynomials for (k, n)-threshold image sharing. Our data embedding has two phases, the internal one performs image sharing with data embedding to generate n marked shared images, and the external one further embeds the additional data into each marked shared image to generate a shared stego image using bit replacement. The receiver can extract the additional data from a shared stego image. When receiving any k shared stego images, all the candidates of replaced bits are traversed to reconstruct the polynomials. The original image as well as secret data can be revealed according to three properties. Theoretical analysis and experimental results demonstrate the proposed method outperforms some RDHSI methods.
Transformers have become essential for RGBT tracking because of their strong global modeling ability, but their attention patterns across layers and modalities are often highly similar, causing redundant computation and low tracking efficiency. To address this issue, we propose AALTrack, an efficient RGBT tracker based on Alternating Attention Learning, which exploits cross-layer and cross-modal attention correlations to reduce computation while preserving reliable attention prediction. Specifically, a lightweight attention prediction module uses historical attention priors to predict current-layer attention and replace redundant self-attention. To improve prediction reliability, these modules are inserted alternately into the RGB and TIR tracking backbones: when one modality predicts attention at a selected layer, the other retains full self-attention and provides a reliable reference for refinement through cross-modal attention propagation. We further introduce a dual knowledge distillation module, where predicted attention is supervised by teacher attention maps and feature-level distillation maintains stable feature representations. This design adds no inference overhead and improves prediction robustness. Extensive experiments on three RGBT tracking benchmarks show that AALTrack significantly accelerates inference while preserving tracking accuracy.
Transformer-Mamba hybrid models, which alternate between attention and state-space model (SSM) layers, have shown outstanding potential in image super-resolution (SR) by fully leveraging the strengths of both. However, existing hybrid models still suffer from the mismatch between Mamba’s inherent causality and the non-causal nature of 2D images. Besides, their heterogeneous operations and complex computational flows hinder efficient deployment on edge platforms. In this paper, we propose an algorithm-hardware co-optimized framework to accelerate lightweight hybrid SR models on edge devices. For algorithm design, we introduce MattenSR, a lightweight Transformer-Mamba hybrid SR model that alternately captures complementary local-global contextual information. As the key component of MattenSR, a novel Global-Aware State-Space Duality Module (GASM) is developed to eliminate Mamba’s causality without sequential scans, enhancing the global modeling capability for SR. For hardware optimization, we design a unified computing array and a reconfigurable base-2 approximation nonlinear unit to support diverse operations with high hardware utilization. To further reduce memory consumption and processing latency, a window-based streamlined (WinStream) dataflow is proposed, which unifies window partitioning and fine-grained pipeline scheduling for both attention and SSM. Finally, the overall hardware accelerator is developed and implemented on the Intel Stratix 10GX platform. Experimental results show that our MattenSR, with only 39K parameters, achieves better visual perception than other baselines. Moreover, compared to prior accelerators, our design achieves up to 27.3× and 7.7× improvements in throughput and energy efficiency, respectively.
Vision–language models such as CLIP provide semantically grounded priors for Unsupervised Domain Adaptation (UDA) in classification. However, existing CLIP-based UDA methods mainly adapt the model through the final-layer global visual embedding, leaving hierarchical representations in intermediate layers largely under-explored. Simply aggregating multi-level features is not a reliable solution for UDA, because shallow and mid-level representations are more sensitive to domain-specific appearance statistics and may introduce biased drift into both feature adaptation and pseudo-label learning. In this paper, we propose Multi-Level Representation Adaptation (MLRA), a CLIP-based UDA framework that exploits hierarchical visual representations while preserving the pretrained semantics of CLIP. Specifically, we introduce a Hierarchical Visual Aggregation (HVA) module to extract and adapt intermediate representations from multiple depths of the frozen CLIP visual encoder, and a Dual Anchoring Mechanism (DAM) to stabilize adaptation in both feature space and logit space. In the feature space, feature anchoring uses the original global CLIP feature as a semantic anchor to bound the HVA-induced residual update, thereby constraining representation drift while preserving CLIP’s pretrained semantic geometry. In the logit space, teacher logits computed from the original CLIP global feature regularize how the anchored visual feature is projected into the same fixed zero-shot class-text embedding space, thereby stabilizing pseudo-label learning and the resulting decision boundary. In this way, MLRA exploits transferable cross-layer cues while preserving CLIP’s pretrained semantic structure under domain shift. Experimental results on four UDA benchmarks show that MLRA achieves superior performance over existing state-of-the-art approaches.
Video Temporal Grounding is a fine-grained video understanding task that requires a nuanced alignment between linguistic semantics and dynamic visual content. However, existing approaches, ranging from global sentence-level to local phrase-level alignments, typically disregard the distinct semantic roles of word-level concepts. To address this, we propose the Hierarchical Concept Matching Network (HCMNet), a framework that explicitly disentangles and hierarchically integrates structured semantic concepts. Specifically, we design an Object Concept Matcher to establish entity-level correspondences using noun-based object cues. Subsequently, we design an Action Concept Matcher, which employs an action-aware cross-attention and a temporal self-attention to capture motion continuity and precise temporal transitions guided by action dynamics. By synergizing these specialized matchers, our framework achieves fine-grained semantic alignment and precise temporal localization, effectively bridging the gap between linguistic roles and visual evolution. Extensive experiments on QVHighlights, Charades-STA, and TACoS benchmarks demonstrate that HCMNet achieves state-of-the-art performance, confirming the superiority of our concept-guided paradigm.
Visible-infrared person re-identification (VI-ReID) aims to retrieve the same pedestrian across heterogeneous cameras, yet the substantial discrepancies between RGB and infrared images in illumination, texture, and spectral distributions severely limit the performance of existing methods. Although several recent studies have explored frequency-domain cues, how to jointly reduce input-level spectral discrepancy and feature-level modality bias remains insufficiently investigated. To address this issue, we propose a Frequency-Aware Cross-Modal Compensation Network, termed FAC-Net, for visible-infrared person reidentification. Specifically, we introduce a parameter-free Fourier style augmentation module at the input stage, which adaptively mixes low-frequency amplitude spectra to reduce cross-modal energy discrepancy at the source while preserving high-frequency information that maintains identity-related structures. During feature extraction, we design a dual-branch architecture consisting of a modality-invariant stream and a modality-compensation stream, and employ per-modality parameterization and an attention mechanism to finely correct statistical biases in the feature space. Furthermore, to mitigate gradient oscillations caused by cross-stream interactions, we adopt a task-specific safely-gated late fusion design with a zero-start mechanism, encouraging a gradual transition from independent learning to cross-stream collaboration. Extensive experiments on SYSU-MM01, RegDB, and LLCM show that the proposed method achieves competitive performance, with particularly strong Rank-1 accuracy across challenging cross-modal settings. The code will be made available at: https://github.com/ykding666/FACNet.
Scaling diffusion-based video generation to high spatial resolutions remains a significant computational bottleneck, as existing super-resolution methods either suffer from domain mismatches in pixel space or incur heavy sampling costs through iterative diffusion refinement. To address this, we introduce a lightweight latent upscaling framework that decouples resolution enhancement from stochastic generation, enabling scalable video synthesis with minimal overhead. Operating as a deterministic post-denoising transformation, the framework centers on a compact Causal Latent Upscaler (CLU) that incorporates a VAE-aware causal attention mechanism to preserve temporal coherence during spatial expansion. To bridge the gap between latent-space alignment and decoded visual fidelity, we further propose a progressive three-stage training strategy that transitions from efficient latent supervision to precise pixel-level refinement. Extensive experiments across real-world datasets and multiple text-to-video backbones demonstrate that our approach consistently enhances spatial resolution and perceptual quality while introducing negligible computational cost. By functioning as a plug-and-play module in latent space, the proposed framework offers a highly scalable and efficient pathway to high-definition video generation without modifying pretrained diffusion pipelines.
Hyperspectral image (HSI) and light detection and ranging (LiDAR) data provide complementary spectral and structural information, which are highly beneficial for land-cover classification. Despite recent progress, two fundamental challenges remain. First, most approaches mainly focus on spatial-domain feature modeling during single-modality enhancement, while the discriminative information embedded in the frequency domain is insufficiently explored. Second, cross-modal fusion lacks explicit amplitude–phase interaction in the frequency domain and does not leverage center semantic cues to guide the fusion process. To address these issues, this paper proposes a multi-scale Spatial–Frequency Cooperative Network (SFCM-Net) with Center-Modulated Fusion for Hyperspectral–LiDAR Classification. Specifically, a Spatial–Frequency Cooperative Enhancement (SFCE) block is designed to jointly model modality-specific features in both spatial and frequency domains, enabling more discriminative feature representations. Furthermore, a Center-Modulated Frequency Fusion (CMFF) block is developed to model amplitude–phase interactions and perform center-aware semantic modulation. The module enhances structural consistency in target regions, thereby improving classification performance. Experimental evaluations conducted on three public datasets verify the effectiveness of the proposed approach, which delivers improved performance compared with existing advanced methods. The code will be released at https://github.com/MSFLabX/SFCM-Net.
Over 29,568 children are reported missing in 2024 only in the United States, requiring reliable age-progressed imagery with natural expressions for effective search and recovery. However, existing diffusion models face fundamental limitations: expression-only methods cannot adjust age; age-only methods lack expression control. The core challenge is achieving simultaneous control over identity, age, and expression while preserving the pretrained backbone’s contextual generation capabilities. We propose DiffAgeX, a unified diffusion framework bridging backbone adaptation with stability for multi-attribute control. Our approach integrates two novel components: (1) Identity-CLIP Age-Aware (ICAA) module, featuring an offline variance-based layer selection strategy (ASEE) that identifies age-sensitive encoder layers and fuses identity, semantic, and age embeddings through learnable composition; (2) Adaptive Residual Fusion (ARF) mechanism, featuring bidirectional cross-attention that models identity-age-expression interactions, MLP-based perchannel scaling that fuses residuals based on mutual compatibility, and gradient-bounded damping that imposes a local bound on backbone and reduces backbone drift while preserving learned priors. Comprehensive evaluations demonstrate balanced performance across identity preservation, expression transfer, and age control. Across all reported evaluations, DiffAgeX achieves 84–88% expression accuracy, 0.25–0.31 identity distance, and 1.82–2.25 years target-age error. It enables coherent multi-attribute control with identity preservation, highlighting its potential for missing-person reconstruction, forensic analysis, and personalized avatar creation. For code, please refer to the GitHub repository: https://github.com/MSAfganUSTC/DiffAgeX.
Text-driven diffusion models have significantly advanced image editing capabilities by leveraging text prompts. However, existing methods generally rely on directly injecting source image attention features, which often overlook the entanglement of visual attributes during the diffusion sampling process. When applied to structural editing tasks beyond pixel-level color adjustments, they are particularly prone to producing artifacts or geometric distortions. To address these limitations, we propose a novel diffusion-based image editing method that introduces a modified diffusion sampling process to optimize and correct latent codes. Specifically, we disentangle semantic and structural representations from diffusion self-attention features and impose dual contrastive constraints. By iteratively optimizing the latent code at each denoising step, our method enables flexible and fine-grained control over the editing process. Furthermore, we integrate a diffusion latent correction module. This module leverages the semantic correspondence of diffusion features to match the feature consistency of latent space, thereby bridging the significant semantic gap caused by large-scale structural changes and achieving a robust balance between content generation and structure preservation. A comprehensive theoretical analysis, along with qualitative and quantitative experiments, demonstrates the superiority of our model in resolving the structural deformation issues while maintaining the fidelity to the source image across a wide range of editing tasks.
Long-range context modeling in computer vision has been transformed by state space models like Mamba, yet their performance is fundamentally bottlenecked by the primitive 2D-to-1D scanning process, which introduces structural discontinuities harmful to dense prediction. In this paper, we propose a vision Mamba-based image segmentation network with hierarchical scanning (HSM-UNet), and make three core contributions to solve this bottleneck problem. First, we propose a Hierarchical Scanning (HS) strategy, a low-redundancy paradigm that synergistically combines global and local spiral paths to generate efficient, anatomically-plausible sequences. Second, we design a Scan-Adaptive Fusion (SAF) module that leverages spatial-channel attention to resolve feature misalignment when integrating our HS decoder with a pre-trained cross-scanning encoder. Extensive experiments across 13 datasets—spanning medical imaging, remote sensing, industrial defect, and agricultural plant disease—validate the effectiveness of our method.
Synthetic Aperture Radar (SAR) has been extensively applied in land-cover classification owing to its all-weather observation capabilities. To improve SAR image classification performance, some methods have attempted to introduce the privileged modality during training to provide complementary information, thus enhancing the discriminative ability of SAR-based classifiers. Among them, the generation-based methods aims to recover the missing privileged modality information during test, thus obtaining more comprehensive multi-modal features for classification. Although these methods have achieved certain success, they have generally struggled to model the complex structures of the privileged modality and ignored the uncertainty of the generated privileged modality as well as the SAR modality, thus limiting the robustness of classification. To this end, a privileged modality generation and trusted feature enhancement network is proposed for SAR image classification. Specifically, benefiting from the generation capabilities of diffusion models, a conditional diffused privileged modality generation module is presented, which incorporates the frequency information into the diffusion model to generate structure-preserved privileged modality. Besides, to capture reliable multi-modal features, an uncertainty-guided feature enhancement module is presented by dynamically weighting the features of different modalities through modality uncertainty learning. Experimental results on two benchmark datasets show that the proposed method outperforms existing state-of-the-art method.
Domain Adaptive Object Detection (DAOD) aims to solve the catastrophic performance degradation of deep learning models caused by domain shift. Most existing methods rely on complex and multi-stage teacher-student frameworks. They need to train multiple detection models and optimize pseudo labels in different stages, thus resulting in inefficiency and error accumulation. To overcome these fundamental limitations, we propose LPR-DA, a novel efficient single-stage framework which is inspired by human cognitive cycle (i.e., Learn, Practice, and Review) that enables the self-evolution of a single detection model. Specifically, in the Learn step, we first use a VLM to generate pseudo labels for the target domain and propose an Uncertainty-Aware Gating mechanism to effectively filter high-quality pseudo labels which are used to train a detector. In the Practice step, the detector performs inference on the target domain. Finally, in the Review step, consistency matching is designed to correct and update the pseudo labels based on external knowledge for the next Learn iteration. This closed-loop process enables an efficient single-stage paradigm by training only a single detector for DAOD. Experiments show that LPR-DA achieves new state-of-the-art performance on multiple popular DAOD benchmarks, while also demonstrating superior efficiency with an order-of-magnitude reduction in training time, real-time inference speeds, and a significantly smaller model size.
RGB-T semantic segmentation aims to achieve robust pixel-level scene understanding by integrating complementary cues from visible and thermal infrared imagery. Existing RGB-T methods mainly emphasize modality complementarity but insufficiently model spectral variations. This may limit the adaptability of cross-modal fusion and lead to degraded segmentation performance across diverse weather conditions. We refer to this limitation as the weather-insensitive representation issue. Moreover, most existing methods process features from different weather conditions in the same inference manner, without explicitly adapting feature processing according to weather characteristics, which leads to the weather-agnostic inference issue. To address these challenges, we propose a spectral-adaptive fusion and MLLM-guided weather-aware routing framework for RGB-T semantic segmentation. To tackle the weather-insensitive representation issue, we design a dynamic spectral-adaptive fusion module (DSFM) to adaptively adjust frequency-band responses during cross-modal fusion, thereby better distinguishing among different weather conditions. Furthermore, to address the weather-agnostic inference issue, we propose an MLLM-guided weather-aware expert routing strategy, which distills quantified weather semantic labels into the routing gate and supports adaptive expert selection under different weather conditions. Experimental results on RGB-T benchmarks demonstrate that the proposed method achieves competitive performance under diverse adverse weather conditions. Code will be released.
