Multi-label zero-shot learning (MLZSL) aims to recognize multiple unseen class labels that may appear in an image, posing a significant challenge in the field of computer vision. While generative methods have achieved remarkable progress by synthesizing visual features of unseen categories, they often suffer from poor visual-semantic consistency and limited generative quality. To address these issues, this paper proposes a novel “Align-then-Generate” paradigm and introduces a unified framework named VLA-CMG, which integrates vision-language alignment with cross-modal feature generation. Specifically, a Language-aware multi-label image encoder (LMIE) is designed to extract both global and local visual features from images, which are aligned with multi-label semantic embeddings generated by the text encoder of the Vision-language pre-training (VLP) Model, thereby enhancing the consistency between semantic and visual representations. This alignment provides high-quality input for the training of the Dual-stream feature generation network (DSFGN), which synthesizes discriminative visual features for unseen classes. Finally, a robust multi-label zero-shot classifier is built upon the generated features. Extensive experiments on two large-scale benchmark datasets (i.e., NUS-WIDE and Open Images) demonstrate that VLA-CMG consistently outperforms existing state-of-the-art methods on both ZSL and GZSL tasks, validating its effectiveness and superiority.

Image style transfer is an important area of research, but existing methods often struggle to maintain content fidelity and produce diverse results. We propose region-aware diverse stylization (RDS) to address these limitations. Our method introduces two key components: an object-background augmented attention unit to improve structural detail, and an efficient pattern aggregation attention unit to capture dominant style features. We also design a color histogram-based contrastive loss to better align color distribution. Furthermore, we present the region-aware diverse stylization unit (RDSU), which generates multiple distinct stylized images from a single-style image without additional training. This enhances the method’s versatility and robustness. We also created a new high-quality dataset of 1000 images to support fine-grained structural learning. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art approaches in both fidelity and diversity.

Generalized Zero-Shot Learning (GZSL) relies on visual-semantic mappings to recognize unseen categories by transferring knowledge from seen classes. However, the domain transfer process often introduces bias, which significantly hinders the model’s generalization ability on unseen classes. Existing GZSL methods struggle to balance debiasing with model performance, frequently adopting simplistic strategies that limit transfer effectiveness. To address this issue, we propose an Attribute Affinity Coordinated Debiasing (AACD) framework. AACD models the correlations among category-level attribute descriptions to capture latent inter-class affinity structures, thereby identifying and correcting incorrect knowledge transfer. Specifically, AACD enhances visual-semantic interactions via attribute-informed domain adaptation, incorporates an Affinity Discriminant Module (ADM) to guide the embedding space through similarity-based affinity discrimination, and introduces an Affinity Constraint Module (ACM) to reinforce intra-class consistency while maximizing inter-class separability. These modules are jointly integrated into the encoder to reduce the over-reliance on seen categories and promote robust domain transfer. Extensive experiments on three standard benchmarks CUB, SUN, and AwA2 demonstrate that our AACD framework achieves substantial improvements. Codes are available at https://github.com/QC-Elord/AACD.

Pseudo-labels play a crucial role in weakly supervised medical image segmentation. However, their quality is often compromised by uncertainty and noise, which can significantly degrade segmentation performance. To address this challenge, we propose PLNet, a novel framework for weakly supervised medical image segmentation that enhances pseudo-label quality through two key strategies. First, an entropy-based weighted averaging mechanism emphasizes low-uncertainty predictions, improving structural consistency. Second, a sensitivity-specificity enhancement module balances attention between foreground and background, reducing false positives and negatives. PLNet integrates these components with a hybrid CNN-Transformer backbone and a hybrid loss function combining scribble supervision, co-teaching, and ensemble strategies. Extensive experiments on public datasets show that PLNet significantly outperforms state-of-the-art methods, especially in complex or sparsely annotated scenarios.

At the core of Deep Learning-based Deformable Medical Image Registration (DMIR) lies a strong foundation. Essentially, this network compares features in two images to identify their mutual correspondence, which is necessary for precise image registration. In this paper, we use three novel techniques to increase the registration process and enhance the alignment accuracy between medical images. First, we propose cross attention over multi-layers of pairs of images, allowing us to take out the correspondences between them at different levels and improve registration accuracy. Second, we introduce a skip connection with residual blocks between the encoder and decoder, helping information flow and enhancing overall performance. Third, we propose the utilization of cascade attention with residual block skip connections, which enhances information flow and empowers feature representation. Experimental results on the OASIS data set and the LPBA40 data set show the effectiveness and superiority of our proposed mechanism. These novelties contribute to the enhancement of 3D DMIR-based on unsupervised learning with potential implications in clinical practice and research.

Low-light image enhancement aims to improve brightness, suppress noise, and recover accurate color and structure, requiring precise illumination modeling and reliable reflectance recovery. However, most Retinex-based methods adopt explicit, multi-stage pipelines prone to decomposition bias, error accumulation, and chromatic entanglement between illumination and reflectance. To tackle these issues, we propose IDAR (Implicit Decomposition, illumination Adjustment, and reflectance Restoration), a unified Retinex-inspired framework with two key innovations. First, we design an implicit decomposition strategy based on dual-branch feature learning: a low-frequency-constrained illumination branch models lighting with chromaticity awareness, while a contrast-guided reflection branch preserves details by decoupling reflectance from illumination. This implicit design avoids intermediate supervision and reduces decomposition bias. Second, we introduce the Illumination Chromaticity Expansion Module (ICEM), which employs text-guided chromaticity learning to enhance chromaticity perception. By learning a reflectance-independent spectral representation, ICEM reduces color shifts and improves fidelity under complex lighting. Experiments on multiple benchmarks validate the superior visual quality, quantitative performance, and physical interpretability of IDAR.

The limited availability of real defect samples and the high cost associated with manual defect generation present significant challenges in automated defect detection systems. In this paper, we propose a novel few-shot defect generation approach based on StyleGAN2-ADA, designed to produce high-quality defect samples for improving defect and anomaly detection performance. The proposed methodology comprises two stages: first, two independent few-shot generation models learn the distributions of defect-rich and defect-free datasets, respectively. Next, an indirect decoupling strategy is employed to incorporate prior knowledge of defect distributions, while still facilitating the unconditional generation of defect samples. To further enhance the generation process, we introduce a defect mask attention mechanism that specifically targets defect regions and the objects of interest, effectively preventing the generation of defect-free images and reducing background degradation. This method not only enhances the precision of defect generation but also allows for the flexible and realistic simulation of random defect variations. Extensive experiments on the MVTec AD and VisA dataset demonstrate the superiority of our approach, outperforming existing GAN-based defect generation methods in few-shot scenarios. Specifically, the proposed method excels across 35 different defects and 80 evaluation metrics, with 55 metrics surpassing all previous methods and 28 ranking second. Our approach achieves an average improvement of 23% on the KID metric.

Semi-supervised learning has proven effective in the challenging field of 3D medical image segmentation. However, existing methods typically focus solely on utilizing 3D features, which limits their ability to capture local differences effectively. To address this limitation, we propose a Dual Stream Fusion Model (DSFM) that fuses both 3D and 2D features, enabling the model to capture both global and local information. The model consists of a Cross-Dimensional Attention Fusion Module (CDAFM) and a Self-Adaptive Fusion Module (SAFM). The CDAFM integrates 3D and 2D features using an attention mechanism, while the SAFM merges the segmentation results from both modalities through adaptive weighting. Together, these modules enhance the model’s ability to focus on local differences while simultaneously learning global features. Specifically, the 3D decoder receives fused features from both the 3D and 2D encoders, and the final segmentation result is produced by combining the outputs of both the 3D and 2D decoders. Furthermore, we introduce an alignment loss to enforce consistency between the 3D and 2D segmentation sub-networks through regularization. Comprehensive experiments on four publicly available 3D medical datasets demonstrate the effectiveness of our approach. Notably, on the BRaTS19 dataset, which contains only 20

Segmentation and classification of breast tumors are critical for computer-aided diagnosis (CAD) using breast ultrasound (BUS), a non-invasive modality for early cancer detection. However, BUS images present challenges such as low contrast, speckle noise, irregular morphology, and indistinct boundaries, which hinder automated analysis. Existing multi-task learning frameworks that utilize shared encoders often suffer from inadequate task-specific features and interference between objectives, resulting in suboptimal performance. To address this issue, we present MDFANet, a novel multi-task learning network designed to optimize tumor segmentation and classification in BUS images. The segmentation network incorporates a Feature Enhancement Aggregation (FEA) module, which enriches semantic representations by fusing multi-scale features from both encoder and decoder stages. Moreover, we introduce a distance auxiliary branch to facilitate accurate tumor boundary localization by integrating hierarchical features with spatial contextual cues. To balance the optimization of segmentation and classification objectives, we further employ a sample-adaptive joint loss function that dynamically adjusts task contributions based on the consistency of cross-task predictions. Experimental results on a BUS dataset demonstrate that our method significantly outperforms existing state-of-the-art approaches in both segmentation accuracy and classification reliability.

Guided depth super-resolution (GDSR) aims to reconstruct high-resolution (HR) depth maps from low-resolution (LR) counterparts with the aid of aligned HR RGB images. However, existing methods exhibit limited capability in learning and representing prior knowledge and high-frequency components, often resulting in degraded structural accuracy and detail fidelity. Moreover, most current approaches lack effective integration of prior information and struggle to recover fine-grained details. To address these limitations, we propose a novel multi-prior guided depth super-resolution framework based on diffusion model. Specifically, a multi-prior guided information extraction block is designed to extract color and edge priors, providing complementary high-frequency guidance. We further introduce a multi-headed channel-wise self-attention (MCSA) module and a feature optimized selection module (FOSM) to enhance feature extraction and preserve critical information. Besides, reconstruction module based on diffusion model is employed to denoise and generate high-quality depth maps, ensuring spatial consistency and edge sharpness. Extensive experiments demonstrate that our proposed method outperforms existing state-of-the-art techniques in both accuracy and visual quality.

Low-light image enhancement (LLIE) aims to restore low-light images to their normal-light counterparts with optimal global illumination distribution and clear local details. With the advancement of deep learning, deep learning-based methods have become the mainstream in the LLIE community. However, most deep learning-based method cannot yet fully exploit the global and local contextual information in the low-light image. In this paper, we introduce a dual-branch module to simultaneously restore global and local features from spatial and frequency domain. To fuse these multi-level features, we propose a perception module to perform feature interaction between global and local features via cross attention and self-gating. By integrating the two developed modules into a U-Net backbone, we present a global-local interaction network for LLIE. Furthermore, recent studies have shown that contrastive learning can be an effective paradigm for the LLIE task. However, previous works typically use semantically-inconsistent under-/over-exposed images as negative samples. These images are very dissimilar to the ground-truth and cannot provide sufficient regularization in contrastive learning. To address this limitation, we explore a practical multi-exposure progressive contrastive regularization framework for LLIE. With a customized sample generation, sample selection, and progressive learning strategy, our proposed framework progressively narrows down the solution space around the optimum, and helps to improve the performance of LLIE methods without additional inference overhead. Combining the proposed network and contrastive regularization, our proposed method achieves favorable results compared to state-of-the-art LLIE methods on benchmark datasets. Extensive experiments further demonstrate the generalization ability of our proposed method.

Open-vocabulary multi-label classification (OV-MLC) aims to leverage the rich multi-modal knowledge from Vision-language pre-training (VLP) models to further improve the recognition ability for unseen (novel) classes beyond the training set in multi-label scenarios. Existing OV-MLC methods only perform predictions on single hierarchical regions, and aggregate the prediction scores of these regions through simple top-k mean pooling. This fails to unleash the potential of rich hierarchical region clues in multi-label images and does not fully exploit the discriminative information from all regions in the image, resulting in sub-optimal performance. In this work, we propose a novel OV-MLC framework to fully harness the power of multiple hierarchical region clues. Specifically, we first design a hierarchical clue gathering (HCG) module to gather different hierarchical clues, enabling more precise recognition of multiple object categories with different sizes in a multi-label image. Then, by viewing multi-label classification as single-label classification of each region within the image, we present a novel hierarchical score aggregation (HSA) approach, thereby better utilizing the predictions of each image region for each class. We also utilize a well-designed region selection strategy (RSS) to eliminate noise or background regions in an image that are irrelevant to classification, achieving higher multi-label classification accuracy. In addition, we propose a hybrid prompt learning (HPL) strategy to enhance visual-semantic consistency while preserving the generalization capability of label embeddings for unseen classes. Extensive experiments on public benchmark datasets demonstrate that our method significantly outperforms the current state-of-the-art.

Skeleton-based action recognition is one of the basic researches in computer vision. In recent years, the unsupervised contrastive learning paradigm has achieved great success in skeleton-based action recognition. However, previous work often treated input skeleton sequences as a whole when performing comparisons, lacking fine-grained representation contrast learning. Therefore, we propose a contrastive learning method for Representation Modeling with Multi-domain Decoupling (RMMD), which extracts the most significant representations from input skeleton sequences in the temporal domain, spatial domain and frequency domain, respectively. Specifically, in the temporal and spatial domains, we propose a multi-level spatiotemporal mining reconstruction module (STMR) that iteratively reconstructs the original input skeleton sequences to highlight spatiotemporal representations under different actions. At the same time, we introduce position encoding and a global adaptive attention matrix, balancing both global and local information, and effectively modeling the spatiotemporal dependencies between joints. In the frequency domain, we use the discrete cosine transform (DCT) to achieve temporal-frequency conversion, discard part of the interference information, and use the frequency self-attention (FSA) and multi-level aggregation perceptron (MLAP) to deeply explore the frequency domain representation. The fusion of the temporal domain, spatial domain and frequency domain representations makes our model more discriminative in representing different actions. Besides, we verify the effectiveness of the model on the NTU RGB+D and PKU-MMD datasets. Extensive experiments show that our method outperforms existing unsupervised methods and achieves significant performance improvements in downstream tasks such as action recognition and action retrieval.

Image captioning has become one of the most popular research problems in the field of artificial intelligence. Although many studies have achieved excellent results, there still are some challenges, for example, cross-modal feature alignment lacks explicit guidance, and model-generated sentences contain grammatical errors. In this paper, we propose a relationship-aligned and grammar-wise BERT model, which integrates a relationship exploration module and a grammar enhancement module into the BERT-based model. Specifically, in the relationship exploration module, to explore relationship tags as anchors to guide semantic alignment, we design a network to calculate the cosine similarity between visual features and word vector information. We construct the grammar enhancement module similarly to the BERT. That means we use two BERT modules in our framework. The first is the main frame for generating captions, and the second is the auxiliary model to determine whether the syntax of the generated caption is correct. To validate the performance of our proposed model, we conduct abundant experiments on the MSCOCO dataset, Flickr30k dataset, and Flickr8k dataset. Experimental results show that our proposed method performs better than state-of-the-art approaches.

Recent advances in image deraining have focused on training powerful models on mixed multiple datasets comprising diverse rain types and backgrounds. However, this approach tends to overlook the inherent differences among rainy images, leading to suboptimal results. To overcome this limitation, we focus on addressing various rainy images by delving into meaningful representations that encapsulate both the rain and background components. Leveraging these representations as instructive guidance, we put forth a Context-based Instance-level Modulation (CoI-M) mechanism adept at efficiently modulating CNN- or Transformer-based models. Furthermore, we devise a rain-/detail-aware contrastive learning strategy to help extract joint rain-/detail-aware representations. By integrating CoI-M with the rain-/detail-aware Contrastive learning, we develop CoIC, an innovative and potent algorithm tailored for training models on mixed datasets. Moreover, CoIC offers insight into modeling relationships of datasets, quantitatively assessing the impact of rain and details on restoration, and unveiling distinct behaviors of models given diverse inputs. Extensive experiments validate the efficacy of CoIC in boosting the deraining ability of CNN and Transformer models. CoIC also enhances the deraining prowess remarkably when real-world dataset is included.

Low-rank matrix decomposition with first-order total variation (TV) regularization exhibits excellent performance in exploration of image structure. Taking advantage of its excellent performance in image denoising, we apply it to improve the robustness of deep neural networks. However, although TV regularization can improve the robustness of the model, it reduces the accuracy of normal samples due to its over-smoothing. In our work, we develop a new low-rank matrix recovery model, called LRTGV, which incorporates total generalized variation (TGV) regularization into the reweighted low-rank matrix recovery model. In the proposed model, TGV is used to better reconstruct texture information without over-smoothing. The reweighted nuclear norm and L1-norm can enhance the global structure information. Thus, the proposed LRTGV can destroy the structure of adversarial noise while re-enhancing the global structure and local texture of the image. To solve the challenging optimal model issue, we propose an algorithm based on the alternating direction method of multipliers. Experimental results show that the proposed algorithm has a certain defense capability against black-box attacks, and outperforms state-of-the-art low-rank matrix recovery methods in image restoration.

We address image deraining under complex backgrounds, diverse rain scenarios, and varying illumination conditions, representing a highly practical and challenging problem. Our approach utilizes synthetic, real-world, and nighttime datasets, wherein rich backgrounds, multiple degradation types, and diverse illumination conditions coexist. The primary challenge in training models on these datasets arises from the discrepancies among them, potentially leading to conflicts or competition during the training period. To address this issue, we first align the distribution of synthetic, real-world and nighttime datasets. Then we propose a novel contrastive learning strategy to extract multi-view (multiple) representations that effectively capture image details, degradations, and illuminations, thereby facilitating training across all datasets. Regarding multiple representations as profitable prompts for deraining, we devise a prompting strategy to integrate them into the decoding process. This contributes to a potent deraining model, dubbed Rainmer. Additionally, a spatial-channel interaction module is introduced to fully exploit cues when extracting multi-view representations. Extensive experiments on synthetic, real-world, and nighttime datasets demonstrate that Rainmer outperforms current representative methods. Moreover, Rainmer achieves superior performance on the All-in-One image restoration dataset, underscoring its effectiveness. Furthermore, quantitative results reveal that Rainmer significantly improves object detection performance on both daytime and nighttime rainy datasets. These observations substantiate the potential of Rainmer for practical applications.

Recent advances in image de-raining have focused on training powerful models on mixed multiple datasets comprising diverse rain types and backgrounds. However, this approach tends to overlook the inherent differences between datasets, resulting in suboptimal optimization and poor generalization. To address this limitation, we propose an approach to learn instance-specific de-raining models by exploring meaningful representations that characterize both the rain and background components in rainy images. Leveraging these representations as instructive guidance, we put forth a Context-based Instance-specific Modulation (CoI-M) mechanism which can modulate CNN- or Transformer-based models. Furthermore, we develop a rain-/detail-aware contrastive learning strategy to help extract joint rain-/detail-aware instance-specific representations. By integrating CoI-M with the rain-/detail-aware Contrastive learning, we develop CoIC, an innovative and effective algorithm for training models on mixed datasets. Moreover, CoIC offers insight into modeling relationships of datasets, quantitatively assessing the impact of rain and details on restoration, and revealing different behaviors of models given diverse inputs. Extensive experiments validate the effectiveness of CoIC in boosting the de-raining ability of CNN- and Transformer-based models, as well as significantly improving their generalization ability.

Multi-label zero-shot learning (MLZSL) is a more realistic and challenging task than single-label zero-shot learning (SLZSL), which aims to recognize multiple unseen classes in a single image. To adapt generative models to the MLZSL task and better recognize multiple unseen object categories in an image, this paper proposes a Transferable Generative Framework (TGF), which consists of a Multi-Label Semantic Embedding Autoencoders (SEAs), a Semantic-Related Multi-Label Feature Transformation Network (FTN) and a Multi-Label Feature Generation Networks (FGNs). First, SEAs adaptively encodes the class-level word vectors corresponding to each sample containing different number of classes into sample-level semantic embeddings with the same dimension. Then, FTN transforms global features extracted by a CNN pre-trained on single-label images into features that are semantic-related and more suitable for multi-label classification. Finally, FGNs generates both global and local features to better recognize the dominant and minor object categories in a multi-label image, respectively. Extensive experiments on three benchmark datasets show that TGF significantly outperforms state-of-the-arts. Specifically, compared with the previous best generative MLZSL method ( i.e ., Gen-MLZSL), TGF improves the mAP of the ZSL (GZSL) task by 5.4% (6.9%), 20.5% (27.9%), and 2.4% (3.9%) on NUS-WIDE, Open Images, and MS-COCO datasets, respectively.