Underwater image enhancement (UIE) face significant challenges due to complex underwater lighting conditions. Recently, mamba-based methods have achieved promising results in image enhancement tasks. However, these methods commonly rely on Vmamba, which focuses only on spatial information modeling and struggles to deal with the cross-color channel dependency problem in underwater images caused by the differential attenuation of light wavelengths, limiting the effective use of deep networks. In this paper, we propose a novel UIE framework called O-Mamba. O-Mamba employs an dual-branch network to separately model spatial and cross-channel information, utilizing the efficient global receptive field of state-space models optimized for underwater images. To enhance information interaction between the two branches and effectively utilize multi-scale information, we design a Multi-scale Bi-mutual Promotion Module. This branch includes MS-MoE for fusing multi-scale information within branches, Mutual Promotion module for interaction between spatial and channel information across branches, and Cyclic Multi-scale optimization strategy to maximize the use of multi-scale information. Extensive experiments demonstrate that our method achieves state-of-the-art (SOTA) results. The code is available at https://github.com/chenydong/O-Mamba .

Underwater visuals undergo various complex degradations, inevitably influencing the efficiency of underwater vision tasks. Recently, diffusion models were employed for underwater image enhancement (UIE) tasks, and gained the best performance. However, these methods fail to consider the physical properties and underwater imaging mechanisms in the diffusion process, limiting the information completion capacity of diffusion models. In this article, we introduce a novel UIE framework, named PA-Diff, designed to exploit physical knowledge to guide the diffusion process. PA-Diff consists of the physics prior generation (PPG) branch, the implicit neural reconstruction (INR) branch, and the physics-aware diffusion transformer (PDT) branch. Our designed PPG branch aims to produce prior knowledge of physics. By utilizing the physics prior knowledge to guide the diffusion process, the PDT branch can obtain underwater-aware ability and model the complex distribution in real-world underwater scenes. The INR branch can learn robust feature representations from diverse underwater images via implicit neural representation, which reduces the difficulty of restoration for the PDT branch. Extensive experiments demonstrate that our method achieves the best performance on UIE tasks. The code is available at https://github.com/chenydong/PA-Diff

Existing unsupervised methods have addressed the challenges of inconsistent paired data and tedious acquisition of ground-truth labels in shadow removal tasks. However, GAN-based training often faces issues such as mode collapse and unstable optimization. Furthermore, due to the complex mapping between shadow and shadow-free domains, merely relying on adversarial learning is not enough to capture the underlying relationship between two domains, resulting in low quality of the generated images. To address these problems, we propose a semantic-guided coarse-to-fine diffusion model for self-supervised shadow removal, which consists of two stages. In the first stage, a semantic-guided generative adversarial network (SG-GAN) is proposed to carry out a coarse result and construct paired synthetic data through a cycle-consistent structure. Then the coarse result is refined with a diffusion-based restoration module (DBRM) to enhance the texture details and edge artifact at second stage. Meanwhile, we propose a multi-modal semantic prompter (MSP) that aids in extracting accurate semantic information from real images and text, guiding the shadow removal network to restore images better in SG-GAN. We conduct experiments on multiple public datasets and the experimental results demonstrate the effectiveness of our method.

Recently, unsupervised image-to-image (i2i) translation methods based on contrastive learning have achieved state-of-the-art results. However, in previous works, the negatives are sampled from the input image itself, which inspires us to design a data augmentation method to improve the quality of the selected negatives. Moreover, the previous methods only preserve the content consistency via patch-wise contrastive learning, which ignores the domain consistency between the generated images and the real images of the target domain. This paper proposes a novel unsupervised i2i translation framework based on multi-cropping contrastive learning and domain consistency, called MCDUT. Specifically, the multi-cropping views are obtained with the aim of further generating high-quality negative examples. To constrain the embeddings in the deep feature space, a new domain consistency loss is formulated, which encourages the generated images to be close to the real images. In many i2i translation tasks, this method achieves state-of-the-art results, and the advantages of this method have been proven through extensive comparison experiments and ablation research. The code of MCDUT is available at .

Existing image-to-image (I2I) translation methods achieve state-of-the-art performance by incorporating the patch-wise contrastive learning into generative adversarial networks. However, patch-wise contrastive learning only focuses on the local content similarity but neglects the global structure constraint, which affects the quality of the generated images. In this paper, we propose a new unpaired I2I translation framework based on dual contrastive regularization and spectral normalization, namely SN-DCR. To maintain consistency of the global structure and texture, we design the dual contrastive regularization using different deep feature spaces respectively. In order to improve the global structure information of the generated images, we formulate a semantic contrastive loss to make the global semantic structure of the generated images similar to the real images from the target domain in the semantic feature space. We use gram matrices to extract the style of texture from images. Similarly, we design a style contrastive loss to improve the global texture information of the generated images. Moreover, to enhance the stability of the model, we employ the spectral normalized convolutional network in the design of our generator. We conduct comprehensive experiments to evaluate the effectiveness of SN-DCR, and the results prove that our method achieves SOTA in multiple tasks. The code and pretrained models are available at https://github.com/zhihefang/SN-DCR .

To tackle the difficulties in fitting paired real-world data for single image deraining (SID), recent unsupervised methods have achieved notable success. However, these methods often struggle to generate high-quality, rain-free images due to a lack of attention to semantic representation and image content, resulting in ineffective separation of content from the rain layer. In this paper, we propose a novel cycle contrastive generative adversarial network for unsupervised SID, called CCLGAN. This framework combines cycle contrastive learning (CCL) and location contrastive learning (LCL). CCL improves image reconstruction and rain-layer removal by bringing similar features closer and pushing dissimilar features apart in both semantic and discriminative spaces. At the same time, LCL preserves content information by constraining mutual information at the same location across different exemplars. CCLGAN shows superior performance, as extensive experiments demonstrate the benefits of CCLGAN and the effectiveness of its components.

Underwater images suffer from complex and diverse degradation, which inevitably affects the performance of underwater visual tasks. However, most existing learning-based Underwater image enhancement (UIE) methods mainly restore such degradations in the spatial domain, and rarely pay attention to the fourier frequency information. In this paper, we develop a novel UIE framework based on spatial-frequency interaction and gradient maps, namely SFGNet, which consists of two stages. Specifically, in the first stage, we propose a dense spatial-frequency fusion network (DSFFNet), mainly including our designed dense fourier fusion block and dense spatial fusion block, achieving sufficient spatial-frequency interaction by cross connections between these two blocks. In the second stage, we propose a gradient-aware corrector (GAC) to further enhance perceptual details and geometric structures of images by gradient map. Experimental results on two real-world underwater image datasets show that our approach can successfully enhance underwater images, and achieves competitive performance in visual quality improvement. The code is available at https://github.com/zhihefang/SFGNet.

Underwater images are subject to intricate and diverse degradation, inevitably affecting the effectiveness of underwater visual tasks. However, most approaches primarily operate in the raw pixel space of images, which limits the exploration of the frequency characteristics of underwater images, leading to an inadequate utilization of deep models' representational capabilities in producing high-quality images. In this paper, we introduce a novel Underwater Image Enhancement (UIE) framework, named WF-Diff, designed to fully leverage the characteristics of frequency domain information and diffusion models. WF-Diff consists of two detachable networks: Wavelet-based Fourier information interaction network (WFI2-net) and Frequency Residual Diffusion Adjustment Module (FR-DAM). With our full exploration of the frequency domain information, WFI2-net aims to achieve preliminary enhancement of frequency information in the wavelet space. Our proposed FRDAM can further refine the high- and low-frequency information of the initial enhanced images, which can be viewed as a plug-and-play universal module to adjust the detail of the underwater images. With the above techniques, our algorithm can show SOTA performance on real-world underwater image datasets, and achieves competitive performance in visual quality. The code is available at https://github.com/zhihefang/WF-Diff.

Existing unsupervised methods have addressed the challenges of inconsistent paired data and tedious acquisition of ground-truth labels in shadow removal tasks. However, GAN-based training often faces issues such as mode collapse and unstable optimization. Furthermore, due to the complex mapping between shadow and shadow-free domains, merely relying on adversarial learning is not enough to capture the underlying relationship between two domains, resulting in low quality of the generated images. To address these problems, we propose a semantic-guided adversarial diffusion framework for self-supervised shadow removal, which consists of two stages. At first stage a semantic-guided generative adversarial network (SG-GAN) is proposed to carry out a coarse result and construct paired synthetic data through a cycle-consistent structure. Then the coarse result is refined with a diffusion-based restoration module (DBRM) to enhance the texture details and edge artifact at second stage. Meanwhile, we propose a multi-modal semantic prompter (MSP) that aids in extracting accurate semantic information from real images and text, guiding the shadow removal network to restore images better in SG-GAN. We conduct experiments on multiple public datasets, and the experimental results demonstrate the effectiveness of our method.

In overcoming the challenges faced in adapting to paired real-world data, recent unsupervised single image deraining (SID) methods have proven capable of accomplishing notably acceptable deraining performance. However, the previous methods usually fail to produce a high quality rain-free image due to neglecting sufficient attention to semantic representation and the image content, which results in the inability to completely separate the content from the rain layer. In this paper, we develop a novel cycle contrastive adversarial framework for unsupervised SID, which mainly consists of cycle contrastive learning (CCL) and location contrastive learning (LCL). Specifically, CCL achieves high-quality image reconstruction and rain-layer stripping by pulling similar features together while pushing dissimilar features further in both semantic and discriminant latent spaces. Meanwhile, LCL implicitly constrains the mutual information of the same location of different exemplars to maintain the content information. In addition, recently inspired by the powerful Segment Anything Model (SAM) that can effectively extract widely applicable semantic structural details, we formulate a structural-consistency regularization to fine-tune our network using SAM. Apart from this, we attempt to introduce vision transformer (VIT) into our network architecture to further improve the performance. In our designed transformer-based GAN, to obtain a stronger representation, we propose a multi-layer channel compression attention module (MCCAM) to extract a richer feature. Equipped with the above techniques, our proposed unsupervised SID algorithm, called CCLformer, can show advantageous image deraining performance. Extensive experiments demonstrate both the superiority of our method and the effectiveness of each module in CCLformer. The code is available at https://github.com/zhihefang/CCLGAN.

Image deblurring presents a complex challenge intending to renew visual clarity in images affected by camera shake or object motion. However, traditional deblurring methodologies tend to emphasize local features, ignoring critical contextual information, which consequently limits their efficacy in addressing common blurry image issues. This paper proposes Efficient Cross Transformer (ECT) to overcome the limitations of inadequate global features in image restoration across different scales. ECT designs the cross-attention layers to achieve interaction between input tokens and network tokens, aiming to efficiently capture image details in the input space and feature information in the latent space of each layer. The cross-attention mechanism works in latent layers, integrating image features from different scales while reducing the computational burden compared to traditional Transformers. Furthermore, ECT employs windowing techniques in a strategic manner to capture local hazy components, duly amplifying its image restoration abilities. Empirical evidence shows that ECT has achieved state-of-the-art results in deblurring images, demonstrating excellent performance without requiring a large training dataset. Consequently, ECT emerges as a promising solution for rectifying blurred images across artificial and real-world environments.

Partially supervised instance segmentation aims to segment objects on both limited seen categories and novel unseen categories (without annotated masks), thereby eliminating expensive demands of mask annotation for new categories. Existing work mainly utilize the pipeline model of detection first and then segmentation, and explores how to provide more discriminative regions of interest for the class-agnostic mask head, but these methods do not perform well when faced with complex scenes. In this work, we propose a novel method, named CCMask, that combines Context Feature Pyramid Network (Context-FPN) and Memory Contrastive Learning Head (MCL Head) to achieve effective class-agnostic mask segmentation. Specifically, we introduce a Context-FPN to obtain context-rich feature map via context extraction module, which will benefit the subsequent task heads. In the MCL Head, we employ foreground/background query memory queue to store queries from recent training batches, this helps the MCL Head learns the general concepts of foreground and background. These strategies collectively contribute to improve the discrimination between foreground and background. Exhaustive experiments on COCO dataset demonstrate that our method achieves state-of-the-art results.

To improve the semantic diversity and visual authenticity of image translation, in this paper we propose Generative Adversarial Networks based on deep feature difference and attention mechanism. In our model, we employ a pre-trained image classification network to extract different level features of the generated images and then calculate the perceptual similarity based on these features. Such perceptual similarities are introduced into the adversarial learning to control the differences in high-level semantic features. Furthermore, in order to ensure the quality of the translated image, the residual network structure with the attention mechanism is utilized in our model. According to the importance of each feature channel, the different attention is strengthened on the useful features. In addition, we not only employ the discriminator on the generated images to discriminate whether the images are real or fake, but also make a discrimination on the reconstructed images. The main contributions of this paper are as follows: (1) Previous studies on the diversity of image translation mainly focus on the low-level features of the image, such as hair color and skin color. Our method focuses on semantic differences through deep features, with the aim of improving the semantic diversity of images in the styles. (2) Our model adopts the channel-attention mechanism and reconstructed-image discriminator, which not only highlight the feature details of the generated image, but also ensure the translated images as real as possible. The results on Celeba-HQ dataset verify the superiority of our model on both visual quality and semantic diversity.

The makeup transfer task aims to transfer makeup styles from a reference makeup image to another non-makeup image. Previous methods achieved great progress with the same face angle, but failed to achieve good results when transferring between images with large spatial misalignment of face angles. In this paper, We propose a method for facial makeup transfer for large-angle spatial misalignment which based on generative adversarial networks. It first utilizes the Neural Head Reconstruction Module to process the reference image to obtain a new reference image. The new reference image maintains the original makeup style but has the same face angle as the source image, and it still has high definition and realism. Thus, the subsequent makeup transfer will be much easier. In addition, in order to ensure the authenticity and clarity of local details as much as possible, we also introduce the concept of local perception to transfer the makeup while keeping the color of the original character’s eyeballs, ears and neck unchanged. Besides, we can also realize controllable and partial makeup transfer. Experimental results show that our method achieves the state-of-the-art compared to existing methods.

Unpaired photo to caricature generation is a challenging but meaningful task. Generating high quality caricatures with rich texture/color and plausible exaggeration is important. Previous methods often respectively deal with the shape transformation and texture/color style. We argue that shape transformation can be treated as same as texture/color. Thereby, shape transformation and texture/color can be transferred at the same time. In this paper, we proposed a new method namely AdsSe-GAN for photo-to-caricature generation, which consists of a new normalization function called AdaSLIN and a new semi-cycle consistency loss. The AdaSLIN adaptively selects Layer Normalization or Instance Normalization to simultaneously transfer texture/color and shape transformation. Besides we present semi-cycle consistency loss which only imposes L1 norm on caricature-to-photo process, which is different from existing methods that apply cycle consistency loss to preserve the original domain information. In fact, while generating caricature, taking no account of the cycle restriction makes our model generate caricature with more dis-tinct exaggeration and higher quality. Experimental results on a public caricature dataset, WebCaricature, show the effectiveness of our proposed method compared with the state-of-the-art models.

Multi-view clustering which integrates the complementary information from different views for better clustering, is a fundamental and important topic in machine learning. In this paper, we present a multi-view co-training clustering algorithm based on global and local structure preserving. Here the global structure is referred to the integration of the within-cluster compactness and between-cluster separation; the local structure is referred to the neighborhood information. Our algorithm at first preserves both the global and local structure to the subspace in each view. And then, this algorithm obtains the clustering result in the subspace of each view, and utilizes the clustering labels of one view to guide the subspace clustering in another view. In this way, the differences and compatibilities among the multiple views are fused together to form the final cluster partition. Therefore, the clustering result takes full account of the global and local structure information of the multi-view data, which is helpful to improve the of clustering accuracy. Experimental results on the multi-view text datasets and image datasets demonstrate the effectiveness and correctness of the proposed algorithm.

Most existing multi-view clustering methods focus on the global structure or local structure among samples, and few methods focus on the two structures at the same time. In this paper, we propose a Multi-view Latent subspace Clustering based on both Global and Local structure (MLCGL). In this method, a latent embedding representation is learned by exploring the complementary information from different views. In the latent space, not only the global reconstruction relationship but also the local geometric structure among the latent variables are discovered. In this way, a unified affinity graph matrix is constructed in the latent space for different views, which indicates a clear between-class relationship. Meanwhile, a rank constraint is introduced on the Laplacian graph to facilitate the division of samples into the required clusters. In MLCGL, the affinity graph also provides positive feedback to optimize the learned latent representation and contribute to divided it into reasonable clusters. Moreover, we present an alternating iterative optimization scheme to optimize objective functions. Compared with the state-of-art algorithms, MLCGL has achieved excellent experimental performance on several real-world datasets.

Image-to-image translation aims at learning the mapping between an input image and an output image using a training set of aligned image pairs. In reality, obtaining paired images is difficult and expensive. Generally, the data often exist in the form of partial pairing, that is, a small number of images are paired and most of the images are not paired. In this paper, we present a semi-paired image-to-image translation approach using neighbor-based generative adversarial networks. Our goal is to break the restriction that training images must be paired, and meanwhile guarantee the quality of image translation. For the unpaired images, we introduce an inverse mapping and cycle consistency loss to enforce the image reconstruction; for the paired images, we make full use of the one-to-one strong correlation to guide the image translation. To further take advantage of the paired images, our approach employs neighbor images to further expand the paired information and establishes the neighbor-based cycle consistency. Our method is characterized by flexibility and adaptability under various scenarios, such as target deformation, day-night transformation, etc. Compared with the previous methods, the experimental results prove the superiority of our method.

In this paper, a novel clustering preserving projection method is presented to retain the cluster (or group) structures hidden in the original data. This method involves two steps: (1) the famous clustering method Fuzzy c-means is utilized to discover the cluster distribution in data; (2) such distribution is preserved by finding a linear embedding that minimizes the intra-cluster compactness in low-dimensional space. The feasibility and effectiveness of the proposed method are demonstrated on UCI dataset and USPS digital handwritten dataset.

With the diversification of data sources, the multi-view data with multiple expressions have been appeared in various application scenarios. These multi-view data generally have high dimensions, large amounts and often lack of label information. Therefore, it is very important to learn multi-view data in an unsupervised way so as to analyze and excavate the potential valuable information. In this paper, we propose a multi-view locality preserving embedding algorithm with view similarity constraint for data dimension reduction. This algorithm not only preserves the local structure into low-dimensional space for each view, but also implements the similarity constraints between different views. On this basis, the algorithm looks for a joint embedding of low-dimensional subspace, so that the neighborhood among samples in original high-dimensional space can be maintained in the subspace, and the structures corresponding to different views are consistent with each other. This algorithm achieves good experimental results both in artificial data sets and multi-view data sets, which prove the correctness and feasibility of the algorithm.