In Face Super-Resolution, it is a challenging problem to preserve the identity features of a person by restoring clear and realistic facial image details. However, under conditions of high illumination, captured face images are often affected by strong light exposure and various degradation factors, resulting in blurring, overexposure, and loss of facial texture. Severe face image degradation further compromises the structural integrity and texture details of high-resolution face images, thereby reducing the restoration effectiveness. To address these challenges, this paper proposes a face super-resolution model specifically designed for overexposed scenarios. We design a normalization-based multi-scale exposure module to address the brightness issues in facial images caused by varying degrees of exposure. By applying normalization to facial images with different levels of overexposure, this module significantly reduces facial brightness discrepancies and lowers the overall brightness levels. Additionally, we introduce a color space conversion module to correct color distortions following exposure normalization, increasing visual coherence. Finally, a dual-branch feature aggregation stage is employed to extract and enhance face feature information, ensuring that face images exhibit clear texture details and complete facial contours. Extensive experiments are conducted on the most representative datasets, including four datasets. Notably, across various tasks, EFSRNet outperforms publicly available state-of-the-art methods by 0.1-0.3 dB on most metrics and demonstrates exceptional performance in facial downstream tasks as well.

Current state-of-the-art (SOTA) methods in 3D Human Pose Estimation (HPE) are primarily based on Transformers. However, existing Transformer-based 3D HPE backbones often encounter a trade-off between accuracy and computational efficiency. To resolve the above dilemma, in this work, we leverage recent advances in state space models and utilize Mamba for high-quality and efficient long-range modeling. Nonetheless, Mamba still faces challenges in precisely exploiting local dependencies between joints. To address these issues, we propose a new attention-free hybrid spatiotemporal architecture named Hybrid Mamba-GCN (Pose Magic). This architecture introduces local enhancement with GCN by capturing relationships between neighboring joints, thus producing new representations to complement Mamba's outputs. By adaptively fusing representations from Mamba and GCN, Pose Magic demonstrates superior capability in learning the underlying 3D structure. To meet the requirements of real-time inference, we also provide a fully causal version. Extensive experiments show that Pose Magic achieves new SOTA results (0.9 mm drop) while saving 74.1% FLOPs. In addition, Pose Magic exhibits optimal motion consistency and the ability to generalize to unseen sequence lengths.

Recent research on Diffusion Models and Transformers has brought significant advancements to 3D Human Pose Estimation (HPE). Nonetheless, existing methods often fail to concurrently address the issues of accuracy and generalization. In this paper, we propose a Geometry-guided Diffusion Model with Masked Transformer (Masked Gifformer) for robust multi-view 3D HPE. Within the framework of the diffusion model, a hierarchical multi-view transformer-based denoiser is exploited to fit the 3D pose distribution by systematically integrating joint and view information. To address the long-standing problem of poor generalization, we introduce a fully random mask mechanism without any additional learnable modules or parameters. Furthermore, we incorporate geometric guidance into the diffusion model to enhance the accuracy of the model. This is achieved by optimizing the sampling process to minimize reprojection errors through modeling a conditional guidance distribution. Extensive experiments on two benchmarks demonstrate that Masked Gifformer effectively achieves a tradeoff between accuracy and generalization. Specifically, our method outperforms other probabilistic methods by > 40% and achieves comparable results with state-of-the-art deterministic methods. In addition, our method exhibits robustness to varying camera numbers, spatial arrangements, and datasets.

Generative diffusion prior captured from the off-the-shelf denoising diffusion generative model has recently attained significant interest. However, several attempts have been made to adopt diffusion models to noisy inverse problems either fail to achieve satisfactory results or require a few thousand iterations to achieve high-quality reconstructions. In this work, we propose a diffusion-based image restoration with error contraction and error correction (DiffECC) method. Two strategies are introduced to contract the restoration error in the posterior sampling process. First, we combine existing CNN-based approaches with diffusion models to ensure data consistency from the beginning. Second, to amplify the error contraction effects of the noise, a restart sampling algorithm is designed. In the error correction strategy, the estimation-correction idea is proposed on both the data term and the prior term. Solving them iteratively within the diffusion sampling framework leads to superior image generation results. Experimental results for image restoration tasks such as super-resolution (SR), Gaussian deblurring, and motion deblurring demonstrate that our approach can reconstruct high-quality images compared with state-of-the-art sampling-based diffusion models.

Although face recognition has achieved great success in many areas, cross-resolution face recognition (CRFR) still remains a challenging task due to the large domain gap between low-resolution (LR) and high-resolution (HR) images. In this paper, we propose an adaptive prototype triplet loss (APTL) for CRFR. The APTL pulls the features close to their own prototypes, and pushes them away from the prototypes of other classes. Thus, the angular distances between features and prototypes from the same class are closer than those from different classes. Furthermore, to better exploit the similarity information among different identities, we adaptively adjust the margin term in the loss. Since the proposed APTL is applied simultaneously to HR and LR features, the gap between two domains can be narrowed naturally. Experiments on LFW and SCface datasets illustrate the superiority of our method.

Diffusion-based image super-resolution (SR) models have attracted substantial interest due to their powerful image restoration capabilities. However, prevailing diffusion models often struggle to strike an optimal balance between efficiency and performance. Typically, they either neglect to exploit the potential of existing extensive pretrained models, limiting their generative capacity, or they necessitate a dozens of forward passes starting from random noises, compromising inference efficiency. In this paper, we present DoSSR, a $\textbf{Do}$main $\textbf{S}$hift diffusion-based SR model that capitalizes on the generative powers of pretrained diffusion models while significantly enhancing efficiency by initiating the diffusion process with low-resolution (LR) images. At the core of our approach is a domain shift equation that integrates seamlessly with existing diffusion models. This integration not only improves the use of diffusion prior but also boosts inference efficiency. Moreover, we advance our method by transitioning the discrete shift process to a continuous formulation, termed as DoS-SDEs. This advancement leads to the fast and customized solvers that further enhance sampling efficiency. Empirical results demonstrate that our proposed method achieves state-of-the-art performance on synthetic and real-world datasets, while notably requiring $\textbf{\emph{only 5 sampling steps}}$. Compared to previous diffusion prior based methods, our approach achieves a remarkable speedup of 5-7 times, demonstrating its superior efficiency.

Numerous CNN-based algorithms have been proposed to reconstruct high-quality face images. However, the inability of convolution operation to model long-distance relationships limits the performance of the CNN-based methods. Moreover, in the high-resolution (HR) image reconstruction stage, with the well decoded feature representations, more efficient architecture design can be explored to synthesize pixel-level image details. In this work, we propose a spatial attention-guided CNN-Transformer aggregation network (SCTANet) for face image super-resolution (FSR) tasks. The core component in the deep feature extraction stage is the Hybrid Attention Aggregation (HAA) block. The HAA block has two parallel paths, one for the Residual Spatial Attention (RSA) block, the other for the Multi-scale Patch embedding and Spatial-attention Masked Transformer (MPSMT) block. The HAA block combines the strengths of CNN and transformer to effectively exploit both local and global information. For the reconstruction stage, we propose to use the Sub-pixel MLP-based Upsampling (SMU) module instead of the conventional CNN architecture. The SMU module promotes the reconstruction of pixel-level image details and reduces computational complexity. Extensive experiments on both synthetic and real-world face datasets demonstrate the superiority of our proposed SCTANet over state-of-the-art methods.

The development of loss functions over the past few years has brought great success to face recognition. Most algorithms focus on improving the intra-class compactness of face features but ignore the inter-class separability. In this paper, we propose a method named EvenFace, which introduces a regularization variance item and a mean term of inter-class separability to further promote the even distribution of class centers on the hypersphere, thereby increasing the inter-class distance. In order to evaluate the inter-class separability, a new index is proposed to better reflect the distribution of class centers and guide the classification. By penalizing the angle between each identity and its surrounding neighbors, the resulting uniform distribution of identities enables full exploitation of the feature space, leading to discriminative face representations. Our proposed loss function can effectively boost the performance of softmax loss variants. Quantitative comparisons with other state-of-the-art methods on several benchmarks demonstrate the superiority of EvenFace.

With the help of convolutional neural networks (CNNs), deep learning-based methods have achieved remarkable performance in face super-resolution (FSR) task. Despite their success, most of the existing methods neglect non-local correlations of face images, leaving much room for improvement. In this paper, we introduce a novel end-to-end trainable attention-driven graph neural network (AD-GNN) for more discriminative feature extraction and feature relation modeling. This is achieved by two major components. The first component is a cross-scale dynamic graph (CDG) block. The CDG block considers cross-scale relationships of patches in distant areas and employs two dynamic graphs to construct enhanced features. The second component is a series of channel attention and spatial dynamic graph (CASDG) blocks. A CASDG block has a channel-wise attention unit and a spatial-aware dynamic graph (SDG) unit. The SDG unit extracts informative features by exploring spatial non-local self-similarity information of the patches using dynamic graph convolution. Using these two components, facial details can be effectively reconstructed with the help of information supplemented by similar but spatially remote patches and structural information of faces. Extensive experiments on two public benchmarks demonstrate the superiority of AD-GNN over the state-of-the-art FSR methods.

Ophthalmologists have used fundus images to screen and diagnose eye diseases. However, different equipments and ophthalmologists pose large variations to the quality of fundus images. Low-quality (LQ) degraded fundus images easily lead to uncertainty in clinical screening and generally increase the risk of misdiagnosis. Thus, real fundus image restoration is worth studying. Unfortunately, real clinical benchmark has not been explored for this task so far. In this paper, we investigate the real clinical fundus image restoration problem. Firstly, We establish a clinical dataset, Real Fundus (RF), including 120 low- and high-quality (HQ) image pairs. Then we propose a novel Transformer-based Generative Adversarial Network (RFormer) to restore the real degradation of clinical fundus images. The key component in our network is the Window-based Self-Attention Block (WSAB) which captures non-local self-similarity and long-range dependencies. To produce more visually pleasant results, a Transformer-based discriminator is introduced. Extensive experiments on our clinical benchmark show that the proposed RFormer significantly outperforms the state-of-the-art (SOTA) methods. In addition, experiments of downstream tasks such as vessel segmentation and optic disc/cup detection demonstrate that our proposed RFormer benefits clinical fundus image analysis and applications.

Full-reference image quality assessment aims to predict the perceptual quality of a distorted image based on its similarity to the pristine reference. In this paper, we propose a robust image similarity metric by fully exploring the representation power of deep learning-based features. A convolutional neu-ral network (CNN) is adopted to extract deep features from multiple scales. We show that such CNN features that con-tain multi -scale visual information are comprehensive and ro-bust enough for quality assessment. We further propose a quality-oriented feature regression (QOFR) module based on the multi-layer perceptron architecture. The QOFR module can efficiently integrate hierarchy CNN features and generate the final quality score. Extensive experiments on the bench-mark datasets demonstrate that our method achieves state-of-the-art performance with outstanding robustness and general-ization ability.

Brain stroke lesion segmentation is of great importance for stroke rehabilitation neuroimaging analysis. Due to the large variance of stroke lesion shapes and similarities of tissue intensity distribution, it remains a challenging task. To help detect abnormalities, the anatomical symmetries of brain magnetic resonance (MR) images have been widely used as visual cues for clinical practices. However, most methods for brain images segmentation do not fully utilize structural symmetry information. This paper presents a novel mirror difference aware network (MDAN) for stroke lesion segmentation. The network uses an encoder-decoder architecture, aiming at holistically exploiting the symmetries of image features. Specifically, a differential feature augmentation (DFA) module is developed in the encoding path to highlight the semantically pathological asymmetries of features in abnormalities. In the DFA module, a Siamese contrastive supervised loss is designed to enhance discriminative features, and a mirror position-based difference augmentation (MDA) module is used to further magnify the discrepancy. Moreover, mirror feature fusion (MFF) modules are applied to efficiently fuse and transfer the information both of the original input and the horizontally flipped features to the decoding path. Extensive experiments on the Anatomical Tracings of Lesions After Stroke (ATLAS) dataset show the proposed MDAN outperforms the state-of-the-art methods.

Existing shadow removal methods often struggle with two problems: color inconsistencies in shadow areas and artifacts along shadow boundaries. To address these two problems, we propose a novel shadow mask-based semantic-aware network (S 2 Net) that uses shadow masks as guidance for shadow removal. The color inconsistency problem is solved in two steps. First, we use a series of semantic-guided dilated residual (SDR) blocks to transfer statistical information from non-shadow areas to shadow areas. The shadow mask-based semantic transformation (SST) operation in SDR enables the network to remove shadows while keeping non-shadow areas intact. Then, we design a refinement block by incorporating semantic knowledge of shadow masks and applying the learned modulated convolution kernels to get traceless and consistent output. To remove artifacts along shadow boundaries, we propose a newly designed boundary loss. The boundary loss encourages spatial coherence around shadow boundaries. By including the boundary loss as part of the loss function, a significant portion of artifacts along shadow boundaries can be removed. Extensive experiments on the ISTD, ISTD+, SRD and SBU datasets show our S 2 Net outperforms existing shadow removal methods.

The main focus of most existing face hallucination methods is to generate visually pleasing results. However, in many applications, the final goal is to identify the person in the low-resolution (LR) image. In this paper, we propose a texture and identity integration network (TIIN) to effectively incorporate identity information into face hallucination tasks. TIIN consists of an identity-preserving denormalization module (IDM) and an equalized texture enhance module (ETEM). The IDM exploits the identity prior and the ETEM improves image quality through histogram equalization. To extract identity information effectively, we propose a resolution-robust identity knowledge distillation network (RIKDN). RIKDN is specifically designed for LR face recognition and can be of independent interest. It employs two teacher-student streams. One stream narrows the performance gap between high-resolution (HR) and LR images. The other distills correlation information from the HR-HR teacher stream to guide learning in the LR-HR student stream. We conduct extensive experiments on multiple datasets to demonstrate the effectiveness of our methods.

Stroke is one of the leading causes of death around the world. Segmenting atherosclerotic plaques in carotid arteries from ultrasound images is of great value for preventing and treating ischemic stroke, yet still challenging due to the ambiguous boundary of plaque and intense noise in ultrasound. In this paper, we introduce a new approach for carotid plaque segmentation, namely Multi-Branch Feature Fusion Network (MBFF-Net). Inspired by the prior knowledge that carotid plaques generally grow in carotid artery walls (CAWs), we design a Multi-Branch Feature Fusion (MBFF) module with three branches. Specifically, the first two branches are well-designed to extract plaque features of multiple scales and different contexts, and the other branch is to exploit the prior information of CAWs. In addition, a boundary preserving structure is applied to alleviate the ambiguity of plaque boundary. With the proposed MBFF and the novel structure, our model is capable of extracting discriminative features of plaques and integrating the location information of CAWs for better segmentation. Experiments on the clinical dataset demonstrate that our model outperforms state-of-the-art methods. Code is available at https://github.com/mishiyu/MBFF.