The rich spatial and spectral information in hyperspectral images (HSIs) makes spectral-spatial relationships essential for HSI classification (HSIC). Recent advancements indicate convolutional neural networks (CNNs) excel in HSIC but often struggle with precise spectral feature extraction. Moreover, the abundance of spectral information presents challenges in efficient feature representation and minimizing cross-domain interference. To address these limitations, we propose an efficient sequential spectral-spatial feature convolution network (S3FCN), employing successive subnetworks for spectral and spatial feature extraction with depthwise separable convolution. This approach balances the preservation of deep spectral and spatial features while significantly reducing network parameters, enhancing both performance and computational efficiency. We also introduce a sequential spectral-spatial attention module (S3AM) to integrate cross-domain correlations. This module utilizes spectral features from the preceding subnetwork and multilevel residual layers for in-depth exploration of spatial features, enabling deep integration for improved classification performance. The proposed architecture's effectiveness is verified on five benchmark HSI datasets, including Pavia University, Salinas Valley, Kennedy Space Center, Indian Pines, and Houston 2013. Experimental results demonstrate that the sequential spectral-spatial connection in the feature extraction and attention mechanism integrated with depthwise separable convolution collectively surpasses current state-of-the-art (SOTA) techniques in classification accuracy with overall accuracies of 98.28%, 97.63%, 99.31%, 96.72%, and 95.38% across different datasets, while limiting the computation overhead, ensuring balanced network efficiency.

Fringe projection profilometry (FPP) has become one of the most powerful techniques for three-dimensional (3D) non-contact measurement. However, in practical scenarios, the various reflectivity of the unknown measured objects often greatly makes the system unable to achieve the theoretical precision under the same system parameter settings. Therefore, the adaptively system parameter setting is essential to be developed. In this paper, we propose a novel metric model, i.e. the accuracy quality function, for initial accuracy evaluation using in-situ acquired images under the current parameter settings. The causes that potentially affects the ultimate accuracy are analyzed via theoretical derivation and further adopted within the evaluation model. In addition, an optimal exposure selection method based just two images is carried out to fast adjusting. Experimental results demonstrated that the proposed accuracy quality model aligns well with the actual condition. Under optimal exposure, it achieved a significant reduction in phase error by 36.15% and by 21.39% in low- and high- exposure, highlighting its strong performance and potential for high-accuracy and in-situ 3D shape measurement applications.

Recently, neural image compression has made significant progress in reducing rate-distortion and has received widespread attention. However, existing methods focus more on perfecting entropy models yet overlook the ability of their encoder networks to extract non-linear features of images, which can promote compression performance. In this paper, we design a learning-based asymmetric image compression network to enhance the feature representation capability for improved compression quality. Firstly, we propose a high-preserving information block (HPIB) consisting of a high-frequency filtering module (HFM) and a feature modulation module (FMM) to fully utilize the different frequency information in images. Secondly, we progressively use the HPIB layer to design a high-performance encoder network for high-fidelity feature extraction. Results from extensive experiments demonstrate that our network performs superior to the prior art in terms of both PSNR and MS-SSIM metrics and achieves 3.91% and 8.88 % BD-rate over VVC on the Kodak and CLIC datasets, respectively.

To merge complementary information from multimodal images, such as thermal saliency from infrared images and texture details from visible images, traditional multi-scale transform-based methods have been extensively studied, with deep learning-based methods gaining significant popularity in recent years. However, there has been limited research on optimally combining the advantages of these two categories in fusion. In this paper, we propose a novel infrared and visible image fusion (IVIF) framework, WaveFusionNet, which integrates precise frequency feature decomposition from the discrete wavelet transform (DWT) with the comprehensive feature extraction from the multi-scale encoder. Firstly, we train an encoder-decoder network for multi- scale feature extraction and image reconstruction. DWT is used for down-sampling with minimal information loss by decomposing extracted features into low and high-frequency sub-bands. Next, a dual-band feature fusion (DBFF) module is trained to merge these sub-bands by integrating a spatial feature transform-based sub-network for low-frequency fusion and a maximum absolute value selection strategy for fusing high- frequencies. Finally, all fused sub-bands are fed into the pre-trained decoder to reconstruct the final image. Experimental results on three benchmark datasets (TNO, Roadscene, and MSRS) demonstrate that the proposed fusion method outperforms recent IVIF methods in both quantitative assessment and visual perception while maintaining competitive time complexity.

Currently, with the extensive application of digital cameras in dynamic capturing, implications such as camera jitter, out-of-focus, and target motion induce various types and degrees of image blurring. Deep learning (DL) is a powerful technique that offers data-adaptive recovery without prior characterization of deblurring filter kernels. However, end-to-end networks can still be improved to restore regions with severe localized blurring. Therefore, we propose a multi-scale circular transformer (MSC-Former) employing averaged neighborhood attention (AvgNA) to solve this problem. It computes the local attention of each feature pixel by learning the correlation between the center and the surrounding windowed neighborhood, then produces integrated attention with direct averaging. We employ a multi-scale circular strategy (MSCS) to compute attention at different spatial scales to expand the receptive field while maintaining a low parameter count. It uses concentric circular regions with varying radii to define neighborhoods at different scales, which expands the receptive field during attention computation while capturing spatial continuity across larger neighborhoods. Experimental results demonstrate that the proposed method surpasses the recent state-of-the-art deblurring techniques on the benchmark dataset.

In recent years, many fusion algorithms based on multi-scale transform or neural networks have been proposed to improve medical image fusion (MIF) performance. However, there is still enormous potential to explore the combination of different fusion theories. In this paper, we propose a novel MIF framework to integrate powerful feature representation abilities of the deep learning model and accurate frequency decomposition characteristics of discrete wavelet transform (DWT). Firstly, a multi-scale encoder-decoder network is well-trained to extract feature information in different scales and achieve efficient image reconstruction. In particular, DWT is introduced into each scale to decompose the extracted features into high- and low-frequency sub-bands for information preservation during down-sampling. An elaborate feature fusion process is designed to achieve multi-scale fusion while merging different frequency sub-bands. Experiment results on benchmark datasets demonstrate that the proposed fusion framework outperforms current state-of-the-art methods with comparable time complexity in both objective and subjective evaluation.

Facial semantic guidance (including facial landmarks, facial heatmaps, and facial parsing maps) and facial generative adversarial networks (GAN) prior have been widely used in blind face restoration (BFR) in recent years. Although existing BFR methods have achieved good performance in ordinary cases, these solutions have limited resilience when applied to face images with serious degradation and pose-varied (e.g., looking right, looking left, laughing, etc.) in real-world scenarios. In this work, we propose a well-designed blind face restoration network with generative facial prior. The proposed network is mainly comprised of an asymmetric codec and a StyleGAN2 prior network. In the asymmetric codec, we adopt a mixed multi-path residual block (MMRB) to gradually extract weak texture features of input images, which can better preserve the original facial features and avoid excessive fantasy. The MMRB can also be plug-and-play in other networks. Furthermore, thanks to the affluent and diverse facial priors of the StyleGAN2 model, we adopt it as the primary generator network in our proposed method and specially design a novel self-supervised training strategy to fit the distribution closer to the target and flexibly restore natural and realistic facial details. Extensive experiments on synthetic and real-world datasets demonstrate that our model performs superior to the prior art for face restoration and face super-resolution tasks.

In recent years, extensive research has been conducted on visible and infrared image fusion (VIF) task using traditional multi-scale transform-based and deep learning model-based methods. However, there is still a need to explore the combination of neural networks and multi-scale transform. This paper proposes a novel fusion framework based on a dual-path encoder-decoder and multi-scale transform. A dual-path encoder is trained to extract rich features at different depths from source images, while a shared decoder is trained to efficiently reconstruct images from the extracted feature space. We apply the discrete wavelet transform (DWT) to generate various frequency components from the extracted features. A fusion module is utilized to achieve fusion for low and high-frequency sub-bands, respectively, which is constrained by a gradient-based fusion loss function and an absolute values maximum-selection strategy. Our proposed method is superior to current state-of-the-art fusion methods, as demonstrated through quantitative and qualitative comparisons of publicly available datasets.

Visual place recognition (VPR) is a challenging task with the unbalance between enormous computational cost and high recognition performance. Thanks to the practical feature extraction ability of the lightweight convolution neural networks (CNNs) and the train-ability of the vector of locally aggregated descriptors (VLAD) layer, we propose a lightweight weakly supervised end-to-end neural network consisting of a front-ended perception model called GhostCNN and a learnable VLAD layer as a back-end. GhostCNN is based on Ghost modules that are lightweight CNN-based architectures. They can generate redundant feature maps using linear operations instead of the traditional convolution process, making a good trade-off between computation resources and recognition accuracy. To enhance our proposed lightweight model further, we add dilated convolutions to the Ghost module to get features containing more spatial semantic information, improving accuracy. Finally, rich experiments conducted on a commonly used public benchmark and our private dataset validate that the proposed neural network reduces the FLOPs and parameters of VGG16-NetVLAD by 99.04% and 80.16%, respectively. Besides, both models achieve similar accuracy.

With the aim of improving the reconstruction quality for image compressive sensing, we propose a channel shuffle reconstruction network (CSRNet) by jointly optimize the sampling and the inverse reconstruction processes. Firstly, we build an initial reconstruction sub-network (IRSN) to adaptively learn the measurement matrix and generate a preliminary reconstructed image. Then, a deep channel shuffle sub-network (CSSN) is added to further improve the image quality. Specially, we combine the merits of the inverted residual structure with channel shuffle operation to propose an efficient channel shuffle block in CSSN. The inverted residual structure endows the network with more powerful feature extraction ability. The channel shuffle operation further promotes information interaction. Besides, we exploit the multi-scale convolution block to make better use of feature information at different scales. Experiments on the benchmark datasets demonstrate that the proposed network outperforms previous state-of-the-art algorithms with a comparable time complexity.

Recently, deep-learning based reconstruction methods have been proposed to improve recovery performance of compressive sensed image and overcome expensive time complexity drawbacks of iteration-based traditional algorithms. In this paper, we propose an end-to-end multi-scale residual convolutional neural network (CNN), dubbed MSRNet, to simulate image compressive sensing (CS) and inverse reconstruction process in real situation. In the reconstruction stage of MSRNet, we apply three parallel channels with different convolution kernel sizes to exploit different-scale feature information. Besides, residual learning is introduced to accelerate training process and enhance prediction accuracy of network. Moreover, different from generating CS measurements by random measurement matrix in previous methods, we integrate compressive sample process into MSRNet, which means measurement matrix can be adaptively learned by training the network. Experiments on benchmark datasets show our method outperforms other state-of-the-art algorithms by large margins and set a new level for CS reconstruction with competitive time complexity.

Deep residual convolutional neural network (CNN) has recently achieved great success in image super-resolution (SR). Because residual learning accelerates convergence rate and eases the difficulty for reconstructing high-resolution (HR) image, these CNN models can achieve higher peak signal to noise ratio (PSNR) values with lower training cost. However, residual image used in present residual network still contains much high frequency information, which increases learning burden and limits learning ability of residual network. Moreover, training a very deep network faces many obstacles and costs too much time. In this paper, we propose a multiple residual learning network (MRLN), which not only further simplifies information complexity of residual image and improves the accuracy of residual network, but also obviously reduces time cost for training a very deep CNN. In MRLN, we use a shallow network formed by 30-layer convolutional layers as basic model and train it for multiple times. The output of previous basic model is used as the HR input of the next one. In this way, an extremely large CNN is converted into a series connection of shallow networks. Fig. 1 shows PSNR of recent state-of-the-art CNN models for scale factor 2 on Set5, our method performs better than other methods and set a new level for SR.