State-space models (SSMs) excel in modeling longrange dependencies, but Mamba is limited in modeling local interactions. Existing methods typically use windowed multi-head self-attention mechanisms (MHSAs) to address this problem, but this brings the challenge of increased parameters. To address this challenge, we propose a Feature Enhancement Aggregation Network (FEANet) that effectively models local dependencies using a Local Feature Enhancement Block (LFEB) and further enhances and fuses global and local features with the help of a Spatial Channel Aggregation Block (SCAB) to achieve advanced super-resolution performance. Specifically, we design a Local Feature Enhancement Block (LFEB) and a Spatial Channel Fusion Block (SCFB). We use the LFEB module to help enhance the image features extracted by Mamba, enhancing the ability to capture local features while maintaining computational efficiency. SCFB, as a substitute for a Feed-Forward Network (FFN), effectively refines deep features through partial convolutions and channel interactions, and SCFB also achieves the fusion of global and local features. Experiments show that our method achieves excellent performance while maintaining model complexity.

Despite addressing the limitations of CNNs, Transformer-based image super-resolution methods construct long-range features solely in the spatial domain, leading to heavy computational burdens. To tackle this problem, we propose a Spatial-Frequency Collaborative Network(SFCN), which effectively leverages frequency domain information to model non-local dependencies while simultaneously utilizing spatial information to capture local features, thereby achieving superior super-resolution performance. Specifically, we propose a Spatial-Frequency Collaborative Block (SFCB) and a Multi-Scale Gated Feed-Forward Network (MSGFN). The SFCB integrates spatial and frequency domain information, endowing it with both local and non-local feature capture capabilities while maintaining computational efficiency. Furthermore, we introduce the MSGFN to enhance non-linear representation via gating mechanisms and asymmetric convolutions, while capturing multi-scale local details. Extensive experiments demonstrate that our method achieves superior performance while maintaining model complexity comparable to other lightweight methods.

MRI (Magnetic Resonance Imaging) super-resolution technology is a crucial research direction in the field of medical image processing. This paper aims to enhance the spatial resolution of MRI, improve image quality, and enhance the diagnostic capabilities of medical imaging. In this study, we first preprocess MR images to ensure the quality of input data. Subsequently, we designed a network, RPCN, based on a residual attention mechanism. This network adaptively focuses on important regions in MRI and employs the concept of residual learning to effectively capture high-frequency information for super-resolution reconstruction. In the RPCN network, we introduce a channel attention mechanism and a pixel attention mechanism, and design an attention-serial-fusion RPC structure to enable the network to more effectively explore correlations between different channels and better capture image features. Extensive experiments demonstrate the significant success of our method in enhancing MRI resolution. The introduction of residual structure and attention mechanisms allows our network to delve deeper into learning image details, enabling better reconstruction of image edges, textures, and microstructures. We also use quantitative evaluation metrics such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) to objectively assess the performance of the RPCN network structure. Multiple sets of comparative experiments validate its robustness and generality on different MRI datasets.

In recent years, there has been significant progress in efficient and lightweight image super-resolution, due in part to the design of several powerful and lightweight attention mechanisms that enhance model representation ability. However, the attention maps of most methods are obtained directly from the spatial domain, limiting their upper bound due to the locality of spatial convolutions and limited receptive fields. In this paper, we shift focus to the frequency domain, since the natural global properties of the frequency domain can address this issue. To explore attention maps from the frequency domain perspective, we investigate and correct some misconceptions in existing frequency domain feature processing methods and propose a new frequency domain attention mechanism called frequency-enhanced pixel attention (FPA). Additionally, we use large kernel convolutions and partial convolutions to improve the ability to extract deep features while maintaining a lightweight design. On the basis of these improvements, we propose a large kernel frequency-enhanced network (LKFN) with smaller model size and higher computational efficiency. It can effectively capture long-range dependencies between pixels in a whole image and achieve state-of-the-art performance in existing efficient super-resolution methods.

This paper provides a comprehensive review of the NTIRE 2024 challenge, focusing on efficient single-image super-resolution (ESR) solutions and their outcomes. The task of this challenge is to super-resolve an input image with a magnification factor of x4 based on pairs of low and corresponding high-resolution images. The primary objective is to develop networks that optimize various aspects such as runtime, parameters, and FLOPs, while still maintaining a peak signal-to-noise ratio (PSNR) of approximately 26.90 dB on the DIV2K_LSDIR_valid dataset and 26.99 dB on the DIV2K_LSDIR_test dataset. In addition, this challenge has 4 tracks including the main track (overall performance), sub-track 1 (runtime), sub-track 2 (FLOPs), and sub-track 3 (parameters). In the main track, all three metrics (ie runtime, FLOPs, and parameter count) were considered. The ranking of the main track is calculated based on a weighted sum-up of the scores of all other sub-tracks. In sub-track 1, the practical runtime performance of the submissions was evaluated, and the corresponding score was used to determine the ranking. In sub-track 2, the number of FLOPs was considered. The score calculated based on the corresponding FLOPs was used to determine the ranking. In sub-track 3, the number of parameters was considered. The score calculated based on the corresponding parameters was used to determine the ranking. RLFN is set as the baseline for efficiency measurement. The challenge had 262 registered participants, and 34 teams made valid submissions. They gauge the state-of-the-art in efficient single-image super-resolution. To facilitate the reproducibility of the challenge and enable other researchers to build upon these findings, the code and the pre-trained model of validated solutions are made publicly available at https://github.com/Amazingren/NTIRE2024_ESR/.

In today's hospitals, doctors often need to use medical equipment to find the cause of the disease, and the clarity of medical images often affects the doctor's judgment of the patient's condition. Having high-resolution medical images can help doctors better treat patients. But nowadays, most super-resolution models cannot effectively restore medical images, resulting in poor image restoration results. This article follows the idea of cyclic generative networks and trains them using Wasserstein distance to solve the task of image reconstruction. The model consists of an image reconstruction network, an image degradation network, and two discriminators. In the generative network, we implement cyclic consistency based on Wasserstein distance to establish a nonlinear end-to-end mapping from noisy LR input images to denoised and deblurred HR output images. This article experimented and validated the performance on the FastMRI dataset, and compared with existing mainstream methods, the results showed that it was superior to current mainstream methods in medical image super-resolution reconstruction.

This study investigates the utilization of sophisticated deep learning techniques, specifically, the integration of UNet and ResNet models with attention processes, to improve the accuracy of retinal blood vessel segmentation in fundus images. It focuses on the urgent requirement for accurate and effective segmentation methods in the diagnosis and monitoring of disorders connected to eyesight. The work introduces a new method that combines UNet's semantic segmentation capabilities with ResNet's gradient problem avoidance to overcome the constraints of previous segmentation approaches. Incorporating attentive feature fusion and a transformer bridge enhances the model's capacity to concentrate on pertinent image characteristics and gather broader contextual information that is essential for precise vessel segmentation. The study encompasses a thorough approach that encompasses data collection, model structure construction, training techniques, and performance assessment criteria. The results indicate that the model outperforms existing techniques in terms of accuracy, sensitivity, specificity, and F1 score. This has significant implications for clinical applications in ophthalmology. It highlights the significance of employing advanced methodologies in the interpretation of medical images and delineates potential areas for additional enhancement and wider implementation. In summary, the study introduces a very efficient and encouraging method for dividing retinal blood vessels, which has the potential to improve the accuracy of diagnoses and the quality of care for patients in the field of ophthalmology.

Because the landscape of data processing created is likely to rise considerably in the next years due to the proliferation of devices that require data processing at edge computing, it is imperative for data processing issues at the network's edge storage frame. The goal of this study is to find out how deep learning can be used to speed up operation processing. Effective resource management is essential in the age of edge computing to maximize the performance of storage edge networked frameworks. The purpose of this work is to investigate how resource management in such systems could be improved by utilizing CNN-CPU scheduling approaches. Our goal is to optimize job allocation, prioritization, and scheduling in storage edge situations by fusing Convolutional Neural Networks (CNNs) with CPU-based scheduling methods. The suggested method aims to improve overall system performance, reduce latency, and maximize resource use. We illustrate the efficacy of CNN-CPU scheduling in improving resource management in storage edge networked frameworks with experimental validation and performance analysis. This research contributes to advancing the capabilities of edge computing systems by leveraging deep learning techniques for efficient resource allocation and management. The proposed approach seeks to overcome the limitations of traditional scheduling methods by leveraging the power of deep learning. CNNs are renowned for their ability to extract meaningful features from data, and by harnessing this capability, we can optimize resource utilization and minimize latency in storage edge networked frameworks. Through a combination of CNN-based feature extraction and CPU-based scheduling decisions, we strive to achieve efficient and intelligent resource management. This study investigates the potential of CNN-CPU scheduling techniques to address these challenges and enhance resource management in such frameworks. By integrating Convolutional Neural Networks (CNNs) with CPU-based scheduling algorithms, we aim to revolutionize task allocation, prioritization, and scheduling within storage edge environments.

Currently, image restoration models have achieved significant advancements. However, in cases where the missing regions are too large, it often leads to erroneous structures and inconsistent blurry textures with the surrounding areas. To further enhance the performance of image restoration algorithms, we propose a two-stage model—based on edge constraints and residual semantic attention for image restoration. In the first stage, an edge structure generation network is employed to learn and restore missing edge structure information. Subsequently, in the second stage, the complete edge structure information assists a content-texture restoration network in generating the entire image. This involves initially using gated convolutions to extract effective features. Furthermore, a Residual Semantic Attention module (RSA) is designed, replacing regular convolutional kernels with regions known from the image itself. It applies residual combination and additional convolution to enhance the semantic coherence of foreground and background regions while maximizing the retention of original feature information.

在大多数自然图像中,当光源被遮挡时阴影就会出现.阴影去除的研究难点在于所设计的算法不仅关注阴影的去除,还要重建阴影覆盖下的背景亮度、色彩、以及纹理,同时保留非阴影覆盖区域不受影响.本文利用图像多曝光融合(MEF)与图像超分辨率(SR)联合成耦合反馈网络,处理单幅图像阴影去除问题.在ISTD+数据集上进行了大量的实验,证明在阴影去除效果和图像超分辨率重建效果上优于目前的先进方法.

Over the years, with the development of televisions, computers, mobile phones and other equipment, the requirements for clear images, or high-resolution images, have become higher and higher. However, the transmission of high-resolution images is limited by hardware, storage space, and bandwidth, and is not suitable for frequent information interaction. Therefore, if super-resolution reconstruction of low-resolution images can be achieved at the receiving end, the above-mentioned problems can be avoided.

In order to enhance the visual effects of the reconstructed images in the single image super-resolution, and solve the problem of instability of the training phase of super-resolution with generative adversary network (SRGAN), a new super-resolution model is built with more depth and width, and the corresponding super-resolution algorithm is proposed. The network structure of the original SRGAN is modified, so that deeper and wider convolutional networks can be used with high efficiency. Furthermore, the loss function is employed to balance the training of the generative network and the discriminator network, and solve the instability problem in the training phase. The training images for the network are obtained from the database of ImageNet on the web as other researchers. Experimental results show that images reconstructed by the proposed algorithm has better visual effects than those by the original SRGAN. In addition, the objective measures of PSNR and SSIM of the reconstructed images have also been improved.

Fusion results of low-resolution multispectral (LRMS) images and high-resolution panchromatic (Pan) images, also called pan-sharpened images, are always difficult to evaluate due to the lack of high-resolution multispectral (HRMS) images and the complexity of the fusion process. By taking spectral information of LRMS images and spatial structural information of Pan images as references, we extract the saturation map and luminance value as spectral features, and construct the optimal contrast map and structure similarity map as spatial features to compute the four indices between the original LRMS, Pan images, and the fused result: saturation similarity, luminance consistency, contrast similarity, and structure similarity to describe distortions from different aspects. Then, we feed the four indices into an extreme learning machine to train a nonlinear pooling strategy, and finally a multifeature and learning-based model is constructed for fusion image quality assessment. Comparisons with state-of-the-art image quality assessment metrics show that the proposed metric gains a much higher consistency with subjective opinions while needing no reference HRMS images. (C) 2020 Society of Photo-Optical Instrumentation Engineers (SPIE)

在图像超分辨率中,现有的卷积神经网络的方法较传统方法有明显的优势.然而,图像超分辨率的更深层卷积神经网络难以训练.同时,由于低分辨率图像包含丰富的低频特征信息,而这些信息在信道上与高频信息一起被平等地对待,因此阻碍了卷积神经网络的表达能力.文章提出了基于信道注意力机制的卷积神经网络,充分考虑信道之间的相互依赖性,自适应地重新调整信道的特征.实验表明,该算法较以前的算法可以实现更好的性能和更清晰的视觉效果.

Rcently, the research of single image super-resolution (SISR) based on deep learning has made great progress. However, most of the methods of super-resolution (SR) study use a simple chain structure to obtain higher super-resolution performance. Additionally, most methods do not fully utilize the hierarchical features generated in the middle of the network, thereby cannot achieve relatively high performance. In this paper, we propose a novel dual-branch feature fusion network (DBFFN) to address this two problems in image SR. The backbone of DBFFN is composed of multiple dual-branch feature fusion block (DBFFB) cascaded. The DBFFB has a dual-branch structure which mainly contains two parallel sub-networks, one extracts image fine features via densely connected convolution layers, the other one sub-network extracts image more contextual features by stacking dilated convolution layers. The features extracted from each hierarchical of the DBFFB are cascaded, and then feature fusion is performed. A skip connection is introduced to learn residual between input and output of the DBFFB. Especially, we use the deconvolution layer at the end of the network to enlarge the image. The results of the experiment are exciting, when the scale factor is 3, the performance of our network has 0.56dB, 0.6dB, and 0.79dB improvement on the set5, set14, and urban100 benchmark datasets compared with RDN.

为了提高医学图像的分辨率,提出一种基于内部样例的邻域回归超分辨率方法.首先,把输入的低分辨率图像当做高分辨率图像去构造基于自身实例的内部图像训练集,不再依赖外部训练集;然后,把高分辨率重建分成高频重建和低频重建,用邻域回归方法重建图像高频细节部分,用双三次插值方法重建低频部分;最后,用迭代组合的方法联合高频分量和低频分量来获得最终输出的高分辨率图像.实验结果表明,该方法性能优于传统的超分辨率重建算法,重建出的医学图像视觉效果更真实.

针对现有的邻域回归超分辨率算法仅通过一次方案从低分辨率空间映射到高分辨率空间,不能很好地表示复杂的映射关系,提出了一种两阶段邻域回归的图像超分辨率重建方法.在第一阶段,用传统的邻域回归方法重建初始高分辨率图像,然后把一个正则化项加入超分辨率重建模型中,以提高重建图像的精度.在第二阶段,以增强的方式训练残差字典和残差回归学习,降低映射误差.与别的邻域回归方法不同,采用了四个方向的Sobel算子代替一阶梯度和二阶梯度来提取低分辨率图像特征.实验结果表明,所提出的方法性能优于传统的超分辨率重建方法.

In the conventional single-image super-resolution algorithms, they assume that the sparse coefficients of the low-resolution patches and the corresponding high-resolution patches are the same. However, from our research, it is found that these coefficients are different most of the times. In this paper, the mapping relationship between the low-resolution coefficients and the high-resolution coefficients are learned based on neural networks. In this method, the low-resolution and high-resolution coefficients are first obtained from training images. Then, they are the inputs for a neural network to train this network. Finally, they are used in the reconstruction of the high-resolution image patches. Experimental results show that the proposed method has better performance than the original state-of-the-art algorithms.

The deep learning technique has been recently used in the image super-resolution. When the deep learning network is too deep, it is difficult to train this network to make it converge. Furthermore, the problem of gradient loss occurs for a very deep network, which makes the gradient at front layers go to zero, and thus it is impossible to train these layers. Current deep learning in image super-resolution has the following two shortcomings. One is that the features of each convolutional layer are not fully utilized. The other is that the low-resolution input features, containing rich low-frequency information, are treated equally for each channel, and not well utilized. Therefore, a DSAB (dense spatial attention block) is proposed as a basic structure for the network. The features extracted from a DSAB can be treated as a feature matrix. All the feature matrices extracted through the DSABs are concated in the row direction, and are then the inputs for a convolution layer with the kernel size of 3 by 3. The whole network of the proposed algorithm is then given and described in details. The implementation details of the algorithm is then described. The data set of DIV2K, which contains 900 high-resolution images, is used for training of the network. The test images are from Set5 and Set14, as in other papers. Experimental results show that the proposed network model is better than several other representative methods, and the edges of the reconstructed image are clearer and sharper.

为了提高单幅图像超分辨方法的性能,提出了新的基于投影矩阵的超分辨率方法.通过引入字典的互不一致性增强求解出的字典的表达能力;对分类后的低分辨率图像特征和相应的高分辨率图像特征的关系学习多个投影矩阵;并通过设置权重矩阵,增强邻近字典原子对当前图像块的表达能力,弱化较远原子的表达能力.在对投影矩阵进行正则化约束的前提下,利用字典原子和特征块之间的相关性以及特征块与其聚类中心的欧式距离关系,重构出拥有更加详细信息的高分辨率图像特征.实验结果显示,该方法的重构结果优于传统方法的重构结果.