Small object detection (SOD) is a key problem in the field of computer vision. Although many existing methods have partially addressed this problem, challenges remain in optimizing SOD and balancing detection accuracy and efficiency. To address these issues, we propose a novel real-time object detector called multi-scale feature reconstruction detection transformer (MSFR-DETR). Specifically, we propose an efficient MSFR module, which is mainly used to mitigate the problem of small object feature loss in deep networks, aiming to improve the model’s perception of multi-scale features, while enhancing small object representations. Subsequently, we introduce the selective boundary aggregation (SBA) module to redesign the Neck network for multi-scale fusion, which utilizes a bidirectional fusion mechanism to solve the problem of mismatch between the spatial information of shallow features and the semantic information of deeper features. Finally, we propose the high-resolution feature fusion mechanism (HRFM) to significantly improve the model’s ability to perceive small objects without notably increasing the computational cost. Extensive experiments on three major small object benchmarks, including VisDrone, AI-TOD, and DOTA, show that MSFR-DETR achieves a good balance of accuracy and efficiency, significantly outperforming other state-of-the-art real-time detectors in small object detection.

Each pixel intensity value in a depth image represents the depth information in the scene. However, during the generation of depth images, some important information may be lost in the form of pieces, which can seriously impact subsequent applications, such as computer vision. Existing methods attempt to apply traditional image restoration methods for filling the lost areas in depth images have proven ineffective in inpainting the structural information in the scene. In this paper, we propose a depth image inpainting method based on pixel-level non-local method. In order to effectively inpaint the depth images, we add 0.7 density salt-and-pepper noise to the original depth image in advance, we then implement image block-matching on the noise-added depth image to obtain some similar image block groups, and then scan each image block to form a column vector. We further stack all vectors to construct a two-dimensional matrix. Finally, we implement row-matching on the constructed matrix to obtain many similar pixel groups. The separable Haar transform is implemented on each similar pixel groups, hard-thresholding on transformed coefficients can effectively remove the added noise and bring the original neighbor pixels in the depth image into the information lost area, thus, the lost information can be effectively inpainted. Inverse Haar transform, row-aggregation and block-aggregation are successfully implemented to complete the depth image inpainting. Experimental results show that the proposed method can achieve higher PSNR values than existing methods on public datasets, and the subjective visual quality of the inpainted images is also satisfactory.

Rail surface defect images are often affected by noise, which degrades image quality and hinders the detection of helpful information. To improve the accuracy of rail surface defect detection, denoising the rail surface defect image has become essential work. The BM3D algorithm is an excellent image-denoising method, and it still shows a strong denoising performance in the image-denoising of rail surface defects. However, the BM3D algorithm ignores the relationship between the internal morphological components of the reference block and the search neighborhood when matching similar blocks, resulting in insufficient block-matching accuracy, which may affect the performance of the denoising methods. To alleviate this problem, an improved BM3D image-denoising algorithm based on an adaptive search neighborhood is proposed. Different from the BM3D block matching process, we add the step of analyzing the morphological components of the reference block and adaptively assign the optimal search neighborhood range according to the morphological components of the reference block. The experimental results based on the dataset show that compared with the baseline method BM3D and other state -of -the -art methods, we have achieved higher PSNR and SSIM values and improved the accuracy of defect detection. Our study has important practical implications. Through noise reduction processing, we successfully eliminated noise interference in railway defect images, thereby improving the visibility of defective areas and helping to improve the safety and efficiency of railway transportation.

The primary purpose of image denoising is to remove noise while better maintaining important detail information in the image. Block-matching and 3D flite-ing (BM3D) is a state-of-the-art image denoising method. However, the intense isolated noise in the high-level noise situations is difficult to be removed due to the unique co-efficient hard-thresholding in the original BM3D method. In this paper, we propose an improved BM3D algorithm using bi-hard thresholding, i.e., coefficient hard-thresholding and structural hard thresholding. The whole procedure consists of two significant steps: essential estimation and final estimation, and each step is divided into three minor steps: block-matching, collaborative filtering, and aggregation. In the basic estimation stage, we further enhance the sparsity of the transformed coefficients by reducing the coefficient hard-thresholding and adding structural hard-thresholding on the third dimension instead of the unique coefficient hard-thresholding in the original BM3D method. Finally, we compare the improved algorithm with the original BM3D method on the public dataset in high-level noise situations. Experimental results show that the proposed method improves the denoising performance.

This paper uses Python and TensorFlow framework. Using low power computer Raspberry as model computing platform and sorting processing platform, the damaged parts identification system based on computer vision technology and deep learning technology is realized. Multi-scale interactive iterative learning algorithm is studied. This method encodes the dense features of each pixel, and realizes the effective capture of key information such as texture, color and edge of each pixel through multi-scale cyclic learning. Edge extraction of objects is carried out to complete the final semantic understanding. CNN algorithm mainly extracts dense features from original images, constructs convolutional networks, and obtains prior information through training large-scale samples. The system adopts the advanced mobile CNN mode, which has the characteristics of simple structure, reliable and easy maintenance, and can identify and classify the damaged parts quickly and efficiently.

Real-world face super-resolution (SR) is a highly ill-posed image restoration task. The fully-cycled Cycle-GAN architecture is widely employed to achieve promising performance on face SR, but is prone to produce artifacts upon challenging cases in real-world scenarios, since joint participation in the same degradation branch will impact final performance due to huge domain gap between real-world and synthetic LR ones obtained by generators. To better exploit the powerful generative capability of GAN for real-world face SR, in this paper, we establish two independent degradation branches in the forward and backward cycle-consistent reconstruction processes, respectively, while the two processes share the same restoration branch. Our Semi-Cycled Generative Adversarial Networks (SCGAN) is able to alleviate the adverse effects of the domain gap between the real-world LR face images and the synthetic LR ones, and to achieve accurate and robust face SR performance by the shared restoration branch regularized by both the forward and backward cycle-consistent learning processes. Experiments on two synthetic and two real-world datasets demonstrate that, our SCGAN outperforms the state-of-the-art methods on recovering the face structures/details and quantitative metrics for real-world face SR. The code will be publicly released at https://github.com/HaoHou-98/SCGAN.

Compressed sensing magnetic resonance imaging (CS-MRI) has made great progress in speeding up MRI imaging. The existing non-local self-similarity (NSS) prior based CS-MRI models mainly take similar image patches as the processing objects, this patch-level non-local sparse representation method can not make full use of the self-similarity among pixels in the image, so it can not recover the weak edge information in the undersampled MRI image well and there will still be some artifacts. In this paper, a pixel-level non-local method based compressed sensing undersampled MRI image reconstruction method is introduced. First, zero filling is performed on the undersampled k-space data to obtain a full-size 2D signal, and IFFT is performed to obtain a preliminary reconstructed MRI image. Block-matching and row-matching are successively performed on the reconstructed image in turn to obtain similar pixel groups, so as to establish a better sparse representation under the non-local self-similarity (NSS) prior. The separable Haar transform is performed on similar pixel groups, and the hard threshold of the transform coefficients and Wiener filtering can effectively remove the artifacts introduced in the undersampled reconstructed MRI images. The proposed pixel-level non-local iterative thinning model based on compressed sensing theory can ensure the removal of artifacts and better restore the details in the image. The qualitative and quantitative results under different undersampling modes and undersampling rates prove the advantages of the proposed method in subjective visual quality and objective evaluation (peak signal to noise ratio and structure similarity index). The performance of this method is not only superior to the existing traditional CS-MRI methods, but also competitive with the existing deep neural network (DNN) based models. The code will be released at https://github.com/HaoHou-98/PNCS.

With the wide application of deep learning in the field of agriculture, how to quickly and accurately identify and process structured and unstructured monitoring data to support intelligent decision-making has become an important research direction in the field of intelligent agriculture. In this paper, the multiple diseases of wild and introduced flowers such as canker, gray mold, brown spot and leaf blight were taken as the research objects, and the disease samples were collected according to the growth cycle of flowers, so as to construct a data set of flower diseases and insect pests under natural scenes. Based on Faster-RCNN framework, a three-stage pest and disease detection model PD-IFRCNN is proposed, which integrates transfer learning and data enhancement technology. In order to ensure the effectiveness of the model, this paper verifies the influence of the disequilibrium of the disease category of the self-built data set on the cost of wrong classification from two different perspectives: category balancing and label balancing, and makes a comparative analysis with the SSD model that takes VGG16 as the feature to extract the network. Experiments show that the proposed method can adapt to the detection of diseases with different scales and multiple leaves in natural scenes, and has high recognition accuracy. The proposed method not only breaks the traditional manual detection method of plant diseases, but also provides a new research idea for the recognition of plant diseases. It fundamentally solves the problems such as the lack of pertinence of traditional disease control and the lack of timely control caused by the inconveniences of manual detection due to bad weather and poor operating environment.

Existing studies on infrared and visible image fusion generally need to first decompose the fusion image and then extract features beneficial to image fusion from the decomposition results for better fusion results. However, they usually focus on decomposing single-modality images though various techniques such as latent low-rank representation (LatLRR), without considering the spatial consistency of both infrared and visible image modalities, which may fail to effectively capture inherent image features. In this paper, we propose a sparse consistency constrained latent low-rank representation (SccLatLRR) method to fuse infrared and visible images. Firstly, infrared and visible images are performed low-rank representation decomposition simultaneously as the inputs of different tasks. In the decomposition process, the L2,1 norm is used to constrain the rank to maintain sparse consistency, and the low-rank consensus representations of infrared and visible images are obtained simultaneously. Secondly, the basic information is further mined respectively using the Very Deep Convolutional Network (VGG) network and non-subsampled contourlet transforms (NSCT) method to extract more effective fusion features. Finally, different fusion strategies are used for the base part and the salient part. An effective iterative algorithm optimization model is proposed. Experimental results on the public dataset TNO suggest the effectiveness of our method compared with several state-of-the-art methods.

Recently, imagesmoothing has gained increasing attention due to its prerequisite role in other image processing tasks, e.g., image enhancement and editing. However, the evaluation of image smoothing algorithms is usually performed by subjective observation on images without corresponding ground truths. To promote the development of image smoothing algorithms, in this paper, we construct a novel Nankai Smoothing (NKS) dataset containing 200 images blended by versatile structure images and natural textures. The structure images are inherently smooth and naturally taken as ground truths. On our NKS dataset, we comprehensively evaluate 14 popular image smoothing algorithms. Moreover, we propose a Pixel-level Non-Local Smoothing (PNLS) method to well preserve the structure of the smoothed images, by exploiting the pixel-level non-local self-similarity prior of natural images. Extensive experiments on several benchmark datasets demonstrate that our PNLS outperforms previous algorithms on the image smoothing task. Ablation studies also reveal the work mechanism of our PNLS on image smoothing. To further show its effectiveness, we apply our PNLS on several applications such as semantic region smoothing, detail/edge enhancement, and image abstraction. The dataset and code are available at https://github.com/zal0302/PNLS .

Retinex model has been applied to low-light image enhancement in many existing methods. More appropriate decomposition of a low-light image can help achieve better image enhancement. In this paper, we propose a new pixel-level non-local Haar transform based illumination and reflectance decomposition method (NLHD). The unique low-frequency coefficient of Haar transform on each similar pixel group is used to reconstruct the illumination component, and the rest of all high-frequency coefficients are employed to reconstruct the reflectance component. The complete similarity of pixels in a matched similar pixel group and the simple separable Haar transform help to obtain more appropriate image decomposition; thus, the image is hardly sharpened in the image brightness enhancement procedure. The exponential transform and logarithmic transform are respectively implemented on the illumination component. Then a minimum fusion strategy on the results of these two transforms is utilized to achieve more natural illumination component enhancement. It can alleviate the mosaic artifacts produced in the darker regions by the exponential transform with a gamma value less than 1 and reduce information loss caused by excessive enhancement of the brighter regions due to the logarithmic transform. Finally, the Retinex model is applied to the enhanced illumination and reflectance to achieve image enhancement. We also develop a local noise level estimation based noise suppression method and a non-local saturation reduction based color deviation correction method. These two methods can respectively attenuate noise or color deviation usually presented in the enhanced results of the extremely dark low-light images. Experiments on benchmark datasets show that the proposed method can achieve better low-light image enhancement results on subjective and objective evaluations than most existing methods.

Security in human-robot interaction is the focus of research in this field. Rapid detection of abnormal events that may cause danger in the interaction process can effectively reduce the probability of occurrence of danger. In general anomaly detection methods, 2D or 3D convolutional autoencoders are widely used for anomaly detection. Among them, 2D convolutional autoencoders are with good real-time performance and lower detection accuracy, while 3D convolutional autoencoders are with higher detection accuracy and insufficient real-time performance. In order to ensure realtime performance and obtain higher accuracy, an end-to-end asymmetric convolutional autoencoder network (ACANet) using both 2D and 3D convolutions is designed. Specifically, 3D convolution is used to build the encoder to learn comprehensive information in continuous input frames, and 2D convolution is used to build the decoder to model the information fast, a dimensional alignment module is constructed to connect the encoder and the decoder while avoiding a large number of calculations in the latent space of the 3D features output by the encoder, and the skip connections module is used to obtain accurate predictions. Anomaly detection can then be completed by evaluating the differences between results predicted by the ACANet and real frames. The experimental results show that our method achieves competitive accuracy on mainstream datasets and at the same time obtains the fastest speed. Compared with mainstream methods, this method is more suitable for anomaly detection tasks in human-robot interaction.

With the rapid development of image processing techniques, image smoothing has gained increasing attention due to its important role in other image processing tasks, e.g., image editing and enhancement. However, the evaluation of image smoothing methods is subjectively performed on datasets without proper ground truth images. Therefore, an image smoothing benchmark with reasonable ground-truths is essential to prosper the image smoothing community. In this paper, we construct a new Nankai Smoothing (NKS) dataset containing 200 versatile images blended by natural textures and structure images. The structure images are inherent smooth and can be safely taken as ground truths. On our NKS dataset, we comprehensively evaluate 14 popular image smoothing algorithms. Moreover, we propose a novel Pixel-level Non-Local Smoothing (PNLS) method, exploiting better the non-local self-similarity of natural images to well preserve the structure of the smoothed images. Extensive experiments on several benchmark datasets demonstrate that our PNLS is very effective on the image smoothing task. Comprehensive ablation studies also reveal the work mechanism of our PNLS on image smoothing. To further show its effectiveness, we apply the proposed PNLS on semantic region smoothing, detail/edge enhancement, and image abstraction. The benchmark and code will be publicly released.

Retinex theory is developed mainly to decompose an image into the illumination and reflectance components by analyzing local image derivatives. In this theory, larger derivatives are attributed to the changes in reflectance, while smaller derivatives are emerged in the smooth illumination. In this paper, we utilize exponentiated local derivatives (with an exponent $\gamma $ ) of an observed image to generate its structure map and texture map. The structure map is produced by been amplified with $\gamma >1$ , while the texture map is generated by been shrank with $\gamma < 1$ . To this end, we design exponential filters for the local derivatives, and present their capability on extracting accurate structure and texture maps, influenced by the choices of exponents $\gamma $ . The extracted structure and texture maps are employed to regularize the illumination and reflectance components in Retinex decomposition. A novel Structure and Texture Aware Retinex (STAR) model is further proposed for illumination and reflectance decomposition of a single image. We solve the STAR model by an alternating optimization algorithm. Each sub-problem is transformed into a vectorized least squares regression, with closed-form solutions. Comprehensive experiments on commonly tested datasets demonstrate that, the proposed STAR model produce better quantitative and qualitative performance than previous competing methods, on illumination and reflectance decomposition, low-light image enhancement, and color correction. The code is publicly available at https://github.com/csjunxu/STAR.

We have investigated the magnetic properties and magnetocaloric effects in polycrystalline GdCoO3 with a low-spin state of Co3+ ion by magnetization and heat capacity measurements. GdCoO3 shows antiferromagnetic ordering due to the Gd3+ magnetic moments below Néel temperature TN = 3.1 K. This compound exhibits a giant magnetocaloric effect around TN. The maximum value of isothermal magnetic entropy change is 39.1 J/kg K, the peak value of adiabatic temperature change is 19.1 K, and the refrigerant capacity is 278 J/kg for a field change of 0–70 kOe around TN. The large magnetocaloric effect is related to the half-filled 4f electronic state of Gd3+ ions. The giant magnetocaloric effect as well as no hysteresis suggest that GdCoO3 could be a potential refrigerant in low-temperature magnetic refrigeration technology.

Non-local self similarity (NSS) is a powerful prior of natural images for image denoising. Most of existing denoising methods employ similar patches, which is a patch-level NSS prior. In this paper, we take one step forward by introducing a pixel-level NSS prior, i.e., searching similar pixels across a non-local region. This is motivated by the fact that finding closely similar pixels is more feasible than similar patches in natural images, which can be used to enhance image denoising performance. With the introduced pixel-level NSS prior, we propose an accurate noise level estimation method, and then develop a blind image denoising method based on the lifting Haar transform and Wiener filtering techniques. Experiments on benchmark datasets demonstrate that, the proposed method achieves much better performance than previous non-deep methods, and is still competitive with existing state-of-the-art deep learning based methods on real-world image denoising. The code is publicly available at https://github.com/njusthyk1972/NLH.

In this paper, we develop a novel linear singularity representation method using spatial K-neighbor block-extraction and Haar transform (BEH). Block-extraction provides a group of image blocks with similar (generally smooth) backgrounds but different image edge locations. An interblock Haar transform is then used to represent these differences, thus achieving a linear singularity representation. Next, we magnify the weak detailed coefficients of BEH to allow for image enhancement. Experimental results show that the proposed method achieves better image enhancement, compared to block-matching and 3D filtering (BM3D), nonsubsampled contourlet transform (NSCT), and guided image filtering.

Various lighting conditions for face image seriously affect the accurate rate of face recognition. This paper proposes a kind of nonlocal illumination normalization method, which compares image block mean with the mean of the whole image, gain or punish the upper-left corner pixel values of the image block according to the mean of the image block by using a little gain or punishment factor, use different block size and various gain or punishment factors to remove the illumination by the multi-step iteration; A lot of noise will be generated after the illumination normalization in the original darker area, so using BM3D to denoise images will achieve the ideal final result. The experimental results show that the obtained images are more natural and can better preserve the image details than most existing illumination normalization methods, thus it can achieve higher face recognition rate than the existing methods.

Image salient region detection is widely used in many fields, such as automatic target detection, image retrieval, object recognition and image segmentation. Although there are many methods related to image salient region detection, improving its accuracy is still one of the hot research areas in image processing. In this paper, we propose a novel salient detection algorithm based on nonlocal filtering, which is mainly used in image denoising. The proposed algorithm stacks several similar blocks to form a three-dimensional matrix by image block matching, in which the filtered image by nonlocal filtering can better preserve image detail information than traditional Gaussian filtering. We calculate the Euclidean distance between the mean of the original image and the filtered image as the saliency map. The experimental results show that the proposed algorithm can accurately obtain the saliency map, especially, the edge of the saliency map image is much better than many existing algorithms.

Convolutional neural network is an important technique in machine learning, pattern recognition and image processing. In order to reduce the computational burden and extend the classical LeNet-5 model to the field of saliency detection, we propose a simple and novel computing model based on LeNet-5 network. In the proposed model, hue, saturation and intensity are utilized to extract depth cues, and then we integrate depth cues and color volume to saliency detection following the basic structure of the feature integration theory. Experimental results show that the proposed computing model outperforms some existing state-of-the-art methods on MSRA1000 and ECSSD datasets.