Algorithms based on deep learning, especially the attention mechanism, have been the preference for hyperspectral remote sensing image classification. Recently, a spectral similarity based spatial attention module (S3AM) was designed to solve the deficiency that the existing attention modules cannot take the center pixel into full account for the deduction of the spatial attention. However, two crucial processes, including the calculation and conversion of spectral similarity, of this module may be affected by its single policy, which is harmful to alleviate the spectral variability and the accurate spatial attention. In this article, an improved version called S3AM++ which integrates the differential strategy into the original attention module is proposed for hyperspectral image classification. The S3AM++ divides each neighborhood of the center pixel into the relevant pixel and the interfering pixel implicitly to make the individual optimization of the network for the two kinds of pixels possible. Benefit from this specific, the spectral variability can be relieved effectively for the similarity calculation and the relevant pixels and the interfering pixels can be preserved and restrained appropriately during the similarity conversion, respectively. Experimental results reveal that the spatial attention derived by the S3AM++ is more representative and the classification performance of the model adopting the new attention module is superior.
Object detection for remote sensing images (RSI) has made significant progress in recent years. However, existing supervised methods typically require a large amount of labeled data, whose acquisition, and particularly that of RSI, can be quite expensive. For natural images, many semi-supervised methods have been proposed to tackle the lack of labeled data, while few studies have focused on semi-supervised object detection for RSI. To address this problem, this paper proposes RS-PCL, which is a semi-supervised method applying Pseudo labeling and Consistency Learning to RSI. The RS-PCL method uses the teacher-student training scheme to generate high-quality pseudo labels for self-training. To deal with biased pseudo boxes, Noisy Pseudo box Assignment (NPA) is proposed to consider the teacher predictions during the label assignments of the unlabeled student. To ensure scale invariance, Scale-Invariant Learning (SIL) is proposed to carry out feature-consistency training on multi-scale images. This study conducted experiments using the DOTA-v1.0 dataset as labeled data and newly added images in the DOTA-v2.0 dataset (DOTA-v2.0 was constructed by adding extra images to DOTA-v1.0) as unlabeled data. The results reveal that there is obvious improvement over the supervised baseline and state-of-the-art semi-supervised methods, which demonstrates the effectiveness of the proposed method.
Considering that visually similar actions may be easily distinguished in textual description, it provides an opportunity to introduce textual description to assist action recognition. This paper proposes a novel action recognition method based on dense action captioning. Considering that an action may include multiple sub-actions with temporal relationship, this paper extends the thought of video captioning into actions. Multiple descriptions are generated from an action video sequence and each description corresponds to one sub-action sequence. In this paper, temporal constraints are added into a dense video captioning model for dense action captioning. With the generated descriptions, actions are recognized by a decision fusion strategy on both visual and textual presentation. The classification of visually similar actions can be refined based on textual classification of generated descriptions. The proposed method could be used in other action recognition models based on only visual representation. Experiments conducted on WorkoutUOW-18 and TAPOS datasets demonstrate the effectiveness of the proposed method on action captioning and classification.
Convolutional Neural Network is a popular algorithm for image and audio processing. However, it contains drawbacks of inefficient classifications with positional and angular differences. Therefore, the Capsule Network algorithm was used to enhance the classification of the type and usage of face masks. The study utilized 9872 images to process, using brightness, contrast, and horizontal flips for data augmentation. The results showed a slightly better performance in CapsNet vs. CNN ResNet-50. The model was improved by training the model in equal images per classification and getting the best epochs and batch size hyperparameters. It is further validated through video input of surroundings in areas of the Philippines and found issues such as misclassification of dark-colored masks, skin-toned masks, and far subjects. It is recommended to increase the dataset size, especially with misclassified cases, change the model's architecture, and try comparisons with other algorithms.
Object detection is an important research branch in the field of computer vision, and personnel detection has rich application scenarios and use values. Deep learning is widely used in the field of personnel detection, but the traditional convolutional neural network is complex and needs the support of high computing power GPU, which is difficult to deploy on embedded devices. At the same time, the lack of feature information caused by too small scale or occlusion is the main reason for the reduction of personnel detection accuracy. To solve these problems, this paper proposes a lightweight personnel detection network based on reinforcement feature learning; At the same time, convolution can be separated by the depth of channel mixing mechanism to further reduce network parameters; Secondly, this paper designs a hole convolution module to obtain more discriminative feature information, and uses the hole space pyramid pool structure and the attention mechanism with position information to carry out effective feature fusion, thus improving accuracy and reasoning speed. Experiments on multiple data sets and multiple hardware platforms show that the proposed algorithm is better than the original YOLOv4 micro-network in terms of accuracy, speed, model parameters and volume, and is more suitable for deployment in embedded devices with limited resources.
Aiming at the challenges of dense targets, complex types, and low recognition rates of small targets, we propose an improved attention-based YOLOv4 for underwater small target detection. Initially, we cluster and use optimization to obtain an accurate size of the anchor. Followed by the attention mechanism “Squeeze and Excitation” and transformer block module to capture long-distance feature dependencies and increase the feature extraction ability of the algorithm. For the small target prediction, a cascade network is added to improve the IoU matching result of the small target. The experimental results show that the improved algorithm's mAP reaches 91.0%, while at the same time, the detection accuracy of small targets is higher, and false detection and missed detection are reduced.
In order to improve the accuracy of target tracking in complex environment, this paper studies a scheme that can achieve high-precision target tracking. This paper first analyzes the shortcomings of the existing particle filter tracking framework and the traditional discriminant model based on the classification idea, proposes a new image classifier based on the Quantitative Theory Ⅰ, and introduces it into the particle filter framework to achieve a target tracking method that integrates target detection and segmentation. Experimental results show that the proposed algorithm has better tracking accuracy and robustness than other algorithms when target deformation, occlusion, background interference and scale change occur in the tracking scene.
To improve the performance of semi-supervised image segmentation, it is important to effectively generate pseudo-labels from unlabeled images. However, the impact of pseudo-label confidence on segmentation performance is often overlooked. Low-confidence pseudo-labels can misguide the model and lead to overfitting, making it challenging to use them effectively. To address this issue, we propose a consistency constraint-based network that employs one encoder and three decoders () to generate distinct pseudo-labels. To assess the confidence of the generated pseudo-labels, we introduce a critic network that learns relevant features and effectively regularizes the confidence of -generated pseudo-labels. For evaluating the unlabeled images, we define a loss function that minimizes entropy, consisting of three sets of losses. We compare the performance of our model with two other semi-supervised segmentation algorithms using Dice, MAE, and F1 indicators. Our results demonstrate that the model outperforms the comparison models on all three metrics. In summary, our proposed consistency constraint-based network with a critic network and entropy-based loss function can effectively generate high-confidence pseudo-labels for semi-supervised image segmentation and improve the overall performance of the model.
Object detection for remote sensing images (RSI) has made significant progress in recent years. However, existing supervised methods typically require a large amount of labeled data, whose acquisition, and particularly that of RSI, can be quite expensive. For natural images, many semi-supervised methods have been proposed to tackle the lack of labeled data, while few studies have focused on semi-supervised object detection for RSI. To address this problem, this paper proposes RS-PCL, which is a semi-supervised method applying Pseudo labeling and Consistency Learning to RSI. The RS-PCL method uses the teacher-student training scheme to generate high-quality pseudo labels for self-training. To deal with biased pseudo boxes, Noisy Pseudo box Assignment (NPA) is proposed to consider the teacher predictions during the label assignments of the unlabeled student. To ensure scale invariance, Scale-Invariant Learning (SIL) is proposed to carry out feature-consistency training on multi-scale images. This study conducted experiments using the DOTA-v1.0 dataset as labeled data and newly added images in the DOTA-v2.0 dataset (DOTA-v2.0 was constructed by adding extra images to DOTA-v1.0) as unlabeled data. The results reveal that there is obvious improvement over the supervised baseline and state-of-the-art semi-supervised methods, which demonstrates the effectiveness of the proposed method.
Medical intelligent navigation systems have attracted increasing attention, and binocular optical tracking systems being an important component to ensure accurate tracking navigation. However, ghost markers may appear when there are multiple markers on the same plane, which can result in tracking errors and significantly impact the surgical process. In this paper, we propose a robust stereo matching algorithm based on spatial constraints under rectification. The algorithm is verified using a binocular vision system based on Xilinx Zynq-7020 platform of field-programmable gate array (FPGA), which meets the requirement of fast, real-time tracking of multiple instruments. The used matching algorithm has a tracking processing time of only 107 under four surgical instruments, which is 82.46% higher than the epipolar geometry method. Furthermore, compared with the traditional sequential matching method, this method improves the tracking failure rate by 6.46% for tracking four instruments, effectively avoiding false matching caused by the appearance of ghost markers.
Defect detection is one of the important research directions in the field of machine vision, and positive sample learning is one of the important algorithms. By focusing on the reconstruction of background region, positive sample learning has been widely used in many occasions. However, in the process of use, a large number of defect-free samples are often required as training data. Obviously, some application situations cannot meet such requirements. Therefore, this paper proposes a preprocessing algorithm based on the Hamming distance of Patch Histogram Coding feature screening algorithm (Hd-PHC) and neighborhood structure. By segmentation and screening of defective images, defect-free data can be obtained for training. In this way, the source problem of training data can be solved, and the network scale can be optimized to some extent to alleviate the problems caused by the differences between different backgrounds. On the simulation data set and the actual data set generated by AITEX data set, the Hd-PHC feature screening algorithm reached 99.67% accuracy, the screening rate reached 64.31%, and the average time was 120.27s. The simulation data set and the actual data set reached 99.96% and 99.93% accuracy, respectively, F1 was 0.8919 and 0.8225.
This paper presents the analysis of Image Pattern Detection using the Hopfield algorithm. Initially two simple letter patterns, L and T are used for mathematical and graphical illustration of pattern classification using Hopfield Algorithm. Mathematical analysis with simple and comprehensive elaboration helps reader to better understand and implement the algorithm in its applications. The analysis is further extended for the patterns L, T, C, U and Y. For each of the patterns, the analysis is done for matrix size of 3 × 3, 5 × 5, 10 × 10 and 28 × 28 with the noise ranging from 10% to 80%. The result of comparative analysis done for different patterns, matrix sizes, presence of noise for the algorithm presented in this paper shows that the convergence ratio decreases with the increase in noise percentage. Additionally, this paper explains the affect of Hebbian learning rule in the convergence ratio of patterns. Finally, Hopfield algorithm is applied for the classification of 20 Baht and 50 Baht Thai banknotes. With image processing in MATLAB and application of Hopfield algorithm, the classification of banknotes is successfully done in the presence of different noise levels.
Power nameplate information is essential for managing and maintaining power equipment. In response to issues such as low contrast and high text density in text detection for power nameplates. To address these issues, we propose a text detection algorithm for power nameplates based on the DBIR network (Differential Binarization with Inception-ResNet v2), which integrates multi-scale feature fusion and a Fusion Attention Module (FAM). The DBIR network can learn features of different scales and use the FAM module to fuse deep and shallow features, thus capturing more spatial location information of the text and improving text localization accuracy. Given the current absence of publicly available power nameplate datasets, we propose a hybrid dataset that combines power nameplates with natural scenes. This approach effectively simulates power scenarios and enhances the generalization capability of the model. We evaluated the model on several datasets, including the power nameplate dataset, and obtained promising results. Specifically, the DBIR network achieved a detection accuracy of 87.1%, a recall of 83.2%, and an F-measure of 85.1%.
Detecting densely arranged objects is challenging due to the lack of generic definitions and the feature coupling between nearby objects. This paper proposes mathematical definitions of the instance-level, image-level, and dataset-level object density by information theory, called Density Index (DI). The DI shows a high consistency with human perception, serving as a powerful guide for aerial object detection, including data assessment and detector customization. Under the guidance of the DI, we design a DeDet to enhance the detector's performance in detecting densely arranged objects. DeDet pursues accurate location for densely arranged objects by the Density-aware Label Assignment (DLA) and Density-aware Feature Extraction (DFE), conquering the heuristic that the sample assignment and feature extraction are performed independently for each object. Experiments on the DOTA-v1.0 and DOTA-v2.0 show that DeDet can bring a significant improvement to the baseline detector.
Implicit neural representation (INR), sometimes also referred to coordinate-based representation or fitting, has gained the state-of-the-art performance in numerous research fields including computer vision and computer graphics due to the powerful continuous representation ability. Recent researches indicate that Fourier embedding is critical for INR to fit realistic images with high-frequency details. However, is Fourier embedding all we need for high-frequency coordinate (image) fitting? In this paper, we revisit the problem of coordinate fitting from a novel perspective of distribution mapping. Fourier embedding, as a preprocessing step of coordinate fitting, essentially performs the operation of mapping the uniform coordinate distribution to a normal distribution, and makes the learning of mapping function between two similar smooth distribution become easier. However, the number of discrete Fourier basis function affects the fitting performance dramatically and cannot be determined automatically. Based on the above analysis, a simple yet efficient INR coordinate fitting method is proposed in this paper, which demonstrates that the Fourier embedding is not the only way to improve INR. The proposed method only adds an element-wise layer normalization (ELN) module to the vanilla multi-layer perception (MLP) with ReLU activation. Experimental results on public database demonstrate that the proposed method outperforms the state-of-the-art methods using Fourier embedding.
Chaotic systems have various advantages, such as unpredictability, non-periodicity, and so on. The algorithm for encrypting digital images using chaotic systems has a large key space, sensitive initial values, and fast encryption speed. Therefore, in this paper, we propose a reversible data hiding in encrypted images (RDHEI) combined with a chaotic system. The algorithm introduced the SHA-256 method to generate the initial value of the chaotic system and the key of RDHEI by manipulating the plaintext image. Moreover, the chaotic system is used to further encrypt the encrypted image after the data hiding in the RDHEI process. The improvement enhances the sensitivity and security of the key and strengthens the image encryption. The experimental results evaluated by histogram, information entropy, correlation of neighboring pixel points, and other indicators show that the proposed algorithm has comparable performance with state-of-the-art and has more intense encryption.
Structured light-based 3D reconstruction technology has emerged as a focal point of research in the field of 3D modeling due to its exceptional speed and accuracy. However, occlusion often occurs due to the limited field of view between the camera and the projector, as well as the complex surface of the object. Consequently, defects and holes are generated in the reconstructed model, impairing the model's reconstruction quality. In this study, we propose a hole repair method based on a triangular mesh that yields superior repaired effects. Our method effectively addresses the deficiencies in point cloud caused by occlusions. The difference in model accuracy between the repaired point cloud and the original point cloud is 3.618%.
We consider the problem of blood pressure estimation. Given a certain physiological signal of the human body, the goal is to estimate the blood pressure. Existing works in this area only use physiological signals and do not utilize personal information such as gender and age. In this paper, we propose a method for blood pressure estimation that uses the Electrocardiogram (ECG), Photoplethysmography (PPG) and personal information to estimate the current blood pressure of the human body. Firstly, we use the slope sum function (SSF) to detect the peaks and valleys of the PPG signal, which lays the foundation for the signal segmentation later. Then, we use two sub-networks to extract features of physiological signals and personal information, respectively. The physiological signal branch uses a residual network with large convolutional kernel attention. Then, we set up a shared space for parameter sharing and subsequently estimated the systolic blood pressure (SBP) and diastolic blood pressure (DBP) values separately using a multi-task structure. Finally, on the MIMIC-III dataset of 795 patients, our method obtained a mean error (ME) ± standard deviation (STD) of 0.15 ± 5.05 mmHg for systolic blood pressure and -0.03 ± 3.31 mmHg for diastolic blood pressure, respectively, meeting international standards.
Since the target's shadow in a synthetic aperture radar (SAR) image can provide significant features, it is becoming a crucial discriminative feature for interpreting SAR images. In this paper, we propose a new segmentation method based on simple linear iterative clustering (SLIC) superpixel segmentation and merging to extract the targets' shadow regions in SAR images. The process is divided into four stages. Firstly, the original SAR image is preprocessed using logarithmic transform and anisotropic diffusion filtering. Secondly, the preprocessed image is segmented with the SLIC method. Then, we propose a technique based on the shadow superpixel marker to merge superpixels to obtain the shadow region. Finally, the merged edge of the shadow region is smoothed using the morphological closing operation to get the final shadow detection result. The experimental results based on the public Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset demonstrate the feasibility of the proposed method.
An improved method based on Radon transform is proposed to detect the wake in an infrared image of submarine. First, the original infrared image is preprocessed by median filtering, which eliminates the negative influence of double X-shaped bright lines in Radon transform domain on wake feature detection. Then, after the edge detection, the effective integration length of the Radon transform is increased by the segment linker, and the Radon transform coefficients are thresholded to extract all possible peak points; Subsequently, cluster analysis is performed on the peak points, making the detected linear clusters scattered; Finally, the direction gradient algorithm is used to search the starting point and ending point of the detected straight wake in the original image to determine the direction of the wake extension. This method can eliminate the false linear features to the maximum extent, reducing the false alarm rate and improving the stability of submarine wake detection.
