Algorithms based on deep learning, especially the attention mechanism, have been the preference for hyperspectral remote sensing image classification. Recently, a spectral similarity based spatial attention module (S3AM) was designed to solve the deficiency that the existing attention modules cannot take the center pixel into full account for the deduction of the spatial attention. However, two crucial processes, including the calculation and conversion of spectral similarity, of this module may be affected by its single policy, which is harmful to alleviate the spectral variability and the accurate spatial attention. In this article, an improved version called S3AM++ which integrates the differential strategy into the original attention module is proposed for hyperspectral image classification. The S3AM++ divides each neighborhood of the center pixel into the relevant pixel and the interfering pixel implicitly to make the individual optimization of the network for the two kinds of pixels possible. Benefit from this specific, the spectral variability can be relieved effectively for the similarity calculation and the relevant pixels and the interfering pixels can be preserved and restrained appropriately during the similarity conversion, respectively. Experimental results reveal that the spatial attention derived by the S3AM++ is more representative and the classification performance of the model adopting the new attention module is superior.
Object detection for remote sensing images (RSI) has made significant progress in recent years. However, existing supervised methods typically require a large amount of labeled data, whose acquisition, and particularly that of RSI, can be quite expensive. For natural images, many semi-supervised methods have been proposed to tackle the lack of labeled data, while few studies have focused on semi-supervised object detection for RSI. To address this problem, this paper proposes RS-PCL, which is a semi-supervised method applying Pseudo labeling and Consistency Learning to RSI. The RS-PCL method uses the teacher-student training scheme to generate high-quality pseudo labels for self-training. To deal with biased pseudo boxes, Noisy Pseudo box Assignment (NPA) is proposed to consider the teacher predictions during the label assignments of the unlabeled student. To ensure scale invariance, Scale-Invariant Learning (SIL) is proposed to carry out feature-consistency training on multi-scale images. This study conducted experiments using the DOTA-v1.0 dataset as labeled data and newly added images in the DOTA-v2.0 dataset (DOTA-v2.0 was constructed by adding extra images to DOTA-v1.0) as unlabeled data. The results reveal that there is obvious improvement over the supervised baseline and state-of-the-art semi-supervised methods, which demonstrates the effectiveness of the proposed method.
Considering that visually similar actions may be easily distinguished in textual description, it provides an opportunity to introduce textual description to assist action recognition. This paper proposes a novel action recognition method based on dense action captioning. Considering that an action may include multiple sub-actions with temporal relationship, this paper extends the thought of video captioning into actions. Multiple descriptions are generated from an action video sequence and each description corresponds to one sub-action sequence. In this paper, temporal constraints are added into a dense video captioning model for dense action captioning. With the generated descriptions, actions are recognized by a decision fusion strategy on both visual and textual presentation. The classification of visually similar actions can be refined based on textual classification of generated descriptions. The proposed method could be used in other action recognition models based on only visual representation. Experiments conducted on WorkoutUOW-18 and TAPOS datasets demonstrate the effectiveness of the proposed method on action captioning and classification.
Convolutional Neural Network is a popular algorithm for image and audio processing. However, it contains drawbacks of inefficient classifications with positional and angular differences. Therefore, the Capsule Network algorithm was used to enhance the classification of the type and usage of face masks. The study utilized 9872 images to process, using brightness, contrast, and horizontal flips for data augmentation. The results showed a slightly better performance in CapsNet vs. CNN ResNet-50. The model was improved by training the model in equal images per classification and getting the best epochs and batch size hyperparameters. It is further validated through video input of surroundings in areas of the Philippines and found issues such as misclassification of dark-colored masks, skin-toned masks, and far subjects. It is recommended to increase the dataset size, especially with misclassified cases, change the model's architecture, and try comparisons with other algorithms.
Object detection is an important research branch in the field of computer vision, and personnel detection has rich application scenarios and use values. Deep learning is widely used in the field of personnel detection, but the traditional convolutional neural network is complex and needs the support of high computing power GPU, which is difficult to deploy on embedded devices. At the same time, the lack of feature information caused by too small scale or occlusion is the main reason for the reduction of personnel detection accuracy. To solve these problems, this paper proposes a lightweight personnel detection network based on reinforcement feature learning; At the same time, convolution can be separated by the depth of channel mixing mechanism to further reduce network parameters; Secondly, this paper designs a hole convolution module to obtain more discriminative feature information, and uses the hole space pyramid pool structure and the attention mechanism with position information to carry out effective feature fusion, thus improving accuracy and reasoning speed. Experiments on multiple data sets and multiple hardware platforms show that the proposed algorithm is better than the original YOLOv4 micro-network in terms of accuracy, speed, model parameters and volume, and is more suitable for deployment in embedded devices with limited resources.
