Deep neural networks are commonly used in computer vision tasks, but they are vulnerable to adversarial samples, resulting in poor recognition accuracy. Although traditional algorithms that craft adversarial samples have been effective in attacking classification models, the attacking performance degrades when facing object detection models with more complex structures. To address this issue better, in this paper we first analyze the mechanism of multi-scale feature extraction of object detection models, and then by constructing the object feature-wise attention module and the perturbation extraction module, a novel adversarial sample generation algorithm for attacking detection models is proposed. Specifically, in the first module, based on the multi-scale feature map, we reduce the range of perturbation and improve the stealthiness of adversarial samples by computing the noise distribution in the object region. Then in the second module, we feed the noise distribution into the generative adversarial networks to generate adversarial perturbation with strong attack transferability. By doing so, the proposed approach possesses the ability to better confuse the judgment of detection models. Experiments carried out on the DroneVehicle dataset show that our method is computationally efficient and works well in attacking detection models measured by qualitative analysis and quantitative analysis.

Deep learning (DL) methodology contributes a lot to the development of hyperspectral image (HSI) analysis community. However, it also makes HSI analysis systems vulnerable to adversarial attacks. To this end, we propose a masked spatial-spectral autoencoder (MSSA) in this article under self-supervised learning theory, for enhancing the robustness of HSI analysis systems. First, a masked sequence attention learning (MSAL) module is conducted to promote the inherent robustness of HSI analysis systems along spectral channel. Then, we develop a graph convolutional network (GCN) with learnable graph structure to establish global pixel-wise combinations. In this way, the attack effect would be dispersed by all the related pixels among each combination, and a better defense performance is achievable in spatial aspect. Finally, to improve the defense transferability and address the problem of limited labeled samples, MSSA employs spectra reconstruction as a pretext task and fits the datasets in a self-supervised manner. Comprehensive experiments over three benchmarks verify the effectiveness of MSSA in comparison with the state-of-the-art hyperspectral classification methods and representative adversarial defense strategies.

Hyperspectral anomaly detection seeks to locate pixels in a scene exhibiting substantial spectral discrepancies from the surrounding background pixels, holding essential applications in both civilian and military domains. However, in real-world scenarios, the inherent properties of hyperspectral imaging, the irregular forms of anomalous targets and the lack of prior information pose significant challenges for anomaly detection methodologies. To address these issues, we first explore a novel reconstruction-based modeling approach for hyperspectral anomaly detection, offering a rational motivation for the modeling approach and a detailed exposition of its effective implementation. Furthermore, we propose a diagonal subsampling reconstruction network (DSRNet) for anomaly detection of hyperspectral data. Specifically, DSRNet consists of a paired training data generation algorithm using subsampling and a self-supervision training process enforced with a reconstruction consistency constraint. The training input is derived by a randomly diagonal averaging subsampler, where training pairs are derived from the same original hyperspectral data. Extensive experiments on four public datasets demonstrate the superiority of our DSRNet compared over several state-of-the-art baselines, with an average AUC score increase of 0.0156.

Deep neural networks (DNNs) have attained remarkable success in aerial detection tasks, yet they remain susceptible to adversarial samples, posing a significant challenge for their practical applications. While numerous transferable attacks have been proposed, they frequently overlook the essential balance between attack effectiveness and the feasibility of physical implementation. In this article, we concentrate our efforts on adversarial attacks against aerial detection, crafting transferable adversarial patches that can be implemented in the physical world. To this end, we introduce localized pattern corruptions, such as light spots and shadows, around the target during the training phase. These corruptions could pull the image distributions closer to the decision boundaries of the surrogate model, thereby enhancing the transferability of patches. In addition, we avoid directly optimizing adversarial patterns with traditional gradient-based techniques. Instead, we opt to update the weights of a specialized generator, which employs multilayer perceptrons (MLPs) as its core component for mapping purposes. To verify the effectiveness of our method, we conduct experiments across both the digital and physical domains. The results reveal that our approach outperforms state-of-the-art methods in terms of attack performance.

Convolutional neural networks (CNNs) have been demonstrated their powerful ability to extract discriminative features for hyperspectral image classification. However, general deep learning methods for CNNs ignore the influence of complex environmental factor which enlarges the intra-class variance and decreases the inter-class variance. This multiplies the difficulty to extract discriminative features. To overcome this problem, this work develops a novel deep intrinsic decomposition with adversarial learning, namely AdverDecom, for hyperspectral image classification to mitigate the negative impact of environmental factors on classification performance. First, we develop a generative network for hyperspectral image (HyperNet) to extract the environmental-related feature and category-related feature from the image. Then, a discriminative network is constructed to distinguish different environmental categories. Finally, a environmental and category joint learning loss is developed for adversarial learning to make the deep model learn discriminative features. Experiments are conducted over three commonly used real-world datasets and the comparison results show the superiority of the proposed method. The implementation of the proposed method and other compared methods could be accessed at https://github.com/shendu-sw/Adversarial Learning Intrinsic Decomposition for the sake of reproducibility.

Hyperspectral underwater target detection (HUTD) is a promising and challenging task in remote sensing image processing. Existing methods face significant challenges when adapting to nearshore environments, where cluttered backgrounds hinder the extraction of target signatures and exacerbate signal distortion. Hyperspectral unmixing (HU) demonstrates potential effectiveness for nearshore underwater target detection (UTD) by simultaneously extracting water background endmembers and separating target signals. To this end, this article investigates a novel nonlinear unmixing network for hyperspectral UTD, denoted as nonlinear unmixing network for hyperspectral-UTD (NUN-UTD), in which a well-designed autoencoder-based unmixing network is used to obtain the abundance map as the detection result. To address the weak underwater target signals, a target prior spectral preservation scheme is employed to guide the unmixing network in learning the accurate target abundance. Besides, to address the complexity of the nearshore environment, a pseudomixed data classification constraint is incorporated into the objective function to enhance the discriminative capability between the background and the target. Moreover, we adopt an additive postnonlinear model in the decoder to deal with the interactions between underwater spectra to account for the nonlinear effects between spectra of underwater substances. To validate the effectiveness of the proposed method, we constructed a hyperspectral dataset for nearshore UTD. Extensive experiments conducted on three real-world datasets and one simulated dataset demonstrate that our method achieves outstanding performance in HUTD.

Autonomous driving safety based on LiDAR perception is increasingly becoming a hot spot. Specifically, 3D adversarial examples always make the prediction results of deep neural network models unpredictable, which poses a major security risk to autonomous driving systems. However, the vulnerability of 3D neural network models to adversarial examples is less explored. At present, existing adversarial attack methods often obtain 3D adversarial examples by perturbing the entire point cloud, which requires a large number of perturbed points, i.e. requires a large perturbation budget. In this paper, we propose a Salient Region Adversarial attack method (SR-Adv) to generate adversarial point clouds, by perturbing fewer regions and fewer points. To our knowledge, we are the first to propose region-based attacks for 3D point clouds. First, the proposed SR-Adv employs game theory to extract salient regions of point clouds. This mechanism assigns a value to each region to measure its importance to the 3D neural network model prediction results and realizes the vulnerability analysis of the 3D model. Second, we propose a novel optimization-based gradient attack algorithm to achieve adversarial attacks on salient regions. We evaluate the proposed SR-Adv attack method on the synthetic datasets ModelNet40 and ShapeNetPart as well as the real-world dataset KITTI and NuScenes. Experimental results show that the proposed SR-Adv achieves a state-of-the-art attack success rate and better imperceptibility by perturbing fewer points on 3D point clouds.

Owing to the capacity of performing full-time target search, cross-modality vehicle re-identification (Re-ID) based on unmanned aerial vehicle (UAV) is gaining more attention in both video surveillance and public security. However, this promising and innovative research has not been studied sufficiently due to the data inadequacy issue. Meanwhile, the cross-modality discrepancy and orientation discrepancy challenges further aggravate the difficulty of this task. To this end, we pioneer a cross-modality vehicle Re-ID benchmark named UAV Cross-Modality Vehicle Re-ID (UCM-VeID), containing 753 identities with 16015 RGB and 13913 infrared images. Moreover, to meet cross-modality discrepancy and orientation discrepancy challenges, we present a hybrid weights decoupling network (HWDNet) to learn the shared discriminative orientation-invariant features. For the first challenge, we proposed a hybrid weights siamese network with a well-designed weight restrainer and its corresponding objective function to learn both modality-specific and modality shared information. In terms of the second challenge, three effective decoupling structures with two pretext tasks are investigated to learn orientation-invariant feature. Comprehensive experiments are carried out to validate the effectiveness of the proposed method. The dataset and codes will be released at https://github.com/moonstarL/UAV-CM-VeID.

The dissection of hyperspectral images into intrinsic components through hyperspectral intrinsic image decomposition (HIID) enhances the interpretability of hyperspectral data, providing a foundation for more accurate classification outcomes. However, the classification performance of HIID is constrained by the model's representational ability. To address this limitation, this study rethinks hyperspectral intrinsic image decomposition for classification tasks by introducing deep feature embedding. The proposed framework, HyperDID, incorporates the Environmental Feature Module (EFM) and Categorical Feature Module (CFM) to extract intrinsic features. Additionally, a Feature Discrimination Module (FDM) is introduced to separate environment-related and category-related features. Experimental results across three commonly used datasets validate the effectiveness of HyperDID in improving hyperspectral image classification performance. This novel approach holds promise for advancing the capabilities of hyperspectral image analysis by leveraging deep feature embedding principles. The implementation of the proposed method could be accessed soon at https://github.com/shendu-sw/HyperDID for the sake of reproducibility.

The existing hyperspectral underwater target detection algorithms are ineffective in adapting to nearshore environments, owing to the difficulty of extracting target signals from cluttered nearshore backgrounds combined with weak target signals. To this end, hyperspectral unmixing (HU) is well suited in this paper for nearshore underwater target detection (UTD) task to address these issues, by extracting background endmembers and separating out weak target signals simultaneously. Specifically, we propose a novel nonlinear unmixing network for nearshore underwater target detection (NUN-UTD). It comprises an unsymmetrical autoencoder and a target prior preservation module to adapt to the complex nearshore environment. Experiments demonstrate that NUN-UTD achieves the best detection performances and visual results compared with other state-of-the-art methods.

Active fire detection is essential for early warning of wildfires to help suppress and mitigate damage. This study presents an AF-Net model based on object-contextual representations (OCR) for active fire segmentation from very high-resolution (VHR) unmanned aerial vehicles (UAVs) remote sensing images. To efficiently detect heat anomalies in forests from large UAV scenes, we have to handle the class imbalance between small active fire pixels and large-area complex background information. Class imbalance affects the model optimization and makes the training process stuck at a local minimum. Our work aims to address this issue by improving the object-contextual feature representations associated with fire in three ways. First, we employ a grid-based sampling strategy by constraining sampling ranges and reducing background samples. It improves the proportion of foreground pixels from 5.6% to 7.9% and maintains at least one active fire pixel in each sample. Then, we simplify the OCR module to strengthen small object representations related to the active fire using a self-attention unit. The OCR module receives multi-scale pixel representations as input from the HRNet-W48 backbone. Lastly, the weighted binary cross-entropy loss and the Lovász hinge loss are combined to improve the detection accuracy by optimizing the foreground IoU. We evaluate the performance of the proposed AF-Net on one aerial active fire benchmark (FLAME dataset). The proposed framework improves the mIoU score from 78.17% (baseline UNet) to 91.14%.

Vision transformers (ViTs) have achieved great success in unmanned aerial vehicle (UAV) target detection tasks. However, little attention has been paid to the adversarial attack against ViT-based detectors, and the generated adversarial examples cannot take physical realizability and attack transferability into account at the same time. To overcome the limitation, we focus on transferable attacks toward ViT-based detectors in optical UAV-based remote sensing images and generate adversarial examples in the physical world. Concretely, we design unique perturbation patches deployed within and beyond the target object rather than requiring the patches to be aligned with image tokens. To narrow the gap between limited digital samples and complex physical scenarios, we conduct data augmentation on training images at global and local levels. In addition, we propose a novel transferable attack method named Jacobian matrix regularization (JMR), which consists of feature variance regularization (FVR) and attention weight regularization (AWR). Specifically, FVR calculates feature variances of different channels within specific layers and then sets the features as zeros for channels with top variances. AWR is achieved by masking the largest self-attention weights. We conduct extensive transferable experiments with typical detectors in both digital and physical UAV-based remote sensing scenarios. The results indicate that our method could achieve competitive transferability compared with state-of-the-art methods.

In recent years, transformers have shown great potential in hyperspectral image processing and have also been gradually applied in hyperspectral target detection (HTD). Nonetheless, applying a typical transformer to HTD remains challenging. The heavy computation burden of the multi-head self-attention (MSA) in transformers limits its efficient HTD, while the limited ability to extract local spectral features can reduce the discrimination of the learned spectral features. To further explore the potential of transformers for HTD, for balance of representation ability and computational efficiency, we propose a dual-branch Fourier-mixing transformer network for hyperspectral target detection (DBFTTD). First, this work explores a dual-branch Fourier-mixing transformer network. The transformer-style network replaces the MSA sublayer in the transformer with a Fourier-mixing sublayer, which shows advantages in improving computational efficiency and learning valuable spectral information effectively for HTD. Second, this work proposes learnable filter ensembles in the Fourier domain that are inspired by ensemble learning to improve detection performance. Third, a simple but efficient dropout strategy is proposed for data augmentation. Sufficient and balanced training samples are constructed for training the dual-branch network, and training samples for balanced learning can further improve detection performance. Experiments on four data sets indicate that our proposed detector is superior to the state-of-the-art detectors.

The research on attack transferability is of great importance as it can guide how to conduct an ad-versarial attack without knowing any information about target models. However, it remains challeng-ing for adversarial examples to maintain a good attack transferability performance, especially for the black-box attack implemented in the physical world. To enhance black-box transferability of physical at-tacks on object detectors, we present a novel adversarial learning method to produce adversarial patches by redistributing separable attention maps. Concretely, we first develop smoothed multilayer attention maps by introducing serial composite transformations, which could suppress model-specific noise on the one hand, and cover objects to be concealed at various resolutions on the other hand. Besides, our method resorts to a scalable mask to separate object attention from the background and adjust their distribution with a novel loss function. Extensive experiments show that our approach outper-forms state-of-the-art methods in both the digital space and the physical world. Our code is available at https://github.com/zhangyu13a/transPhyAtt .(c) 2023 Elsevier Ltd. All rights reserved.

Recently, deep learning methods have demonstrated their potentials in extracting spectral information for hyperspectral images and have been widely applied in hyperspectral target detection (HTD). However, prior deep learning methods, represented by the convolutional neural networks, mainly focus on the local information and representation, which cannot well capture the long-range dependence. Besides, limited target references cannot meet the need of massive labeled samples for the training process. This work develops a triplet spectralwise transformer-based target detector (TSTTD) to deal with these problems. First, this work explores a novel triplet spectralwise transformer network for the HTD task, and a data augmentation method is used to construct sufficient and balanced training samples for balanced learning. The proposed network shows advantages in learning local features from multiple adjacent bands and global features with long-range dependence. Second, for improving the separability between the targets and backgrounds, a novel intercategory separation and intracategory aggregation (ISIA) loss function is proposed, which joints the hard-negative-mining (HNM) triplet loss and the binary cross-entropy (BCE) loss. Third, the experimental results on six datasets show that our proposed method is effective in leading to excellent detection performance when compared with other state-of-the-art methods.

Deep Neural Networks have been successfully applied in hyperspectral image classification. However, most of prior works adopt general deep architectures while ignore the intrinsic structure of the hyperspectral image, such as the physical noise generation. This would make these deep models unable to generate discriminative features and provide impressive classification performance. To leverage such intrinsic information, this work develops a novel deep learning framework with the noise inclined module and denoise framework for hyperspectral image classification. First, we model the spectral signature of hyperspectral image with the physical noise model to describe the high intraclass variance of each class and great overlapping between different classes in the image. Then, a noise inclined module is developed to capture the physical noise within each object and a denoise framework is then followed to remove such noise from the object. Finally, the CNN with noise inclined module and the denoise framework is developed to obtain discriminative features and provides good classification performance of hyperspectral image. Experiments are conducted over two commonly used real-world datasets and the experimental results show the effectiveness of the proposed method. The implementation of the proposed method and other compared methods could be accessed at https://github.com/shendu-sw/noise-physical-framework.

Understanding and analyzing radar work modes play a key role in electronic support measure system. Many classifiers, for example those based on convolutional neural network (CNN) and recurrent neural network (RNN), are available for recognizing radar work modes as well as emitter types from their waveform parameters. However, the performance of these methods may suffer significantly when confronting different types of signal degradation, e.g., measurement error, lost pulse and spurious pulse. To tackle this issue, we in this paper develop a Bayesian attention belief network (BABNet) based on Bayesian neural networks in which the probability distribution over weights can help to enhance the model robustness for corrupted data. In particular, we adopt pre-trained CNN as the Bayesian inference prior. This not only accelerates the convergence speed, but also avoids the training process getting stuck in bad local minima. Meanwhile, instead of using RNNs which are difficult to be implemented in parallel, the combination of padding operation and attention module in the proposed BABNet enables CNN, as the backbone, to process sequential data with variable length. Extensive experiments are conducted to demonstrate the recognition capability and robustness of the BABNet in different environments.

Compared with existing vehicle reidentification (VeID) tasks conducted with datasets collected by fixed surveillance cameras, VeID for an unmanned aerial vehicle (UAV) is still under-explored and could be more challenging. Vehicles with the same color and type show extremely similar appearances from the UAV's perspective so that mining fine-grained characteristics becomes necessary. Recent works tend to extract distinguishing information by regional features and component features. The former requires input images to be aligned and the latter entails detailed annotations, both of which are difficult to meet in UAV application. To extract efficient fine-grained features and avoid tedious annotating work, this letter develops an unsupervised self-aligned network consisting of three branches. The network introduced a self-alignment module to convert the input images with variable orientations to a uniform orientation, which is implemented under the constraint of a triple loss function designed with spatial features. On this basis, spatial features, obtained by vertical and horizontal segmentation methods, and global features are integrated to improve the representation ability in embedded space. Extensive experiments are conducted on UAV-VeID dataset, and our method achieves the best performance compared with recent reidentification (ReID) works.

Deep learning with its rapid development and advancement has achieved unparalleled performance in many areas like computer vision as well as cognitive radio and signal recognition. However, the performance of most deep neural networks would suffer from degradation in the data mismatch scenario, e.g., the test dataset has a related but nonidentical distribution with the training dataset. Considering the noise corruption, a classifier’s accuracy might drop sharply when it is tested on a dataset with much lower signal-to-noise ratio compared to its training dataset. To address this dilemma, in this work, we propose an efficient denoising and classification network (DNCNet) for radar signals. The DNCNet consists of denoising and classification subnetworks. First, a radar signal detection and synthetic mechanism is designed to generate pairwise clean data and noisy data for the DNCNet to train its denoising subnetwork. Then, a two-phase training procedure is proposed to train the denoising subnetwork in the first phase and strengthen the mapping between the denoising results and perceptual representation in the second. Experiments on synthetic and benchmark datasets validate the excellent performance of the proposed DNCNet against state-of-the-art methods in terms of both signal restoration quality and classification accuracy.

Transferable adversarial attack is always in the spotlight since deep learning models have been demonstrated to be vulnerable to adversarial samples. However, existing physical attack methods do not pay enough attention on transferability to unseen models, thus leading to the poor performance of black-box attack.In this paper, we put forward a novel method of generating physically realizable adversarial camouflage to achieve transferable attack against detection models. More specifically, we first introduce multi-scale attention maps based on detection models to capture features of objects with various resolutions. Meanwhile, we adopt a sequence of composite transformations to obtain the averaged attention maps, which could curb model-specific noise in the attention and thus further boost transferability. Unlike the general visualization interpretation methods where model attention should be put on the foreground object as much as possible, we carry out attack on separable attention from the opposite perspective, i.e. suppressing attention of the foreground and enhancing that of the background. Consequently, transferable adversarial camouflage could be yielded efficiently with our novel attention-based loss function. Extensive comparison experiments verify the superiority of our method to state-of-the-art methods.