Multiloss Adversarial Attacks for Multimodal Remote Sensing Image Classification

The challenge of classifying multimodal remote sensing images has garnered significant interest in light of the growing diversity of remote sensing image data modalities. Current studies primarily concentrate on increasing the classification task’s accuracy by improving the fusion strategy or incorporating auxiliary architectures. However, there is currently a lack of research in the area of adversarial attack for multimodal remote sensing image classification models as compared to the study of unimodal classification models. To overcome this issue, our research firstly investigates the adversarial robustness of multimodal remote sensing image classification under different fusion strategies, which adopts the improved classical adversarial attack methods to test the adversarial robustness of multimodal remote sensing image classification model architectures with three different fusion strategies; Then a new multimodal adversarial attack method is proposed for the multimodal model, which adopts balanced perturbation loss and cooperative adversarial loss, in which the balanced perturbation loss is used to balance the level of perturbation of different modalities, and the cooperative adversarial loss is used to reduce the conflict of different modality perturbations on the classification result. By combining balanced perturbation loss and cooperative adversarial loss to attack multimodal models, the cooperation between modalities is continuously optimized.Finally, the study demonstrates the weak adversarial robustness of the multimodal remote sensing image classification model, which robustness is easily influenced by the fusion strategies and the attack methods. Additionally, a better attack effect is obtained by the multimodal multiloss cooperative adversarial attack method proposed in this paper.