The use of supervised deep learning techniques to monitor the health of forests through the analysis of satellite data is rapidly increasing. However, the key challenge with supervised deep learning techniques is that they require big volumes of accurate, error-free annotations to boost accurate model development. Although, several Earth observation satellite datasets are today available free of charge, the fieldwork for collecting their accurate annotations is time-consuming and costly. On the other hand, the emerging Data-Centric Artificial Intelligence (DCAI) paradigm promises to mitigate this issue saving time and money, while gaining accuracy promoting the reuse of foundation semantic segmentation models to specific Earth observation problems. In particular, this study addresses the task of mapping bark beetle outbreaks causing forest tree dieback through the lens of the model reuse in the DCAI paradigm. To this aim, we explore the performance of fine-tuning as a learning strategy to reuse a pre-trained, sophisticated semantic segmentation model developed for land cover segmentation with a big amount of accurately annotated multi-temporal Sentinel-2 data. We assess the effectiveness of the model reuse approach in two case studies regarding forest scenes that were annotated with bark beetle outbreaks observed in October 2028 in the Northeast of France and September 2020 in the Czech Republic.

Europe is one of the most forest-rich regions in the world, with forestry mainly based on the management of coniferous trees. However, the spruce forest ecosystem is vulnerable to several disturbance agents. In particular, bark beetle outbreaks have been the scourge of spruce trees in the last decade, and they are expected to further intensify due to climate change, with significant adverse effects on forest ecosystems. Hence, the monitoring of forest disturbances caused by the rapidly escalating bark beetle outbreaks represents a significant ecological and forestry challenge. This monitoring is traditionally performed by foresters during field surveys. On the other hand, open Sentinel-2 images, available with the Copernicus mission and processed with sophisticated deep learning techniques, have been recently established as an alternative to field surveys performed by foresters to monitor various environmental phenomena such as bark beetle outbreaks. In particular, several deep learning approaches have been recently proposed to map bark beetle tree dieback using Sentinel-2 images of forests. However, the current effectiveness of deep learning approaches, as a means to monitor bark beetle outbreaks in Sentinel-2 data is often limited by the reduced availability of ground truth information to supervise semantic segmentation models for this specific downstream task. In this study, we propose ULISSE, a deep learning semantic segmentation methodology for mapping forest tree dieback caused by bark beetle outbreak disturbances using Sentinel-2 image time series. ULISSE leverages a U-Net-like architecture with a multi-temporal encoder specifically designed to handle Sentinel-2 image time series. The framework integrates vision encoders pretrained on a large volume of Sentinel-2 images for land-cover classification. To capitalize on the representational capabilities of these pretrained encoders, we employ a Parameter-Efficient Fine-Tuning (PEFT) mechanism that adapts the multi-temporal encoder to our downstream segmentation task. This approach enables ULISSE to achieve high accuracy even with a limited amount of labeled data available at training time. Experimental results demonstrate the effectiveness of the proposed methodology across two case studies on mapping bark beetle disturbances in the Czech Republic and Romania study areas.

Reusing complex deep neural models trained by leveraging a big amount of annotated data and computation resources is one of the major challenges recently addressed with the emerging Data-Centric Artificial Intelligence paradigm, to pave the way for the effective development of a Green Artificial Intelligence technology. In this paper, we consider the foundation ResNet50 model as it is pre-trained for Sentinel-2 land cover image classification in BigEarthNet. We reuse this pre-trained model as backbone of deep neural models developed for semantic segmentation, pixel image classification, and CVA in the down-stream task of mapping bark beetle outbreaks in Sentinel-2 images of forest areas. The evaluation study explores the effectiveness of the considered solutions to reuse a foundation deep model in a case study regarding forest scenes that are annotated with bark beetle outbreaks observed in September 2020 in the Czech Republic.

In the last years, several gradient-based attacks have been developed to disclose vulnerabilities of deep neural models and strengthen evasion and transferability abilities of adversarial examples created in various domains. In Windows PE malware detection, various gradient-based methods have been recently tested to optimize the editing or injection of adversarial bytes in unused file sections, and create adversarial malicious PE files to evade deep neural models developed for malware detection. On the other hand, gradient information is commonly used in eXplainable AI (XAI) to obtain explanations for opaque decision-making processes of deep neural black-boxes, while Adversarial XAI has recently emerged as an Adversarial Learning field to investigate the security landscape of XAI. In this study, we describe a holistic, gradient-based attack method, named GAME4EXE, formulated to optimize editing of adversarial bytes in the DOS Header and DOS Stub sections of Windows PE malware. The optimization is driven by the dual adversary purpose of generating adversarial malware that evades a target deep neural model, and aligning the explanation of the evading decisions with that of a goodware-like explanation. A preliminary evaluation shows that a gradient-based attack can be effectively formulated in the Windows PE malware domain, to equip the evasion of a deep neural model with the capability to coherently deceive its decision process explainer and harden attack detection.

Android OS is today the most used Operating System for mobile devices. However, it is susceptible to several malware attacks that may seriously compromise the privacy and security of individuals and organizations. This paper proposes an approach based on a static analysis of decompiled Android PacKages (APKs) to extract critical APIs and detect Android malware. The main contributions lie in the adoption of a graph-based data engineering schema to represent APIs taken from the Function Call Graphs of decompiled APKs and the formulation of a graph-based deep learning approach for explainable malware detection. In particular, the proposed approach, named ANAKIN, implements a Graph Neural Network (GNN) for binary classification (malware versus goodware), and integrates GNNExplainer algorithm to disclose how specific API classes and control-flow edges between API calls influence malware alerts. The proposed approach was evaluated by considering 26,527 Android APKs. The results of an extensive and in-depth evaluation show that the presented GNN model achieves higher accuracy than deep neural models trained with traditional API call sequence representations and publicly available related methods. On the other hand, it produces decision explanations that yield interesting insights into the malicious patterns of APKs and support root cause analysis of missed malware alarms.

Open remote sensing science has been recently boosted by the free availability of Sentinel-2 images of planet Earth acquired with the Copernicus programme. In particular, processing open Sentinel-2 images with Artificial Intelligence (AI) techniques holds great potential for revolutionizing data science applications in many domains of Earth sciences. In this paper, we explore the potential of an unsupervised learning method designed to process Sentinel-2 images of Earth’s forest scenes and automate the inventory of forest tree dieback caused by bark beetle outbreaks. Specifically, we describe PHANTASM: a method to identify forest tree dieback patches performing the Change Vector Analysis (CVA) of bi-temporal Sentinel-2 images of forest scenes. While the traditional CVA strategy is based on the analysis of pixel-wise differences in spectral values, we enrich the Sentinel-2 spectrum with both a selection of Spectral Vegetation Indexes and a Spectral-Spatial Deep Embedding. The Spectral Vegetation Indexes are pre-defined combinations of spectral bands commonly designed to enhance the accuracy of semantic segmentation models trained to map bark beetle stress in spectral data. The Deep Embedding is a spectral-spatial representation of Sentinel-2 pixels trained with a deep neural network. In particular, we use a pre-trained, semantic segmentation U-Net to obtain the Deep Embedding that models the spatial relationship among neighbouring spectral pixels. We assess the effectiveness of the proposed method in a case study regarding bark beetle outbreaks in Sentinel-2 images of forest scenes in the Czech Republic.

The ever-increasing evolution of Deep Learning methods has enabled their use in many areas, including cybersecurity. With the exponential growth in the volume of data circulating in computer networks, their security is a paramount necessity. Nowadays, network security is mainly pursued using preventive techniques but also by detecting intrusions as soon as possible, when they occur. Different types of Machine Learning and Deep Leaning models have been recently studied for network intrusion detection, but surprisingly, although Network Intrusion Detection Systems (NIDSs) scrutinize flow data exchanges on a network, graph-based models have been little explored so far. We propose in this article to consider the relevance of Graph Neural Network (GNNs) to detect intrusions and also to explain them. For this purpose, we adapt the GNNExplainer method, that is, the pioneer method for explaining GNN decisions, to edge-level classification models.

A multitude of recent studies have repeatedly shown the accuracy of deep neural models in several malware detection problems. Although deep learning has recently achieved amazing results in cybersecurity, deep neural models remain complex models, which often produce non-transparent decisions, and which are vulnerable to adversarial attacks. Hence, the evaluation of a deep neural model in cybersecurity should include the analysis of the simplicity and vulnerability of the model, in addition to its accuracy. In this study, we investigate how XAI can disclose useful information concerning the robustness of the input characteristics in deep neural models and how this knowledge can be used in malware detection problems to pursue simpler deep neural models that are still accurate, as well as to fool deep neural models. In particular, AI defenders are interested in identifying the minimum amount of input characteristics to train a simple deep neural model by preserving high accuracy. AI attackers are interested in identifying the minimum amount of input characteristics to perturb, in order to evade deep neural models. We explore how simplicity can be realized in malware detection problems by accounting for explanations of input characteristics, which are produced with either a global XAI technique or a Mutual Information analysis.

Forests play a key role in maintaining the balance of ecosystems, regulating climate, conserving biodiversity, and supporting various ecological processes. However, insect outbreaks, particularly bark beetle outbreaks, pose a significant threat to European spruce forest health by causing an increase in forest tree mortality. Therefore, developing accurate forest disturbance inventory strategies is crucial to quantifying and promptly mitigating outbreak diseases and boosting effective environmental management. In this paper, we propose a deep learning-based approach, named AVALON, that implements a CNN to detect tree dieback events in Sentinel-2 images of forest areas. To this aim, each pixel of a Sentinel-2 image is transformed into an imagery representation that sees the pixel within its surrounding pixel neighbourhood. We incorporate an attention mechanism into the CNN architecture to gain accuracy and achieve useful insights from the explanations of the spatial arrangement of model decisions. We assess the effectiveness of the proposed approach in two case studies regarding forest scenes in the Northeast of France and the Czech Republic, which were monitored using Sentinel-2 satellite in October 2018 and September 2020, respectively. Both case studies host bark beetle outbreaks in the considered periods.

Artificial Intelligence (AI) is transforming cybersecurity practices thanks to the amazing accuracy performance achieved with several AI-based malware detection systems. However, several recent studies have shown that AI decision models can be vulnerable to adversarial attacks. In malware detection scenarios, adversarial attacks are realistic manipulations of existing malware, which preserve the executable and malicious behaviour but evade the malware detection measures. In this study, we consider Windows Portable Executable (PE) malware, which is currently trending to prominent malware types, and we show that counterfactual explanations can be used to drive the generation of realistic adversarial Windows PE malware to evade AI-based detection. In particular, the proposed method OLIVANDER works in a black-box manner, which is the most restrictive attack option, as the evasion method interacts with the target decision system to evade by merely knowing the model input and output. The evaluation study explores the effectiveness of the proposed evasion method in terms of evasion ability, efficiency of computation, and attack transferability compared to two state-of-the-art evasion methods. In addition, the performed evaluation accounts for performances on commercial anti-malware systems.

Forest tree dieback inventory has a crucial role in improving forest management strategies. This inventory is traditionally performed by forests through laborious and time-consuming human assessment of individual trees. On the other hand, the large amount of Earth satellite data that are publicly available with the Copernicus program and can be processed through advanced deep learning techniques has recently been established as an alternative to field surveys for forest tree dieback tasks. However, to realize its full potential, deep learning requires a deep understanding of satellite data since the data collection and preparation steps are essential as the model development step. In this study, we explore the performance of a data-centric semantic segmentation approach to detect forest tree dieback events due to bark beetle infestation in satellite images. The proposed approach prepares a multisensor data set collected using both the SAR Sentinel-1 sensor and the optical Sentinel-2 sensor and uses this dataset to train a multisensor semantic segmentation model. The evaluation shows the effectiveness of the proposed approach in a real inventory case study that regards non-overlapping forest scenes from the Northeast of France acquired in October 2018. The selected scenes host bark beetle infestation hotspots of different sizes, which originate from the mass reproduction of the bark beetle in the 2018 infestation.

Massive tree dieback events triggered by various disturbance agents, such as insect outbreaks, pests, fires and windstorms, have recently compromised the health of forests in numerous countries with a significant impact on ecosystems. The inventory of forest tree dieback plays a key role in understanding the effects of forest disturbance agents and improving forest management strategies. In this article, we illustrate a deep learning approach that trains a U-Net model for the semantic segmentation of Sentinel-2 images of forest areas. The proposed U-Net architecture integrates an attention mechanism to amplify the crucial information and a self-distillation approach to transfer the knowledge within the U-Net architecture. Experimental results demonstrate the significant contribution of both attention and self-distillation to gaining accuracy in two case studies in which we perform the inventory mapping of forest tree dieback caused by insect outbreaks and wildfires, respectively.

Ensemble learning is a strategy commonly used to fuse different base models by creating a model ensemble that is expected more accurate on unseen data than the base models. This study describes a new cyber-threat detection method, called PANACEA, that uses ensemble learning coupled with adversarial training in deep learning, in order to gain accuracy with neural models trained in cybersecurity problems. The selection of the base models is one of the main challenges to handle, in order to train accurate ensembles. This study describes a model ensemble pruning approach based on eXplainable AI (XAI) to increase the ensemble diversity and gain accuracy in ensemble classification. We base on the idea that being able to identify base models that give relevance to different input feature sub-spaces may help in improving the accuracy of an ensemble trained to recognise different signatures of different cyber-attack patterns. To this purpose, we use a global XAI technique to measure the ensemble model diversity with respect to the effect of the input features on the accuracy of the base neural models combined in the ensemble. Experiments carried out on four benchmark cybersecurity datasets (three network intrusion detection datasets and one malware detection dataset) show the beneficial effects of the proposed combination of adversarial training, ensemble learning and XAI on the accuracy of multi-class classifications of cyber-data achieved by the neural model ensemble.

The bark beetle is one of the most critical, biotic disturbance agents causing tree dieback in several coniferous forest ecosystems around Europe. Forest dieback inventory plays a crucial role to study the effect of this biotic forest disturbance and improve forest management strategies. In this study, we explore the performance of remote sensing methods used to perform the inventory mapping of bark beetle-induced forest dieback. Specifically, we analyse the performance of classification models trained with Random Forest and XGBoost, as well as semantic segmentation models trained with U-Net by accounting for both spectral bands of Sentinel-2 images and some developed spectral vegetation indices. In addition, we investigate the effect of accounting for temporal knowledge on the performance of remote sensing methods. To this aim, we consider a dataset of Sentinel-2 time series acquired from May to October 2018 in non-overlapping forest scenes from the Northeast of France. The selected scenes host bark beetle infestation hotspots of different sizes, which originate from the mass reproduction of the bark beetle in the 2018 infestation. The results of this study show that the Random Forest model trained taking into account the temporal patterns in both spectral bands and vegetation indices achieves the highest accuracy in the study inventory task. Finally, we use an eXplainable Artificial Intelligence technique to explain the effect of temporal knowledge on the Random Forest inventory decisions.

Download This Paper Open PDF in Browser Add Paper to My Library Share: Permalink Using these links will ensure access to this page indefinitely Copy URL Copy DOI

Vision Transformers (ViTs) denote a family of attention-based deep learning techniques that have recently achieved amazing results in various problems related to the field of computer vision. In this paper, we explore the use of ViTs in problems of cyber-threat detection related to malware and network intrusion detection. In particular, we propose VINCENT, that is a novel deep neural method, which resorts to a color imagery representation of cyber-data by encoding related cyber-data features into neighboring color pixels. ViTs are trained from cyber-data images as teacher models, to extract explainable imagery signatures of cyber-data classes. This knowledge is extracted by leveraging the self-attention mechanism to give paired attention values between pairs of imagery patches. The signature knowledge, extracted through the ViT teacher, is, finally, used to train a smaller neural student model according to the knowledge distillation theory. Experiments with various benchmark cybersecurity datasets assess the accuracy of the student model VINCENT also compared to that of several state-of-the-art methods. In addition, it shows that VINCENT can obtain insights from explanations recovered through the self-attention mechanism of the ViT teacher.

Forest tree dieback inventory plays a crucial role to improve forest management strategies. In this study, we explore the performance of a spectral-spatial machine learning approach used to analyse Sentinel-2 images to detect forest tree dieback events due to bark beetle infestation. We analyse the performance of classification models trained with Random Forest, XGBoost and Multi-Layer Perceptron, as well as semantic segmentation models trained with U-Net by accounting for both spectral and spatial information contained in the remote sensing data. We consider a set of Sentinel-2 images acquired in non-overlapping forest scenes from a region located in the Northeast of France. The selected scenes host bark beetle infestation hotspots originated from the mass reproduction of the bark beetle in the 2018 infestation. Results show that the U-Net model, trained accounting for spectral and spectral-spatial data, achieves the best performance. However, the simpler Random Forest model achieves competitive results with respect to the more complex one, namely U-Net.

Change Detection (CD) aims to distinguish surface changes based on bi-temporal remote sensing images. In recent years, deep neural models have made a breakthrough in CD processes. However, training a deep neural model requires a large volume of labelled training samples that are time-consuming and labour-intensive to acquire. With the aim of learning an accurate CD model with limited labelled data, we propose SENECA: a method based on a CD Siamese network, which takes advantage of both Transfer Learning (TL) and Active Learning (AL) to handle the constraint of limited supervision. More precisely, we jointly use AL and TL to adapt a CD model trained on a labelled source domain to a (related) target domain featured by restricted access to labelled data. We report results from an experimental evaluation involving five pairs of images acquired via Sentinel-2 satellites between 2015 and 2018 in various locations picked all over Asia and USA. The results show the beneficial effects of the proposed AL and TL strategies on the accuracy of the decisions made by the CD Siamese network and depict the merit of the proposed approach over competing CD baselines.