The advance of foundation models (FM) makes it possible to avoid parametric tuning for transfer learning, taking advantage of pretrained feature spaces. In this study, we define a framework called IDEAL (Interpretable- by-design DEep learning ALgorithms) which tackles the problem of interpretable transfer learning by recasting the standard supervised classification problem into a function of similarity to a set of prototypes derived from the training data. This framework generalises previously-known prototypical approaches, such as ProtoPNet, xDNN and DNC, and decomposes the overall problem into two inherently connected stages: (A) feature extraction (FE), which maps the raw features of real-world data into a latent space, and (B) identification of representative prototypes and decision making based on similarity and association between the query and the prototypes. This addresses the issue of interpretability (stage B) while retaining the benefits of pretrained deep learning (DL) models. On a range of datasets (CIFAR-10, CIFAR-100, CalTech101, STL-10, Oxford-IIIT Pet, EuroSAT), we demonstrate, through an extensive set of experiments, how the choice of the latent space, prototype selection, and finetuning of the latent space affect accuracy and generalisation of the models on transfer learning scenarios for different backbones. Building upon this knowledge, we demonstrate that the proposed framework helps achieve an advantage over state-of-the-art baselines in class-incremental learning. The key findings can be summarised as follows: (1) the setting allows interpretability through prototypes, (2) lack of finetuning helps circumvent the issue of catastrophic forgetting, allowing efficient class-incremental transfer learning, while mitigating the issue of confounding bias, and (3) ViT architectures narrow the gap between finetuned and non-finetuned models allowing for transfer learning in a fraction of time without finetuning of the feature space on a target dataset with iterative supervised methods.

In this paper, we address two critical challenges in the domain of flood detection: the computational expense of large-scale time series change detection and the lack of interpretable decision-making processes on explainable AI (XAI). To overcome these challenges, we proposed an interpretable multi-stage approach to flood detection, IMAFD has been proposed. It provides an automatic, efficient and interpretable solution suitable for large-scale remote sensing tasks and offers insight into the decision-making process. The proposed IMAFD approach combines the analysis of the dynamic time series image sequences to identify images with possible flooding with the static, within-image semantic segmentation. It combines anomaly detection (at both image and pixel level) with semantic segmentation. The flood detection problem is addressed through four stages: (1) at a sequence level: identifying the suspected images (2) at a multi-image level: detecting change within suspected images (3) at an image level: semantic segmentation of images into Land, Water or Cloud class (4) decision making. Our contributions are two folder. First, we efficiently reduced the number of frames to be processed for dense change detection by providing a multi-stage holistic approach to flood detection. Second, the proposed semantic change detection method (stage 3) provides human users with an interpretable decision-making process, while most of the explainable AI (XAI) methods provide post hoc explanations. The evaluation of the proposed IMAFD framework was performed on three datasets, WorldFloods, RavAEn and MediaEval. For all the above datasets, the proposed framework demonstrates a competitive performance compared to other methods offering also interpretability and insight.

This paper describes the automatic identification of ivory using Raman spectroscopy data and deep neural network (DNN) models pre-trained on open-source data from inorganic minerals. The proposed approach uses transfer learning (TL) from foundation models trained on a larger inorganic (minerals) spectroscopy dataset (MLROD). The results demonstrate, for the first time, the ability to transfer machine learning (ML) models from a Raman spectroscopy dataset of geological substances to classify biological ivory samples. Current identification methods, such as DNA analysis and radiocarbon dating, are costly and destructive. Recently, it was demonstrated that the use of Raman spectroscopy, a laser-based, non-destructive technique, in combination with well-known statistical techniques, has the potential to differentiate between mammoth and elephant ivory. However, this previous study had a small sample size due to difficulties in obtaining large amounts of labeled ivory data. To date, there has been no reported work on ivory classification using DNNs, and only limited studies using Raman spectra. The work proposed in this paper suggests that ML can provide high levels of accuracy in the classification of Raman spectroscopy data from ivory samples of different elephant species (up to 99.7%). This has the potential to create a quick and inexpensive method of identifying legal and illegal types of ivory to aid in enforcement of ivory trade bans. This study also demonstrated that DNN models initially pre-trained on inorganic minerals (from the MLROD dataset) that were not finetuned on ivory data had a high accuracy rate of 92%, alleviating the need for large amounts of training data from ivory specimens. Finally, the approach proposed in this paper, provides insight into the decision making and interpretation of the results using prototype-based models. This novel work demonstrates that: (1) ML methods can provide highly accurate classification of ivory from different species of elephant using data obtained using Raman spectroscopy and providing insight into the decision making (2) TL enables re-purposing the models trained on larger mineral datasets of inorganic materials (such as MLROD) to discriminating between the classes of ivory, containing inorganic and organic biological components, for the first time transgressing between non-biological and biological samples (3) the proposed method allows both training from labelled samples of ivory and the identification of unknown ivory samples through prototype-based methods.

Vision graph neural networks have emerged as a popular approach for modeling the global and spatial context for image recognition. However, a significant drawback of these methods is that they do not offer an inherent interpretation of the relevant spatial interactions for their prediction. We address this problem by introducing i-WiViG, an approach that enables interpretable model reasoning based on a sparse subgraph in the image. i-WiViG is based on two key postulates: 1) constraining the graph nodes' receptive field to disjoint local windows in the image, and 2) an inherently interpretable graph bottleneck with learnable sparse attention that identifies the relevant interactions among the local image windows. We evaluate our approach on both scene classification and regression tasks using natural and remote sensing imagery. Our results, supported by quantitative and qualitative evidence, demonstrate that the method delivers semantic, intuitive, and faithful explanations through the identified subgraphs. Furthermore, extensive experiments confirm that it achieves competitive performance to its black-box counterparts, even on datasets exhibiting strong texture bias. The implementation is available on https://github.com/zhu-xlab/i-WiViG.

Aligning machine representations with human understanding is key to improving interpretability of machine learning (ML) models. When classifying a new image, humans often explain their decisions by decomposing the image into concepts and pointing to corresponding regions in familiar images. Current ML explanation techniques typically either trace decision-making processes to reference prototypes, generate attribution maps highlighting feature importance, or incorporate intermediate bottlenecks designed to align with human-interpretable concepts. The proposed method, named COMIX, classifies an image by decomposing it into regions based on learned concepts and tracing each region to corresponding ones in images from the training dataset, assuring that explanations fully represent the actual decision-making process. We dissect the test image into selected internal representations of a neural network to derive prototypical parts (primitives) and match them with the corresponding primitives derived from the training data. In a series of qualitative and quantitative experiments, we theoretically prove and demonstrate that our method, in contrast to post hoc analysis, provides fidelity of explanations and shows that the efficiency is competitive with other inherently interpretable architectures. Notably, it shows substantial improvements in fidelity and sparsity metrics, including 48.82 ImageNet dataset over the best state-of-the-art baseline.

Many of the existing deep learning methods are trained for scenarios which: (1) use (costly) fine-tuning of the latent spaces on the target dataset in the ”downstream task” (2) do not account for continual and open-set learning (3) do not provide interpretability. Instead of trying to solve the problem of semi- and unsupervised learning through representation learning, we propose recasting it into the problem of analysing existing foundational models’ feature spaces. We show that a simple baseline, based on non-parametric clustering analysis of the latent feature spaces and pre-trained classifiers on large-scale datasets, can help solve a set of such problems even without finetuning. It can also be seen as a set of metrics for assessment of generalisation within the latent feature spaces. We argue that better generalising pre-trained architectures can solve a number of problems without finetuning, providing the basis for lifelong learning without catastrophic forgetting and with a means of interpretation.

Foundation Models (FM) developed on very large generic data sets transformed the landscape of machine learning (ML). Vision transformers (ViT) closed the gap in performance between fine-tuned and unsupervised transfer learning. This opens the possibility to abandon the widely used until recently end-to-end approach. Instead, we consider a two-stage ML pipeline, where the first stage constitutes extracting features by pre-training large, multi-layer model with billions of parameters, and the second stage is a computationally lightweight learning of an entirely new, simpler model architecture based on prototypes within this feature space. In this paper we consider such two-stage approach to ML. We further analyse the use of several alternative light-weight methods in the second stage, including strategies for semi-supervised learning and a variety of strategies for linear fine-tuning. We demonstrate on the basis of nine well known benchmark data sets that the ultra-light-weight ML alternatives for the second stage (such as clustering, PCA, LDA and combinations of these) offer for the price of negligible drop in accuracy a significant (several orders of magnitude) drop of computational costs (time, energy and related CO 2 emissions) as well as the ability to use no labels (fully unsupervised approach) or limited amount of labels (one per cluster labels) and the ability to address interpretability.

As various deep learning applications have been deployed in diverse areas, the explainability of neural networks is becoming increasingly important in the research field. Besides being desirable on its own account, explainability also often helps further improve performance of deep learning models. In this work, we introduce float neurons and fixed neurons to describe the neuron-level stability in a network based on the activation pattern of neurons on given input. With the proposed concept, we quantify the expressive ability and robustness of a neural network with a neuron entropy metric and illustrate their relationship by decomposing the computational graph of a neural network. We find theoretically that networks with better generalization have more diverse activation patterns across the input space, which results in a higher neuron entropy globally. On the other hand, the prediction of neural networks is prone to be affected by perturbation when there are locally more float neurons, which respond with additional impulses to local stimuli. Empirically, we show that the proposed analytical framework can be applied to downstream applications, including network pruning and randomized smoothing of network prediction.

The vision transformer-based foundation models, such as ViT or Dino-V2, are aimed at solving problems with little or no finetuning of features. Using a setting of prototypical networks, we analyse to what extent such foundation models can solve unsupervised domain adaptation without finetuning over the source or target domain. Through quantitative analysis, as well as qualitative interpretations of decision making, we demonstrate that the suggested method can improve upon existing baselines, as well as showcase the limitations of such approach yet to be solved.

The proposed framework named IDEAL (Interpretable-by-design DEep learning ALgorithms) recasts the standard supervised classification problem into a function of similarity to a set of prototypes derived from the training data, while taking advantage of existing latent spaces of large neural networks forming so-called Foundation Models (FM). This addresses the issue of explainability (stage B) while retaining the benefits from the tremendous achievements offered by DL models (e.g., visual transformers, ViT) pre-trained on huge data sets such as IG-3.6B + ImageNet-1K or LVD-142M (stage A). We show that one can turn such DL models into conceptually simpler, explainable-through-prototypes ones. The key findings can be summarized as follows: (1) the proposed models are interpretable through prototypes, mitigating the issue of confounded interpretations, (2) the proposed IDEAL framework circumvents the issue of catastrophic forgetting allowing efficient class-incremental learning, and (3) the proposed IDEAL approach demonstrates that ViT architectures narrow the gap between finetuned and non-finetuned models allowing for transfer learning in a fraction of time \textbf{without} finetuning of the feature space on a target dataset with iterative supervised methods.

The process of recognizing and distinguishing between real content and content generated by deep learning algorithms, often referred to as deepfakes, is known as deepfake detection. In order to counter the rising threat of deepfakes and maintain the integrity of digital media, research is now being done to create more reliable and precise detection techniques. Deep learning models, such as Stable Diffusion, have been able to generate more detailed and less blurry images in recent years. In this paper, we develop a deepfake detection technique to distinguish original and fake images generated by various Diffusion Models. The developed methodology for deepfake detection takes advantage of features from fine-tuned Vision Transformers (ViTs), combined with existing classifiers such as Support Vector Machines (SVM). We demonstrate the proposed methodology's ability of interpretability-through-prototypes by analysing support vectors of the SVMs. Additionally, due to the novelty of the topic, there is a lack of open datasets for deepfake detection. Therefore, to evaluate the methodology, we have also created custom datasets based on various generative techniques of Diffusion Models on open datasets (ImageNet, FFHQ, Oxford-IIIT Pet). The code is available at https://github.com/lira-centre/DeepfakeDetection.

Continuous-depth neural networks, such as Neural ODEs, have refashioned the understanding of residual neural networks in terms of non-linear vector-valued optimal control problems. The common solution is to use the adjoint sensitivity method to replicate a forward-backward pass optimisation problem. We propose a new approach which explicates the network's `depth' as a fundamental variable, thus reducing the problem to a system of forward-facing initial value problems. This new method is based on the principle of `Invariant Imbedding' for which we prove a general solution, applicable to all non-linear, vector-valued optimal control problems with both running and terminal loss. Our new architectures provide a tangible tool for inspecting the theoretical--and to a great extent unexplained--properties of network depth. They also constitute a resource of discrete implementations of Neural ODEs comparable to classes of imbedded residual neural networks. Through a series of experiments, we show the competitive performance of the proposed architectures for supervised learning and time series prediction.

This paper describes the preliminary results of an ancient history and computer science collaboration to apply machine learning to the identification of text within images of inscriptions, one of our major sources of evidence for the Roman world. This work contributes to our understanding of their planning processes. We describe a machine learning method for locating text within several thousand images, and discuss the next steps for establishing the extent to which Roman workshops used specific controlling modules in the creation of inscriptions. The aims are twofold: to improve our knowledge of Roman monumental practices and craftsmanship, and to enable accurate restorations of damaged texts through a more comprehensive understanding of how they were planned and executed.

Precipitation nowcasting, the high-resolution forecasting of precipitation up to two hours ahead, supports the real-world socio-economic needs of many sectors reliant on weather-dependent decision-making. State-of-the-art operational nowcasting methods typically advect precipitation fields with radar-based wind estimates, and struggle to capture important non-linear events such as convective initiations. Recently introduced deep learning methods use radar to directly predict future rain rates, free of physical constraints. While they accurately predict low-intensity rainfall, their operational utility is limited because their lack of constraints produces blurry nowcasts at longer lead times, yielding poor performance on more rare medium-to-heavy rain events. To address these challenges, we present a Deep Generative Model for the probabilistic nowcasting of precipitation from radar. Our model produces realistic and spatio-temporally consistent predictions over regions up to 1536 km x 1280 km and with lead times from 5-90 min ahead. In a systematic evaluation by more than fifty expert forecasters from the Met Office, our generative model ranked first for its accuracy and usefulness in 88% of cases against two competitive methods, demonstrating its decision-making value and ability to provide physical insight to real-world experts. When verified quantitatively, these nowcasts are skillful without resorting to blurring. We show that generative nowcasting can provide probabilistic predictions that improve forecast value and support operational utility, and at resolutions and lead times where alternative methods struggle.

A 'nowcast' is a type of weather forecast which makes predictions in the very short term, typically less than two hours - a period in which traditional numerical weather prediction can be limited. This type of weather prediction has important applications for commercial aviation; public and outdoor events; and the construction industry, power utilities, and ground transportation services that conduct much of their work outdoors. Importantly, one of the key needs for nowcasting systems is in the provision of accurate warnings of adverse weather events, such as heavy rain and flooding, for the protection of life and property in such situations. Typical nowcasting approaches are based on simple extrapolation models applied to observations, primarily rainfall radar. In this paper we review existing techniques to radar-based nowcasting from environmental sciences, as well as the statistical approaches that are applicable from the field of machine learning. Nowcasting continues to be an important component of operational systems and we believe new advances are possible with new partnerships between the environmental science and machine learning communities.

Building upon the recent success of deep reinforcement learning methods, we investigate the possibility of on-policy reinforcement learning improvement by reusing the data from several consecutive policies. On-policy methods bring many benefits, such as ability to evaluate each resulting policy. However, they usually discard all the information about the policies which existed before. In this work, we propose adaptation of the replay buffer concept, borrowed from the off-policy learning setting, to create the method, combining advantages of on- and off-policy learning. To achieve this, the proposed algorithm generalises the $Q$-, value and advantage functions for data from multiple policies. The method uses trust region optimisation, while avoiding some of the common problems of the algorithms such as TRPO or ACKTR: it uses hyperparameters to replace the trust region selection heuristics, as well as the trainable covariance matrix instead of the fixed one. In many cases, the method not only improves the results comparing to the state-of-the-art trust region on-policy learning algorithms such as PPO, ACKTR and TRPO, but also with respect to their off-policy counterpart DDPG.

Manually labelling new datasets for image classification remains expensive and time-consuming. A promising alternative is to utilize the abundance of images on the web for which search queries or surrounding text offers a natural source of weak supervision. Unfortunately the label noise in these datasets has limited their use in practice. Several methods have been proposed for performing unsupervised label noise cleaning, the majority of which use outlier detection to identify and remove mislabeled images. In this paper, we argue that outlier detection is an inherently unsuitable approach for this task due to major flaws in the assumptions it makes about the distribution of mislabeled images. We propose an alternative approach which makes no such assumptions. Rather than looking for outliers, we observe that mislabeled images can be identified by the detrimental impact they have on the performance of an image classifier. We introduce training-value as an objective measure of the contribution each training example makes to the validation loss. We then present the training-value approximation network (Training-ValueNet) which learns a mapping between each image and its training-value. We demonstrate that by simply discarding images with a negative training-value, Training-ValueNet is able to significantly improve classification performance on a held-out test set, outperforming the state of the art in outlier detection by a large margin.

Reinforcement learning methods have recently achieved impressive results on a wide range of control problems. However, especially with complex inputs, they still require an extensive amount of training data in order to converge to a meaningful solution. This limitation largely prohibits their usage for complex input spaces such as video signals, and it is still impossible to use it for a number of complex problems in a real world environments, including many of those for video based control. Supervised learning, on the contrary, is capable of learning on a relatively small number of samples, however it does not take into account reward-based control policies and is not capable to provide independent control policies. In this article we propose a model-free control method, which uses a combination of reinforcement and supervised learning for autonomous control and paves the way towards policy based control in real world environments. We use SpeedDreams/TORCS video game to demonstrate that our approach requires much less samples (hundreds of thousands against millions or tens of millions) comparing to the state-of-the-art reinforcement learning techniques on similar data, and at the same time overcomes both supervised and reinforcement learning approaches in terms of quality. Additionally, we demonstrate the applicability of the method to MuJoCo control problems.

Reinforcement learning methods have recently achieved impressive results on a wide range of control problems. However, especially with complex inputs, they still require an extensive amount of training data in order to converge to a meaningful solution. This limits their applicability to complex input spaces such as video signals, and makes them impractical for use in complex real world problems, including many of those for video based control. Supervised learning, on the contrary, is capable of learning on a relatively limited number of samples, but relies on arbitrary hand-labelling of data rather than task-derived reward functions, and hence do not yield independent control policies. In this article we propose a novel, model-free approach, which uses a combination of reinforcement and supervised learning for autonomous control and paves the way towards policy based control in real world environments. We use SpeedDreams/TORCS video game to demonstrate that our approach requires much less samples (hundreds of thousands against millions or tens of millions) comparing to the state-of-the-art reinforcement learning techniques on similar data, and at the same time overcomes both supervised and reinforcement learning approaches in terms of quality. Additionally, we demonstrate applicability of the method to MuJoCo control problems.