This paper investigates the use of Mean Opinion Score (MOS), a common image quality metric, as a user-centric evaluation metric for XAI post-hoc explainers. To measure the MOS, a user experiment is proposed, which has been conducted with explanation maps of intentionally distorted images. Three methods from the family of feature attribution methods - Gradient-weighted Class Activation Mapping (Grad-CAM), Multi-Layered Feature Explanation Method (MLFEM), and Feature Explanation Method (FEM) - are compared with this metric. Additionally, the correlation of this new user-centric metric with automatic metrics is studied via Spearman's rank correlation coefficient. MOS of MLFEM shows the highest correlation with automatic metrics of Insertion Area Under Curve (IAUC) and Deletion Area Under Curve (DAUC). However, the overall correlations are limited, which highlights the lack of consensus between automatic and user-centric metrics.

The most popular methods in AI-machine learning paradigm are mainly black boxes. For user acceptance of these methods in various tasks, explanations are necessary for why an AI tool takes such a decision, as it is a matter of user trust in AI. Although dedicated explanation tools are being massively developed, the users and decision makers do not know which are the most appropriate for them. In this chapter, we will propose a taxonomy of the AI-explanation methods. This contribution is aimed at helping users in their choice of explanation methods for AI (XAI).

In many 2D visualizations, data points are projected without considering their surface area, although they are often represented as shapes in visualization tools. These shapes support the display of information such as labels or encode data with size or color. However, inappropriate shape and size selections can lead to overlaps that obscure information and hinder the visualization's exploration. Overlap Removal (OR) algorithms have been developed as a layout post-processing solution to ensure that the visible graphical elements accurately represent the underlying data. As the original data layout contains vital information about its topology, it is essential for OR algorithms to preserve it as much as possible. This article presents an extension of the previously published FORBID algorithm by introducing a new approach that models OR as a joint stress and scaling optimization problem, utilizing efficient stochastic gradient descent. The goal is to produce an overlap-free layout that proposes a compromise between compactness (to ensure the encoded data is still readable) and preservation of the original layout (to preserve the structures that convey information about the data). Additionally, this article proposes SORDID, a shape-aware adaptation of FORBID that can handle the OR task on data points having any polygonal shape. Our approaches are compared against state-of-the-art algorithms, and several quality metrics demonstrate their effectiveness in removing overlaps while retaining the compactness and structures of the input layouts.

Due to their great performance in many challenges, Deep Learning (DL) techniques keep gaining popularity in many fields. They have been adapted to process graph data structures to solve various complicated tasks such as graph classification and edge prediction. Eventually, they reached the Graph Drawing (GD) task. This article is an extended version of the previously published (DNN)2 and presents a framework to leverage DL techniques for graph drawing (DL4GD). We demonstrate how it is possible to train a Deep Learning model to extract features from a graph and project them into a graph layout. The method proposes to leverage efficient Convolutional Neural Networks, adapting them to graphs using Graph Convolutions. The graph layout projection is learned by optimizing a cost function that does not require any ground truth layout, as opposed to prior work. This paper also proposes an implementation and benchmark of the framework to study its sensitivity to certain deep learning-related conditions. As the field is novel, and many questions remain to be answered, we do not focus on finding the most optimal implementation of the method, but rather contribute toward a better understanding of the approach potential. More precisely, we study different learning strategies relative to the models training datasets. Finally, we discuss the main advantages and limitations of DL4GD.

Current deep learning approaches are cutting-edge methods for solving classification tasks. Arising transfer learning techniques allows applying large generic model to simple tasks whereas simpler models could be used. Large models raise the major problem of their memory consumption and processor usage and lead to a prohibitive ecological footprint. In that paper, we present a novel visual analytics approach to interactively prune those networks and thus limit that issue. Our technique leverages a novel sparkline matrix visualization technique as well as a novel local metric which evaluates the discriminatory power of a filter to guide the pruning process and make it interpretable. We assess the well- founded of our approach through two realistic case studies and a user study. For both of them, the interactive refinement of the model led to a significantly smaller model having similar prediction accuracy than the original one.

The rise of Deep Learning (DL) has led to a breakthrough in the research field of content-based multimedia indexing. Newly developed systems based on complex models outperform classic machine learning algorithms in object detection, image segmentation or classification tasks. However, despite their high performance, these systems still make mistakes. To be used in industrial conditions, these systems must be able to provide trustworthy decisions with guarantees or justifications. Therefore, it is crucial to provide means to analyze and comprehend the decision process that leads a model to its decision. Image classification implies tracking and understanding which input features the model relies on to make its prediction. This paper focuses on features attribution techniques and proposes Heatmaps by Hierarchical Occlusion (H2O), a novel method for detecting pattern-relevant features in an image. We also propose two new pairs of metrics that overcome some evaluation issues: (a) Insertion and Deletion Spearman correlation coefficients which both estimate a correlation between the computed scores in a saliency map and the importance for the model of the associated pixels in the image. (b) Insertion Positive and Deletion Negative Gradient Sums both estimate the coherence of the scores in the saliency maps. Both visual inspection and evaluation on 7 metrics show that H2O is competitive against state-of-the-art methods.

While many graph drawing algorithms consider nodes as points, graph visualization tools often represent them as shapes. These shapes support the display of information such as labels or encode various data with size or color. However, they can create overlaps between nodes which hinder the exploration process by hiding parts of the information. It is therefore of utmost importance to remove these overlaps to improve graph visualization readability. If not handled by the layout process, Overlap Removal (OR) algorithms have been proposed as layout post-processing. As graph layouts usually convey information about their topology, it is important that OR algorithms preserve them as much as possible. We propose a novel algorithm that models OR as a joint stress and scaling optimization problem, and leverages efficient stochastic gradient descent. This approach is compared with state-of-the-art algorithms, and several quality metrics demonstrate its efficiency to quickly remove overlaps while retaining the initial layout structures.

In 2D visualizations, visibility of every datum's representation is crucial to ease the completion of visual tasks. Such a guarantee is barely respected in complex visualizations, mainly because of overdraws between datum representations that hide parts of the information (e.g., outliers). The literature proposes various Layout Adjustment algorithms to improve the readability of visualizations that suffer from this issue. Manipulating the data in high-dimensional, geometric or visual space; they rely on different strategies with their own strengths and weaknesses. Moreover, most of these algorithms are computationally expensive as they search for an exact solution in the geometric space and do not scale well to large datasets. This article proposes GIST, a layout adjustment algorithm that aims at optimizing three criteria: (i) node visibility guarantee (at least 1 pixel), (ii) node size maximization, and (iii) the original layout preservation. This is achieved by combining a search for the maximum node size that enables to draw all the data points without overlaps, with a limited budget of movements (i.e., limiting the distortions of the original layout). The method's basis relies on the idea that it is not necessary for two data representations to be strictly not overlapping in order to guarantee their visibility in visual space. Our algorithm therefore uses a tolerance in the geometric space to determine the overlaps between pairs of data. The tolerance is optimized such that the approximation computed in the geometric space can lead to visualization without noticeable overdraw after the data rendering rasterization. In addition, such an approximation helps to ease the algorithm's convergence as it reduces the number of constraints to resolve, enabling it to handle large datasets. We demonstrate the effectiveness of our approach by comparing its results to those of state-of-the-art methods on several large datasets.

come from teaching and research institutions in France or abroad, or from public or private research centers.L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

The most popular methods for Artificial Intelligence such as Deep Neural Networks are, for the vast majority, considered black boxes. It is necessary to explain their decisions to understand the input data which influence most the result. Methods presented in this paper aim at an explanation in image classification tasks: which data in the input are the most important for the result. We further extend the Feature Explanation Method (FEM) from our previous work, transforming it into a multi-layered FEM (MLFEM). The evaluation of the method is designed by comparison of explanation maps with human Gaze Fixation Density maps (GFDM). We show that proposed MLFEM outperforms FEM and popular DNN explanation methods in terms of classical comparison metrics with GFDM.

Projecting a set of $n$ points on a grid of size $\sqrt{n}\times\sqrt{n}$ provides the best possible information density in two dimensions without overlap. We leverage the Voronoi Relaxation method to devise a novel and versatile post-processing algorithm called VRGrid: it enables the arrangement of any 2D data on a grid while preserving its initial positions. We apply VRGrid to generate compact and overlap-free visualization of popular and overlap-prone projection methods (e.g., t-SNE). We prove that our method complexity is $O(\sqrt{n}.i.n.log(n))$ , with i a determined maximum number of iterations and $n$ the input dataset size. It is thus usable for visualization of several thousands of points. We evaluate VRGrid's efficiency with several metrics: distance preservation (DP), neighborhood preservation (NP), pairwise relative positioning preservation (RPP) and global positioning preservation (GPP). We benchmark VRGrid against two state-of-the-art methods: Self-Sorting Maps (SSM) and Distance-preserving Grid (DGrid). VRGrid outperforms these two methods, given enough iterations, on DP, RPP and GPP which we identify to be the key metrics to preserve the positions of the original set of points.

Current machine learning and deep learning approaches are cutting-edge methods for solving classification tasks. Comparing the performances of classification models has become a prominent task since the outbreak of these techniques. The performance of such classification models is measured by the ratio between the correctly predicted samples and the others. The most widely used visualization to represent this information is the Confusion matrix. Yet, if this technique is suited to apprehend one model performances, very few works use this representation to compare models. In that paper, we present the Relative Confusion Matrix (RCM), a new matrix visualization that leverages Confusion matrices and a color encoding to expose the class-wise differences of performances between two models. We conduct a user evaluation to compare RCM with two confusion matrix variants. Our results show that RCM encoding leads to a more efficient comparison of two models than existing approaches.

The design of efficient representations is well established as a fruitful way to explore and analyze complex or large data. In these representations, data are encoded with various visual attributes depending on the needs of the representation itself. To make coherent design choices about visual attributes, the visual search field proposes guidelines based on the human brain’s perception of features. However, information visualization representations frequently need to depict more data than the amount these guidelines have been validated on. Since, the information visualization community has extended these guidelines to a wider parameter space.This paper contributes to this theme by extending visual search theories to an information visualization context. We consider a visual search task where subjects are asked to find an unknown outlier in a grid of randomly laid out distractors. Stimuli are defined by color and shape features for the purpose of visually encoding categorical data. The experimental protocol is made of a parameters space reduction step (i.e., sub-sampling) based on a machine learning model, and a user evaluation to validate hypotheses and measure capacity limits. The results show that the major difficulty factor is the number of visual attributes that are used to encode the outlier. When redundantly encoded, the display heterogeneity has no effect on the task. When encoded with one attribute, the difficulty depends on that attribute heterogeneity until its capacity limit (7 for color, 5 for shape) is reached. Finally, when encoded with two attributes simultaneously, performances drop drastically even with minor heterogeneity.

The design of efficient representations is well established as a fruitful way to explore and analyze complex or large data. In these representations, data are encoded with various visual attributes depending on the needs of the representation itself. To make coherent design choices about visual attributes, the visual search field proposes guidelines based on the human brain perception of features. However, information visualization representations frequently need to depict more data than the amount these guidelines have been validated on. Since, the information visualization community has extended these guidelines to a wider parameter space. This paper contributes to this theme by extending visual search theories to an information visualization context. We consider a visual search task where subjects are asked to find an unknown outlier in a grid of randomly laid out distractor. Stimuli are defined by color and shape features for the purpose of visually encoding categorical data. The experimental protocol is made of a parameters space reduction step (i.e., sub-sampling) based on a machine learning model, and a user evaluation to measure capacity limits and validate hypotheses. The results show that the major difficulty factor is the number of visual attributes that are used to encode the outlier. When redundantly encoded, the display heterogeneity has no effect on the task. When encoded with one attribute, the difficulty depends on that attribute heterogeneity until its capacity limit (7 for color, 5 for shape) is reached. Finally, when encoded with two attributes simultaneously, performances drop drastically even with minor heterogeneity.

This paper presents a prototype for the visualization of food-drug interactions implemented in the MIAM project, whose objective is to develop methods for the extraction and representation of these interactions and to make them available in the Thériaque database. The prototype provides users with a graphical visualization showing the hierarchies of drugs and foods in front of each other and the links between them representing the existing interactions as well as additional details about them, including the number of articles reporting the interaction. The prototype is interactive in the following ways: hierarchies can be easily folded and unfolded, a filter can be applied to view only certain types of interactions, and details about a given interaction are displayed when the mouse is moved over the corresponding link. Future work includes proposing a version more suitable for non-health professional users and the representation of the food hierarchy based on a reference classification.

In information visualization, it has become mandatory to assess visualization techniques efficiency either to write a survey, optimize a technique or even design a new one. To do so, the common way is to conduct user evaluations through which human subjects are asked to solve a task on different visualization techniques while their performances are measured to assess which technique is the most efficient. These evaluations can be complex to design and setup in order not to be biased and, in the end, their results can become contestable when the evaluation methods standards evolve. To overcome these flaws, new evaluation methods are emerging, mostly making use of modern and efficient computer vision techniques such as deep learning. These new methods rely on a strong assumption that has not been studied deeply enough yet: humans and deep learning models performances can be correlated. This paper explores the performances of both a state-of-the-art deep neural network and human subjects on an outlier detection task taken from a previous experiment of the literature. The objective is to study whether the machine and humans behaviors were different or if some correlations can be observed. Our study shows that their results are significantly correlated and a machine learning model efficiently learned to predict human performances using deep neural network metrics as input. Hence, this work presents a use case where using a deep neural network to assess human subjects performances is efficient.

Deep Neural Networks are becoming the prominent solution when using machine learning models. However, they suffer from a black-box effect that renders complicated their inner workings interpretation and thus the understanding of their successes and failures. Information visualization is one way among others to help in their interpretability and hypothesis deduction. This paper presents a novel way to visualize a trained DNN to depict at the same time its architecture and its way of treating the classes of a test dataset at the layer level. In this way, it is possible to visually detect where the DNN starts to be able to discriminate the classes or where it could decrease its separation ability (and thus detect an oversized network). We have implemented the approach and validated it using several well-known datasets and networks. Results show the approach is promising and deserves further studies.