Recurrence-Aware Long-Term Cognitive Network (rLTCN) classifiers have reported comparable performance to mainstream black-box models, including tree ensembles and support vector machines, in tabular pattern classification tasks. These classifiers use a two-step learning algorithm to address issues that arise during the training of recurrent neural networks. While the weights in the recurrent block are computed using unsupervised learning, recurrence-aware weights are determined using a one-step learning rule based on the Moore-Penrose inverse. However, the related least-squares learning problem tends to favor easy instances and common patterns, particularly those associated with the majority class in imbalanced datasets. In such scenarios, a loss function that directly optimizes a robust metric, such as the F1 score, would lead to models with stronger generalization capabilities. Unfortunately, incorporating such a metric into the Moore-Penrose inverse learning procedure presents challenges from a mathematical viewpoint. In this paper, we propose four gradient-based correction methods that modify the output logits of rLTCN classifiers once the two-step training process is done. Inspired by procedures such as Platt or Beta scaling, the proposed post-optimization correction methods seek to maximize the F1 score rather than produce calibrated probabilities. The simulations using real-world datasets show that adding a correction layer to rLTCNs improves their performance significantly at the expense of occasional reductions in the precision metric.

Explainable AI (XAI) methods are commonly evaluated with functional metrics such as correctness, which computationally estimate how accurately an explanation reflects the model's reasoning. Higher correctness is assumed to produce better human understanding, but this link has not been tested experimentally with controlled levels. We conducted a user study (N=200) that manipulated explanation correctness at four levels (100

Concept Bottleneck Models (CBMs) are devoted to melanoma classification route predictions through a clinically grounded concept layer, which binds interpretability to concept-label consistency. When a dataset contains concept-level inconsistencies, identical concept profiles mapped to conflicting diagnosis labels create an unresolvable bottleneck that imposes a hard ceiling on achievable accuracy. In this paper, we apply rough set theory to the Derm7pt dermoscopy benchmark and characterize, for the first time, the full extent and clinical structure of this inconsistency. Among 305 unique concept profiles formed by the dermoscopic criteria of the 7-point melanoma checklist, 50 (16.4%) are inconsistent and span 306 images (30.3% of the dataset). This yields a theoretical accuracy ceiling of 92.1% for any hard CBM trained on the full raw data, where over half of all melanoma images carry concept signatures belonging to the boundary region. This disproportionate fraction shows that the checklist concepts are less discriminative for melanoma than for non-melanoma lesions. We characterize the conflict-severity distribution and identify the clinical features most responsible for boundary ambiguity. Two filtering strategies are proposed to remove images with inconsistent concept signatures, both producing a benchmark that we refer to as Derm7pt+. The symmetric strategy yields a fully consistent subset of 705 images, while the asymmetric strategy retains all melanoma images and removes only the conflicting non-melanoma counterparts, producing 841 images. We evaluate a hard CBM across 19 backbone architectures from the EfficientNet, DenseNet, ResNet, and Wide ResNet families on both Derm7pt+ variants. Under symmetric filtering, DenseNet-169 achieves the best test macro F1 of [Formula: see text], and EfficientNet-B4 leads under asymmetric filtering with a test macro F1 of [Formula: see text]. Macro-averaged concept accuracy remains moderate yet stable across all configurations, and this level of concept prediction suffices to produce solid label macro F1 scores. This shows that annotation noise rather than backbone capacity is the binding constraint on bottleneck quality. We further apply the Sparseness-Optimized Feature Importance (SOFI) explainer to the true positive melanoma predictions of the best-performing model on the symmetric Derm7pt+. We found that irregular dots and globules drives every melanoma prediction, while irregular streaks, atypical pigment network, and present blue-whitish veil form a consistent secondary tier.

Interpreting the impedance response of perovskite solar cells (PSCs) is challenging due to the complex coupling of ionic and electronic motion. While drift-diffusion (DD) modelling is a reliable method, its mathematical complexity makes directly extracting physical parameters from experimental data infeasible. This work uses DD modelling to generate a large synthetic dataset of impedance spectra for a standard TiO2/MAPI/spiro configuration. This dataset trains machine learning (ML) models to predict recombination and ionic parameters from impedance measurements. A Gradient Boosting Regressor, using features from a generalized equivalent circuit, showed the best performance. Interpretative analysis indicates that open-circuit impedance experiments best probe recombination losses, while short-circuit conditions are more adequate for extracting ionic features like concentrations and mobilities. The trained ML models were tested on experimental spectra, confirming that the inferred physical parameters could reproduce the data. For the studied configuration, predicted ion concentrations were (1.3-3.3) & times; 101 7 cm- 3, ion mobilities were (5-7) & times; 10- 1 1 cm2V- 1s- 1, and surface recombination velocities were 7-9 and 23-40 ms-1. This approach provides insights into the physical information extractable from impedance measurements and paves the way for ML models to unambiguously derive efficiency-determining parameters for solar cells.

Sparseness Optimized Feature Importance (SOFI) is a post-hoc method that produces explanations using minimal feature sets, reducing the cognitive burden on human experts by highlighting only the most critical factors. In practice, explanations take the form of a ranking of features whose cumulative marginalization leads to rapid degradation in model performance. However, SOFI employs hill climbing for ranking optimization, which increases the risk of convergence to local optima when the number of features grows. In addition, like other mainstream explainers, SOFI lacks a mechanism for exploiting prior knowledge during optimization. In this paper, we propose Sparseness Optimized Feature Importance with Prior Knowledge (SOFI-P), an extension of SOFI that integrates prior knowledge into a reinforcement learning framework to optimize explanation sparsity. In this explainer, the exploration is guided by a probabilistic swapping strategy that maximizes model performance degradation under cumulative feature marginalization. Prior knowledge is incorporated as a learnable parameter vector, initially defined by domain experts and later updated during optimization. In addition, we derive upper bounds on the change in explanation sparsity induced by adjacent and arbitrary swaps in a feature ranking. The proposed theorems provide practical value by establishing concrete limits for expected explanation sparsity post-swapping, thereby characterizing the problem’s search space complexity. Empirical evaluation on 40 structured classification datasets shows that SOFI-P produces more sparse explanations than state-of-the-art explainers. Furthermore, ablation studies confirm the benefits of incorporating prior knowledge to guide reinforcement learning, even when such knowledge is imprecise. Toward the end, a case study on chest X-ray images illustrates the practical applicability of the method.

Long Short-term Cognitive Networks (LSTCNs) are recurrent neural networks for efficient forecasting of univariate and multivariate time series. They consist of a sequential ensemble of Short-term Cognitive Networks (STCNs), each processing a time patch and passing knowledge to the next block. Originally designed to handle long time series, the LSTCNs’ performance on shorter, real-world time series remains unexplored. Besides their efficiency, LSTCNs are deemed interpretable to some extent since both neural concepts and weights have a precise meaning in the domain-specific problem. However, determining feature importance goes beyond inspecting the learned weights and requires more advanced post-hoc methods that operate on the network’s internal representations. In this paper, we present two main contributions that address these experimental and theoretical research gaps. As the first contribution, we propose the Sparseness-Optimized Feature Importance for Time Series Forecasting (SOFI-TSF) explainer to rank neural concepts in LSTCN models according to their relevance for the forecast. While this explainer can be applied to any forecaster, our proposal includes knowledge-specific methods to initialize the feature ranking to be improved by SOFI-TSF’s optimizer. As a second contribution, besides performing a hyperparameter sensitivity analysis, we conducted an extensive comparative analysis contrasting LSTCNs’ performance against other forecasters based on recurrent neural networks, transformers, and hierarchical interpolation. In these studies, we used 25 real-world time series datasets at different prediction horizons and the M5 forecasting competition dataset. The simulations show that (i) all SOFI-TSF variants tested in our simulations largely outperform the baselines in terms of fidelity and explanation sparsity, (ii) LSTCNs outperform the other recurrent forecasters in terms of both forecasting error and runtime.

The literature reports a wide variety of attribution methods for explaining the predictions made by time series classification (TSC) algorithms. These post-hoc explanation methods span from model-specific to agnostic procedures that operate at different granularity levels. Despite their relative success, they often fail to generate sparse explanations, thereby increasing the cognitive overhead for experts seeking to isolate the most relevant features associated with model performance. Another limitation of segment-based explanation methods for TSC problems is that they do not allow any expert intervention. In this paper, we present an agnostic explainer termed Sparseness-Optimized Feature Importance (SOFI) that can be used to explain the predictions generated by any black-box TSC model. In practice, the explanation takes the form of a ranking of time series segments whose cumulative perturbation leads to fast degradation in model performance. Those segments should ideally be provided or defined by experts to ensure that explanations are meaningful and aligned with domain knowledge. As a second contribution, we mathematically demonstrate that under the modularity assumption, the optimal segment ranking associated with SOFI is unique. If the modularity assumption is dropped, we prove that multiple segment importance rankings lead to the optimal model performance degradation. In our experiments, we study the effect of different strategies for computing time series segments and perturbation operators on the explanation results. Simulation results show that SOFI generates explanations that are equally robust, up to 16 times more faithful, and 1.4 times sparser than those generated by the state-of-the-art explainers used for comparison.

Fuzzy Cognitive Maps (FCMs) are a type of recurrent neural network with built-in meaning in their architecture, originally devoted to modeling and scenario simulation tasks. These knowledge-based neural systems support feedback loops that handle static and temporal data. Over the last decade, there has been a noticeable increase in the number of contributions dedicated to developing FCM-based models and algorithms for structured pattern classification and time series forecasting. These models are attractive since they have proven competitive compared to black boxes while providing highly desirable interpretability features. Equally important are the theoretical studies that have significantly advanced our understanding of the convergence behavior and approximation capabilities of FCM-based models. These studies can challenge individuals who are not experts in Mathematics or Computer Science. As a result, we can occasionally find flawed FCM studies that fail to benefit from the theoretical progress experienced by the field. To address all these challenges, this survey paper aims to cover relevant theoretical and algorithmic advances in the field, while providing clear interpretations and practical pointers for both practitioners and researchers. Additionally, we will survey existing tools and software implementations, highlighting their strengths and limitations towards developing FCM-based solutions.

Quasi-Nonlinear Fuzzy Cognitive Maps (q-FCMs) are an algorithmic generalization of Fuzzy Cognitive Maps (FCMs) used for modeling and simulation. The key advantages of q-FCMs include their interpretability and hybrid reasoning capabilities where expert knowledge and historical data can be exploited to build the model. Another distinctive feature of neural cognitive mapping is that it allows the aggregation of different models that represent the same problem into a unified neural system. Unfortunately, existing aggregation algorithms focus on producing an aggregated model that resembles the structure of the individual q-FCMs, while neglecting the functional aspect. The ramification of this oversight is that the simulation results produced by the aggregated model often differ significantly from those generated by the individual models. In this paper, we introduce a parameterized learning-based method for aggregating q-FCMs that considers both aspects. Firstly, it ensures that the aggregated model’s weight matrix is reasonably similar to those associated with the individual models, thus maintaining the structural integrity of the aggregation. Secondly, it ensures that the aggregated model’s outputs closely align with those produced by the individual models when operating under the same initial conditions. The core of our aggregation method lies in an analytically derived loss function that is minimized using a gradient-based optimizer which approximates the Jacobian and Hessian while using a limited amount of memory. Extensive simulations on synthetically generated models and a case study with diverse structural properties and complexities demonstrate that our approach significantly outperforms representative state-of-the-art methods.

Technical and ethical concerns impede the establishment of trust among healthcare professionals (HCPs) in developing artificial intelligence (AI)-based decision support. Yet, our understanding of trust models is constrained, and a standard accepted approach to evaluating trust in AI models is still lacking. We introduce a novel methodology to assess and quantify HCPs' perceived trust in an interpretable machine learning model that serves as clinical decision support for diagnosing COVID-19 cases. Our approach leverages fuzzy cognitive maps (FCMs) to elicit and quantify HCPs' trust mental models for understanding trust dynamics in clinical diagnosis. Our study reveals that HCPs rely predominantly on their own expertise when interacting with the developed interpretable clinical decision support. Although the model's interpretations offer limited assistance in diagnostic tasks, they facilitate the HCPs' utilization of it. However, the impact of these interpretations on the establishment of perceived trust varies among HCPs, which can lead to an increase in trust for some while decreasing it for others. To validate quantified perceived trust, we employ the degree of agreement metric, which quantitatively assesses whether HCPs lean more towards their own expertise or rely on the model's recommendations in diagnostic tasks. We found significant alignment between the conclusions of the two metrics, indicating successful modeling and quantification of perceived trust. Plus, a moderate to strong positive correlation between the two metrics confirmed this conclusion. This means that FCMs can quantify HCPs perceived trust, aligning with their actual diagnostic advice shift after interacting with the model.

This paper introduces a novel zero-data learning algorithm tailored for Fuzzy Cognitive Map (FCM) models utilized in control applications where we must maintain concepts’ activation values within predefined intervals. Our approach allows domain experts to specify these intervals and optionally impose weight constraints, ensuring the algorithm produces feasible models. At the core of our approach lies a mathematical formalism that approximates the smallest feasible activation space for each neural concept, which translates into lower and upper bounds for concepts’ activation values. Moreover, a parameterized quasi-nonlinear reasoning rule allows controlling whether or not the network converges to a unique fixed point. The learning goal of our algorithm narrows down to computing a weight matrix minimizing the error between the analytical bounds and the target intervals specified by domain experts. To address such a constrained minimization problem, we employ numerical methods operating with approximate gradients, which provide highly accurate solutions with short execution times. The main contribution of our learning algorithm is that it does not require any training data to compute the network structure. Therefore, by accurately approximating the specified activation intervals, our learning algorithm guarantees that the outputs produced by the FCM model will remain within these intervals regardless of the initial conditions used to start the recurrent reasoning process.

Distal myopathy represents a genetically heterogeneous group of skeletal muscle disorders with broad clinical manifestations, posing diagnostic challenges in radiology. To address this, we propose a novel multimodal attention-aware fusion architecture that combines features extracted from two distinct deep learning models, one capturing global contextual information and the other focusing on local details, representing complementary aspects of the input data. Uniquely, our approach integrates these features through an attention gate mechanism, enhancing both predictive performance and interpretability. Our method achieves a high classification accuracy on the BUSI benchmark and a proprietary distal myopathy dataset, while also generating clinically relevant saliency maps that support transparent decision-making in medical diagnosis. We rigorously evaluated interpretability through (1) functionally grounded metrics, coherence scoring against reference masks and incremental deletion analysis, and (2) application-grounded validation with seven expert radiologists. While our fusion strategy boosts predictive performance relative to single-stream and alternative fusion strategies, both quantitative and qualitative evaluations reveal persistent gaps in anatomical specificity and clinical usefulness of the interpretability. These findings highlight the need for richer, context-aware interpretability methods and human-in-the-loop feedback to meet clinicians' expectations in real-world diagnostic settings.

Progress in operations research allowed for the widespread use of mathematical optimization in supply chain planning. Despite its numerous practical and economic benefits, human planners often doubt the solutions provided by automated optimizers, which limits their potential effectiveness. Although Explainable Artificial Intelligence (XAI) offers innovative methods to improve the transparency of various models, the tools available to explain optimization algorithms remain underdeveloped. Existing solutions tend to present explanations in numerical formats difficult to interpret. This study explores the application of Large Language Models (LLMs) to enhance the interpretability and persuasiveness of these explanations. Specifically, it investigates whether LLMs can convert numerical explanations into clear, context-aware narratives, thereby fostering greater trust among planners in the optimizer outputs. We worked on top of a supply chain planning optimizer with a LIME-inspired algorithm to generate explanations for typical supply chain scenarios. Explanations generated by LLMs were evaluated using various metrics and compared to the expectations of experienced experts in the field. Our results show that LLMs can substantially improve the clarity and persuasiveness of XAI explanations, increasing human planners' confidence in the optimizer's outputs. We also identify future improvements needed to fully meet the ideal standards set by expert planners.

Quasi-Nonlinear Fuzzy Cognitive Maps (q-FCMs) generalize the classic Fuzzy Cognitive Maps (FCMs) by incorporating a nonlinearity coefficient that is related to the model’s convergence. While q-FCMs can be configured to avoid unique fixed-point attractors, there is still limited knowledge of their dynamic behavior. In this paper, we propose two iterative, mathematically-driven algorithms that allow estimating the limit state space of any q-FCM model. These algorithms produce accurate lower and upper bounds for the activation values of neural concepts in each iteration without using any information about the initial conditions. As a result, we can determine which activation values will never be produced by a neural concept regardless of the initial conditions used to perform the simulations. In addition, these algorithms could help determine whether a classic FCM model will converge to a unique fixed-point attractor. As a second contribution, we demonstrate that the covering of neural concepts decreases as the nonlinearity coefficient approaches its maximal value. However, large covering values do not necessarily translate into better approximation capabilities, especially in the case of nonlinear problems. This finding points to a trade-off between the model’s nonlinearity and the number of reachable states.

The increasing use of artificial intelligence (AI) models across various fields has raised concerns about whether these models can meet user trust expectations. As a result, researchers are focusing on assessing AI models' performance relative to user expectations to determine trust levels. Evidence suggests that effective interaction with eXplainable AI (XAI) techniques can mitigate over-reliance on AI models and better align user expectations with the actual capabilities of these models in decision-making. In this study, we analyze trust from two perspectives: perceived trust, based on user self-reported trust, and demonstrated trust, which evaluates whether users, when given a choice, prefer to rely on AI or make decisions independently. We also explore how different interactions between human subjects and XAI models, along with varying levels of task risk, influence trust. Our findings reveal that these two types of trust are substantially different; human subjects do not always exhibit trust behavior in actual decision-making tasks, even when they perceive themselves as trusting the AI. Furthermore, we show that an AI model's low error rate in making correct decisions can influence human subjects' mental models, leading them to report a higher tendency to trust the AI. Finally, we conclude that human perceptions of trust are fragile and may change based on ongoing interactions with the model.

In this paper, we propose a model-agnostic post-hoc explanation procedure devoted to computing feature attribution. The proposed method, termed Sparseness-Optimized Feature Importance (SOFI), entails solving an optimization problem related to the sparseness of feature importance explanations. The intuition behind this property is that the model's performance is severely affected after marginalizing the most important features while remaining largely unaffected after marginalizing the least important ones. Existing post-hoc feature attribution methods do not optimize this property directly but rather implement proxies to obtain this behavior. Numerical simulations using both structured (tabular) and unstructured (image) classification datasets show the superiority of our proposal compared with state-of-the-art feature attribution explanation methods. The implementation of the method is available on https://github.com/igraugar/sofi.

This paper proposes an algorithm called Forward Composition Propagation (FCP) to explain the predictions of feed-forward neural networks operating on structured classification problems. In the proposed FCP algorithm, each neuron is described by a composition vector indicating the role of each problem feature in that neuron. Composition vectors are initialized using a given input instance and subsequently propagated through the whole network until reaching the output layer. The sign of each composition value indicates whether the corresponding feature excites or inhibits the neuron, while the absolute value quantifies its impact. The FCP algorithm is executed on a post-hoc basis, i.e., once the learning process is completed. Aiming to illustrate the FCP algorithm, this paper develops a case study concerning bias detection in a fairness problem in which the ground truth is known. The simulation results show that the composition values closely align with the expected behavior of protected features. The source code and supplementary material for this paper are available at https://github.com/igraugar/fcp.

Fuzzy Cognitive Maps (FCMs) hold promise as a mathematical tool for modeling and simulating complex systems due to their transparency, flexibility to operate on prior knowledge structures and recurrent reasoning characteristics. However, they suffer from significant shortcomings that have prevented them from being more widely used. Some of these issues include discrepancies in component interpretation, saturation of neural concepts, arbitrary nonlinearities, and dynamic behaviors that are difficult to align with the problem domain. By integrating theoretical advances with practical needs, this paper proposes a revised modeling and simulation methodology termed “neural cognitive mapping” that addresses these issues holistically. Firstly, we redefine concepts’ activation values in terms of changes rather than absolute values, ensuring a unified interpretation of the model’s components. Secondly, we propose a parameterized activation function, called “exponential normalized activator”, which allows experts to control the neurons’ nonlinearities while avoiding saturation states. Furthermore, we provide a twofold reasoning rule that simultaneously computes the concepts’ changes and the amounts of resources attached to problem variables. Thirdly, we introduce a framework for interpreting simulation results across various dynamic behaviors, including scenarios with unique fixed-point attractors. The simulations using both real-world case studies and synthetically generated data illustrate the superiority of our proposal compared with the traditional approach in terms of clarity, usefulness, consistency, and controllability. Moreover, the empirical studies opened new research directions to be explored in future research.

Predictive process monitoring methods predict ongoing case outcomes by analyzing historical process data. Recent studies highlighted the increasing need to enhance the interpretability of these prediction models. This is often achieved by exploiting post-hoc explainable methodologies to assess the importance of different process features on the predicted outcome. However, the significance of the location of process activities on prediction models remains unexplored. In several real-life contexts, there might be potential meaningful relations between the location of the activities and process outcome. This information facilitates insights into process management optimization and decision-making. This paper introduces a novel post-hoc explainable artificial intelligence technique inspired by permutation feature importance to assess the impact of activity locations in predictive models. The experimental results on real-life event logs validate the feasibility of the proposed method, showcasing the influence of the location of (group of) activities on outcome predictions.

In this paper, we integrate the concepts of feature importance with implicit bias in the context of pattern classification. This is done by means of a three-step methodology that involves (i) building a classifier and tuning its hyperparameters, (ii) building a Fuzzy Cognitive Map model able to quantify implicit bias, and (iii) using the SHAP feature importance to active the neural concepts when performing simulations. The results using a real case study concerning fairness research support our two-fold hypothesis. On the one hand, it is illustrated the risks of using a feature importance method as an absolute tool to measure implicit bias. On the other hand, it is concluded that the amount of bias towards protected features might differ depending on whether the features are numerically or categorically encoded.