Ensuring reliability in human-AI collaboration is crucial for fostering appropriate trust in hybrid decision-making systems, which depends not only on predictive performance but also on transparency and awareness of model limitations. Selective classification addresses this need by allowing models to reject uncertain instances and provide predictions only on confident cases. However, existing approaches typically provide little insight into the rationale behind the abstention decisions. In this work, we introduce a novel selective classification method that leverages the distance between an instance and its counterfactuals as a proxy for prediction uncertainty. This formulation naturally enables human-interpretable explanations of the rejection policy, clarifying whether a closed-box predictor is sufficiently stable to issue a decision or should refrain from doing so. The resulting abstention policy is locally interpretable, post-hoc and model-agnostic with respect to the closed-box predictor, and can be flexibly combined with different counterfactual generation methods and distance functions. Extensive experiments on diverse tabular datasets demonstrate that our selective classifier matches or exceeds the performance of state-of-the-art baselines while inherently providing local contrastive explanations for abstention decisions as a byproduct of its local counterfactual analysis.
Predicting cancer drug response from transcriptomic profiles is a cornerstone of precision oncology, yet the scientific value of machine learning models hinges not solely on predictive accuracy, but also on their capacity to generate reliable biological insights. Current explainability approaches in this setting are computationally costly, lack robustness, and reduce complex drug response to univariate gene importance scores, overlooking the coordinated gene activity that drives sensitivity and resistance. In this work, we present ILLUME+, a scalable post-hoc explainability framework that moves beyond single-gene assessments to capture multiple, complementary forms of explanation. Integrated into our end-to-end pipeline, ILLUME+ produces more stable gene importance scores than existing baselines, recovers established drug-gene associations and mechanisms of action, and enables AI-assisted hypothesis generation to uncover novel interaction-driven molecular signals in cancer biology.
Recommender systems shape individual choices through feedback loops in which user behavior and algorithmic recommendations coevolve over time. The systemic effects of these loops remain poorly understood, in part due to unrealistic assumptions in existing simulation studies. We propose a feedback-loop model that captures implicit feedback, periodic retraining, probabilistic adoption of recommendations, and heterogeneous recommender systems. We apply the framework on online retail and music streaming data and analyze systemic effects of the feedback loop. We find that increasing recommender adoption may lead to a progressive diversification of individual consumption, while collective demand is redistributed in model- and domain-dependent ways, often amplifying popularity concentration. Temporal analyses further reveal that apparent increases in individual diversity observed in static evaluations are illusory: when adoption is fixed and time unfolds, individual diversity consistently decreases across all models. Our results highlight the need to move beyond static evaluations and explicitly account for feedback-loop dynamics when designing recommender systems.
The rapid integration of AI systems into high-stakes domains has revealed persistent issues of user distrust, algorithmic aversion, and over-reliance, highlighting the need for decision-making frameworks in which humans and machines synergistically collaborate towards the solution of the task. Hybrid Decision-Making Systems (HDMS) have emerged as a paradigm where humans and AI jointly contribute to the same task, leveraging and integrating human strengths like domain expertise, contextual understanding and flexible reasoning, alongside machines’ computational power. This survey offers a structured overview of learning paradigms for HDMS, with a particular focus on uncertainty-driven abstention mechanisms, which determine when an AI system should act autonomously or when it should call for human intervention. We formalise and compare algorithmic approaches that embed machine learning models with the capacity to “know what they don’t know”, analysing how abstention policies and system architectures integrate human expertise into the decision pipeline. Beyond abstention, we examine frameworks that support direct human–machine interaction during and after the learning process, outlining emerging approaches that foster bidirectional collaboration between humans and AI. Building on this analysis, we propose a taxonomy of three learning paradigms characterising progressively tighter human–machine integration.
AI-assisted decision-making systems are increasingly used across domains, prompting research into design factors that influence their effectiveness. Prior work has examined elements such as explanation type, interface design, and cognitive support interventions; however, the complexity of real-world tasks makes the generalization of findings difficult. In particular, in real-world decision-making settings like Fake News Detection studies on human-AI collaboration remain underexplored. This study investigates how explanation type and system interactivity affect human-AI collaboration in the context of fake news detection. In a controlled online experiment (N = 161), we varied the explanation type and the interactivity of an AI system that assists with the fake news detection task. Results show that conversational systems increased user reliance, while feature importance explanations appeared to be more effective than counterfactual explanations. These findings highlight the importance of explanation design and interaction modality in shaping effective AI-assisted decision-making in real world settings where underlying regularities are difficult for non-expert users to detect.
Human disagreement is ubiquitous and well-known in labeling. However, variation in explanations, captured through token-level human rationales, remains far less explored. At the same time, it is unclear how to best evaluate human labels and rationales – or even how to best aggregate rationales beyond majority vote – in light of this variation. Yet, rationales may provide additional insights into the richness of human reasoning, that may differ in style, values and interpretations – especially in subjective NLP tasks like hate speech detection. In this work, we unify diverse models, training strategies, loss functions, and existing evaluation metrics under a single protocol by systematically re-implementing them across different label and rationale representation spaces. Classification metrics are organized around two key properties – predictive and distributional – while explainability metrics through three complementary dimensions: plausibility, faithfulness, and complexity. In this unified supervision framework, we evaluate model behavior across classification and explainability metrics, as well as metric sensitivity to the choice of label (hard and soft) and rationale representation space (hard, intermediate and soft). Results show that both hard and soft metrics favor softer representations, highlighting their effectiveness in capturing variation and the need to rethink evaluation in subjective NLP.
Large-scale foundation models exhibit behavioral shifts when subjected to interventions such as scaling, fine-tuning, reinforcement learning with human feedback, or in-context learning. Current explainability methods are structurally ill-suited to explain these shifts, because they either treat models as static objects, as traditional eXplainable AI (XAI) approaches do, or merely compare independent explanations across different checkpoints of a model. As a result, these approaches fail to explain the functional transition between two model instances in which a certain behavior has shifted following an intervention. This gap creates significant governance risks across jurisdictions including the EU AI Act, US state legislation, and Chinese AI regulations, which require documenting causal chains for substantial system modifications. This position paper argues that explaining behavioral shifts in large language models requires a principled approach that treats the shift itself as the primary object of explanation: namely, one that explains how and why an intervention transforms a reference model into an updated model with different behavior. To support this claim, we introduce Comparative XAI (XAI_Δ), a novel XAI paradigm aimed at explaining the difference between two model checkpoints where a behavior has shifted, together with a set of desiderata specifying what XAI_Δ explainers and explanations must satisfy, including comparability, validity, actionability, and monitoring, with the goal of grounding model auditing in explicit, measurable requirements. Finally, we provide preliminary evidence suggesting the need for XAI_Δ in practice through illustrative experiments, compiling the resulting findings into a transition report directly usable for governance and incident documentation.
Large Language Models (LLMs) can generate natural language explanations that rationalize their own decisions, a phenomenon commonly referred to as self-explanations.Such explanations have emerged as a promising direction for explainable artificial intelligence (XAI), particularly for interpreting LLM behavior.However, while self-explanations often appear plausible, whether they faithfully reflect a model's underlying reasoning process remains an open question. In this opinion paper, we argue that self-explanations can be highly plausible, questionably faithful, and yet highly actionable. From a traditional XAI perspective, we identify the limitations of standard evaluation protocols for LLM-generated self-explanations and propose practical guidelines for assessing their plausibility and faithfulness. Moreover, we argue that evaluation should extend beyond these criteria to actionability, highlighting applications of LLM rationalization capabilities that support informed decision-making and appropriate action across diverse stakeholders.
Explainability for transformer-based language models is becoming crucial as these systems are deployed in high-stakes applications. As a result, the ecosystem of explainability tools is rapidly evolving, becoming richer, but also more fragmented and harder to navigate. To address this challenge, we present Virgil, an interactive system that lets practitioners and researchers, including non-experts, navigate explainability tools for transformer language models. Supported by a curated knowledge base, the system enables users to discover and compare explainability tools within a unified interface.
Digital labour platforms are reshaping the world of work across a wide range of sectors, offering greater flexibility and accessibility than traditional labour markets. However, existing research suggests that platform work is often associated with low-quality working conditions and may exacerbate inequalities. This study examines the economic and social dimensions of digital platform labour in Italy—a country characterised by labour market fragmentation and the widespread use of non-standard employment—using official survey data collected in 2018 and 2021. Applying advanced machine learning (ML) and explainable artificial intelligence (XAI) techniques, the analysis explores the demographic, occupational, and economic factors that predict participation in platform work and drive segmentation within the platform workforce. The findings reveal that platform work in Italy is a heterogeneous and stratified phenomenon, deeply embedded in longstanding labour market fragmentation and regional disparities. Economic vulnerability is concentrated not among the youngest workers, as often suggested in the literature, but among older or more established individuals facing job instability, underemployment, or declining income from traditional occupations. Moreover, the analysis reveals that platform work is associated with structural vulnerabilities typical of non-standard employment, including unstable contracts, gender inequalities, and economic insecurity, and it primarily functions as a compensatory mechanism to supplement insufficient earnings from precarious jobs. Among jobseekers, engagement with platforms is more likely among younger individuals experiencing moderate—rather than severe—financial strain, suggesting that platform work is not generally perceived as a last-resort strategy but rather as a temporary or adaptive response to limited labour market opportunities. The COVID-19 pandemic further intensified these dynamics, acting as a catalyst for workers experiencing economic and social stress. During this period, platform work expanded as a fallback option for the unemployed, providing an informal buffer amid declining employment opportunities and persistent income insecurity.
Large Language Models (LLMs) excel in language comprehension and generation but are prone to hallucinations, producing factually incorrect or unsupported outputs. Retrieval Augmented Generation (RAG) systems address this issue by grounding LLM responses with external knowledge. This study evaluates the relationship between retriever effectiveness and hallucination reduction in LLMs using three retrieval approaches: sparse retrieval based on BM25 keyword search, dense retrieval using semantic search with Sentence Transformers, and a proposed hybrid retrieval module. The hybrid module incorporates query expansion and combines the results of sparse and dense retrievers through a dynamically weighted Reciprocal Rank Fusion score. Using the HaluBench dataset, a benchmark for hallucinations in question answering tasks, we assess retrieval performance with metrics such as mean average precision and normalised discounted cumulative gain, focusing on the relevance of the top three retrieved documents. Results show that the hybrid retriever achieves better relevance scores, outperforming both sparse and dense retrievers. Further evaluation of LLM-generated answers against ground truth using metrics such as accuracy, hallucination rate, and rejection rate reveals that the hybrid retriever achieves the highest accuracy on fails, the lowest hallucination rate, and the lowest rejection rate. These findings highlight the hybrid retriever's ability to enhance retrieval relevance, reduce hallucination rates, and improve LLM reliability, emphasising the importance of advanced retrieval techniques in mitigating hallucinations and improving response accuracy.
Recommender systems continuously interact with users, creating feedback loops that shape both individual behavior and collective market dynamics. This paper introduces a simulation framework to model these loops in online retail environments, where recommenders are periodically retrained on evolving user-item interactions. Using the Amazon e-Commerce dataset, we analyze how different recommendation algorithms influence diversity, purchase concentration, and user homogenization over time. Results reveal a systematic trade-off: while the feedback loop increases individual diversity, it simultaneously reduces collective diversity and concentrates demand on a few popular items. Moreover, for some recommender systems, the feedback loop increases user homogenization over time, making user purchase profiles increasingly similar. These findings underscore the need for recommender designs that balance personalization with long-term diversity.
In the realm of Natural Language Processing (NLP), common approaches for handling human disagreement consist of aggregating annotators' viewpoints to establish a single ground truth. However, prior studies show that disregarding individual opinions can lead to the side-effect of under-representing minority perspectives, especially in subjective tasks, where annotators may systematically disagree because of their preferences. Recognizing that labels reflect the diverse backgrounds, life experiences, and values of individuals, this study proposes a new multi-perspective approach using soft labels to encourage the development of the next generation of perspective-aware models-more inclusive and pluralistic. We conduct an extensive analysis across diverse subjective text classification tasks including hate speech, irony, abusive language, and stance detection, to highlight the importance of capturing human disagreements, often overlooked by traditional aggregation methods. Results show that the multi-perspective approach not only better approximates human label distributions, as measured by Jensen-Shannon Divergence (JSD), but also achieves superior classification performance (higher F1-scores), outperforming traditional approaches. However, our approach exhibits lower confidence in tasks like irony and stance detection, likely due to the inherent subjectivity present in the texts. Lastly, leveraging Explainable AI (XAI), we explore model uncertainty and uncover meaningful insights into model predictions. All implementation details are available at our github repo.
In subjective tasks like stance detection, diverse human perspectives are often simplified into a single ground truth through label aggregation i.e. majority voting, potentially marginalizing minority viewpoints. This paper presents a Multi-Perspective framework for stance detection that explicitly incorporates annotation diversity by using soft labels derived from both human and large language model (LLM) annotations. Building on a stance detection dataset focused on controversial topics, we augment it with document summaries and new LLM-generated labels. We then compare two approaches: a baseline using aggregated hard labels, and a multi-perspective model trained on disaggregated soft labels that capture annotation distributions. Our findings show that multi-perspective models consistently outperform traditional baselines (higher F1-scores), with lower model confidence, reflecting task subjectivity. This work highlights the importance of modeling disagreement and promotes a shift toward more inclusive, perspective-aware NLP systems.
Counterfactual explanations play a crucial role in interpreting and understanding the decision-making process of complex machine learning models, offering insights into why a particular prediction was made and how it could be altered. However, individual counterfactual explanations generated by different methods may vary significantly in terms of their quality, diversity, and coherence to the black-box prediction. This is especially important in financial applications such as churn analysis, where customer retention officers could explore different approaches and solutions with the clients to prevent churning. The officer's capability to modify and explore different explanations is pivotal to his ability to provide feasible solutions. To address this challenge, we propose an evaluation framework through the implementation of an ensemble approach that combines state-of-the-art counterfactual generation methods and a linear combination score of desired properties to select the most appropriate explanation. We conduct our experiments on three publicly available churn datasets in different domains. Our experimental results demonstrate that the ensemble of counterfactual explanations provides more diverse and comprehensive insights into model behavior compared to individual methods alone that suffer from specific weaknesses. By aggregating, evaluating, and selecting multiple explanations, our approach enhances the diversity of the explanation, highlights common patterns, and mitigates the limitations of any single method, offering to the user the ability to tweak the explanation properties to their needs.
Human-AI coevolution, defined as a process in which humans and AI algorithms continuously influence each other, increasingly characterises our society, but is understudied in artificial intelligence and complexity science literature. Recommender systems and assistants play a prominent role in human-AI coevolution, as they permeate many facets of daily life and influence human choices through online platforms. The interaction between users and AI results in a potentially endless feedback loop, wherein users' choices generate data to train AI models, which, in turn, shape subsequent user preferences. This human-AI feedback loop has peculiar characteristics compared to traditional human-machine interaction and gives rise to complex and often “unintended” systemic outcomes. This paper introduces human-AI coevolution as the cornerstone for a new field of study at the intersection between AI and complexity science focused on the theoretical, empirical, and mathematical investigation of the human-AI feedback loop. In doing so, we: (i) outline the pros and cons of existing methodologies and highlight shortcomings and potential ways for capturing feedback loop mechanisms; (ii) propose a reflection at the intersection between complexity science, AI and society; (iii) provide real-world examples for different human-AI ecosystems; and (iv) illustrate challenges to the creation of such a field of study, conceptualising them at increasing levels of abstraction, i.e., scientific, legal and socio-political.
Franco Turini合作论文数Dipartimento di Informatica;Universit?? di Pisa19
Fabio Pinelli合作论文数ISTI - CNR14
Maurizio Atzori合作论文数KDD Lab11