Explainability in recommender systems (RS) remains a pivotal challenge. Counterfactual explanations have emerged as a particularly actionable paradigm, offering intuitive “what-if” reasoning. However, their evaluation lacks principled standards. Current metrics primarily assess whether explanations change the top-ranked recommendation, overlooking two fundamental aspects of explanation quality. First, evaluation results are frequently inconsistent , as metrics are tightly coupled to the underlying recommender’s performance. Second, explanations are rarely assessed for compactness —whether changes are sufficiently small to remain interpretable. Oversized counterfactuals may technically succeed but fail to provide practical insight. In this work, we advocate for a holistic evaluation perspective centered on consistency and compactness . We systematically analyze how extending evaluation beyond top-1 to top-k recommendations improves metric stability and reduces dependence on recommender fluctuations. In parallel, we introduce compactness-aware evaluation criteria that quantify the minimality of counterfactual modifications. Through extensive experiments across multiple datasets and models, we demonstrate that jointly considering these dimensions yields more reliable assessments. Our findings expose key factors driving metric instability, highlight the trade-off between effectiveness and explanation size, and provide practical guidelines toward standardized, faithful, and compact evaluation of counterfactual explanations in recommender systems.
Sequential recommendation (SR) aims to predict users’ next interactions by modeling contextual patterns in their historical behavior sequences. Recent advancements in SR introduce multi-interest models that use multiple vectors to represent users’ diverse interests. However, existing methods predefine the number of interest vectors as a hyperparameter, limiting their adaptability to real-world scenarios where the number of interests varies across users. Our experiments show that this fixed-number interest representation suffers from a critical limitation: overfitting for users with simple interests, underfitting for users with complex interests, and a lack of interest prioritization. To address these limitations, we propose A daptive i nterest r epresentation learning for multi-interest Rec ommendation (AirRec), a two-stage framework that adaptively adjusts the number of interest representations for each user’s behavior sequence. Specifically, the first stage focuses on interest foundation construction: to compensate for the lack of auxiliary information, we decompose item co-occurrence data to extract atomic interests, which act as the basic units for interest representation modeling. In the second stage, we conduct user-adaptive interest modeling, retaining only atomic interests relevant to the user’s interaction history to filter out noise and infer the actual number of underlying interests. The selected atomic interests are then aggregated into personalized, adaptively sized interest representations for each user sequence. The entire process is designed for parallel computation, guaranteeing efficiency in both training and inference. Experiments on three benchmark datasets demonstrate that AirRec improves NDCG@20 by an average of 14.94% and NDCG@50 by 24.73% compared to state-of-the-art methods. The source code is available at https://github.com/yaokunliu/AirRec.git.
Modern conversational recommender systems excel at technical optimisation but often fail to operationalise the latent experiential dimensions that govern human decision-making. Current architectures predominantly prioritise recommendation accuracy, frequently neglecting the nuanced, context-sensitive behaviours required for meaningful interaction. We introduce Recommendation-as-Experience (RAE), a framework designed to bridge this gap by encoding experiential objectives as adaptive state variables. Through a multi-domain study ( N = 168), we employ Bayesian hierarchical ordinal regression to quantify the interplay between three primary interactional pillars: educative (justification and transparency), explorative (discovery and serendipity), and affective (social and emotional resonance). Our findings demonstrate that domain-specific profiles and perceived item value function as systematic modulators of these priorities. Notably, we identify a strong preference for user autonomy within the tested apparel e-commerce context, suggesting that interactional agency is an important component of the conversational experience, though further cross-domain validation is required for higher-stakes scenarios. Drawing on these empirical insights, the RAE framework formalises the mapping of contextual and individual signals onto structured state representations. These representations drive experience-aligned dialogue policies through retrieval diversification, heuristic constraints, or controllable Large Language Model generation. RAE framework provides a principled approach for balancing predictive precision with high-fidelity experiential quality.
Leveraging Large Language Models (LLMs) for generative recommendation has attracted significant research interest, where item tokenization is a critical step. It involves assigning item identifiers for LLMs to encode user history and generate the next item. Existing approaches leverage either token-sequence identifiers, representing items as discrete token sequences, or single-token identifiers, using ID or semantic embeddings. Token-sequence identifiers face issues such as the local optima problem in beam search and low generation efficiency due to step-by-step generation. In contrast, single-token identifiers fail to capture rich semantics or encode Collaborative Filtering (CF) information, resulting in suboptimal performance. To address these issues, we propose three fundamental principles for item identifier design: 1) integrating both CF and semantic information to fully capture multi-dimensional item information, 2) designing order-agnostic identifiers without token dependency, mitigating the local optima issue and achieving simultaneous generation for generation efficiency, and 3) disentangling semantics across different tokens, unlocking the potential of scaling order-agnostic identifier. Accordingly, we introduce a novel set identifier paradigm, representing each item as a set of order-agnostic tokens. To implement, we propose SETRec, which leverages CF and semantic tokenizers to obtain order-agnostic multi-dimensional tokens. To eliminate token dependency, SETRec uses a sparse attention mask for user history encoding and a query-guided generation mechanism for simultaneous token generation. We instantiate SETRec on T5 and Qwen (from 1.5B to 7B). To reinforce disentanglement between tokens, we propose SETRec++, which introduces two disentanglement strategies, i.e., disentangled regularization loss and token masking mechanism. Experiments on four datasets demonstrate its effectiveness across various scenarios ( e.g., full ranking, warm- and cold-start ranking, and various item popularity groups). Moreover, results validate SETRec’s superior efficiency and scalability on cold-start items as model sizes increase, and SETRec++’s potential in the scalability of order-agnostic identifier.
Human-in-the-loop experiments have been increasingly adopted in AI research. Nevertheless, they usually require elevated time/monetary costs and ethical considerations. Hence, developing user behaviour simulators to synthesise reliable and realistic human behaviours has emerged as an attractive alternative. Although various simulators have been proposed, there lacks a comprehensive experimental study that systematically benchmarks these simulators under a unified evaluation framework. In this paper, we fill this gap by comparing two mainstream simulators experimentally and comprehensively: reinforcement learning (RL) environments and large language models (LLMs) based, focusing on how they respond to different data settings, e.g., data sparsity and application domains. Our extensive experimental results reveal several key findings. First, LLM-based simulators demonstrate stronger performance in general, highlighting their significant potential to serve as an underpinning component of future works. However, they are found to be more sensitive to data density and biased towards positive interactions: their performance on sparse and negative bias conditions on datasets like Book Crossing drops significantly by 44% to 58% despite their strong performances on denser datasets. RL environment simulators are more effective in capturing users’ long-term preference profiles, evidenced by their capability in modelling the user conformity phenomenon over time. Finally, LLM-based simulators tend to overemphasise positive feedback, often overlooking negative signals compared to RL environment simulators.
Recommender systems are among the most popular artificial intelligence applications. Ensuring that recommendations not only support improving business-oriented performance indicators but also comply with existing domain-specific Ethical, Legal, and Societal Aspects (ELSA) is paramount. This article introduces an ELSA-compliant Explainable Recommender System, an architecture for implementing ELSA-compliant recommendations. Our proposed solution assumes that information regarding items, users, their interactions, and ELSA is encoded in Knowledge Graphs (KG). Given recommendations provided by a base off-the-shelf KG-based recommendation system, our system re-ranks and filters out recommendable items based on their ELSA-compliance, encoded in pre-defined domain-specific ELSA constraints. Our approach includes an explainer that leverages Large Language Models to provide explanations of the adequacy of recommended items based on ELSA. The proposed components have been assessed in the movie and animal treatment recommendation tasks. Experimental results suggest that the proposed solution effectively ensures ELSA-compliant recommendations with adequate humanized explanations in natural language to the end user.
Quantum annealers offer a promising hardware platform for solving combinatorial optimization problems, especially those formulated as Quadratic Unconstrained Binary Optimization (QUBO). In this work, we propose PDQUBO ( P erformance- D riven Q uadratic U nconstrained B inary O ptimization), a QUBO-based feature selection method, that is, directly executable on quantum annealers. Unlike prior QUBO-based feature selection approaches on quantum annealers, PDQUBO explicitly quantifies the performance impact of both individual features and feature pairs on recommender system models. This alignment between QUBO optimization objectives and model performance ensures that the solution direction is closely tied to recommendation quality, making it well-suited for practical deployment on quantum hardware. Moreover, by leveraging counterfactual analysis, PDQUBO is model-agnostic and evaluation-metric-independent, making it broadly applicable across diverse recommender architectures and assessment criteria. In addition, we investigate the instability of quantum annealing on real quantum devices with respect to varying problem sizes and problem difficulties. Extensive experiments on real-world datasets demonstrate that PDQUBO consistently outperforms prior QUBO-based feature selection methods on quantum annealers. Furthermore, we compare PDQUBO against classical feature selection baselines on click-through rate (CTR) prediction tasks, showing its strong performance and highlighting the potential of using quantum annealers for real-world feature selection applications. Our findings suggest that integrating quantum optimization with counterfactual analysis provides a promising direction for effective feature selection in recommender systems.
Recommender systems guide users through vast amounts of data, yet recommendations can feel opaque and difficult to trust. Prior research shows that explanations help users become more confident and satisfied, and that explanation effectiveness depends not only on what is said but also on how it is communicated. This work investigates personality-aligned framings of explanations in a music recommender system. Using generative AI, we produce natural language explanations aligned with the Big Five personality traits alongside a feature-based baseline and examine whether linguistic framing shapes user perceptions beyond personalization. In a controlled study with N = 348 participants, users completed a Big Five inventory and evaluated music recommendations presented with six framing conditions. Results show that linguistic framing influences how users perceive recommendations, with some framings perceived as more persuasive, clearer, or more accurate than others. Effects vary across personality traits, but aligning framing with a user’s dominant trait does not consistently improve perceptions, while emotionally stable framings are broadly effective. These findings suggest that linguistic framing is a meaningful design parameter for recommender explanations, and that domain fit and emotional appropriateness, rather than user-trait matching, are the operative factors. We discuss implications for adaptive explanation design using generative AI.
This resource article presents Informfully, an open source software project for content distribution to support user studies. It is a free resource for the research community to help conduct online studies investigating the impact of item recommendations on users’ consumption behavior. Informfully allows for the push of text, image, audio, and video content to users and automatically generates a detailed log of their consumption history. It features three core components: (1) a front end for displaying and interacting with recommended items, (2) a back end for researchers to create and maintain user experiments, and (3) a simple JSON-based exchange format for ranked item recommendations to interface with third-party frameworks. We provide a system overview and outline the platform’s three core components. A sample workflow is presented for conducting field studies that incorporate multiple user groups, personalize recommendations, and measure the impact of algorithms on user engagement.
Large Language Models (LLMs) offer a promising approach to recommendation by enabling the generation of user profiles in Natural Language (NL) form. When used as summarization devices, LLMs can produce interpretable and editable alternatives to opaque collaborative filtering representations, potentially increasing transparency and user control. However, it remains unclear whether users perceive these profiles as accurate representations of their preferences, which is key for trust and usability. Moreover, because LLMs inherit societal and data-driven biases, profile quality may systematically vary across user and item characteristics. In this paper, we investigate these issues in the context of music streaming, where personalization is challenged by large and culturally diverse catalogs. We conduct a user study in which participants evaluate NL profiles generated from their own listening histories. We analyze whether user identification with these profiles is biased by user attributes, such as mainstreamness and taste diversity, and by item features, including genre and country of origin. We further assess the usefulness of the generated profiles in a downstream recommendation task by analyzing their representations in a shared embedding space. Our results reveal systematic differences across models and user groups, highlighting both the potential and the limitations of scrutable, LLM-based profiling for personalized systems.
The application of offline reinforcement learning (RL) in recommendation systems has proven valuable in improving user engagement. However, a significant challenge in real-world scenarios is the delayed nature of user feedback, which can degrade algorithm performance and complicate the task of reducing user churn, i.e. losing clients to competitors. In this paper, we present an end-to-end framework called Churn Reduction Agent-Based (CRAB), which is specifically designed to efficiently incorporate delayed feedback into the recommendation process. CRAB uses a conservative Q-learning model to recommend offers to customers, incorporating delayed churn feedback directly into the reward function of the RL model. This allows the RL agent to optimize recommendations not only based on immediate acceptance probabilities, but also considering long-term user retention. Our approach also uses additional auxiliary model predictions to improve the representation of customer states in the RL environment, which significantly improves the framework’s performance. We validate our method using large-scale proprietary data from a real-world telecommunication company, addressing the complexities of scalability and practical implementation. Extensive offline experiments show that CRAB outperforms existing methods, including both internal company policies and state-of-the-art approaches. In addition, our online testing results confirm the framework’s effectiveness in delivering superior recommendations and effectively reducing user churn by 4.95%.
Recommender systems play an important role in our digital lives, affecting our choices in areas such as e-commerce, music, movies, and news. As these systems become more complex, it is crucial to understand how and why items are recommended. Providing explanations in recommender systems can help users better understand these recommendations, which can increase trust, satisfaction, and engagement. However, it is challenging to design and present these explanations effectively so that users can easily understand them. Many studies have investigated user preferences on how to design explanations, but there is a lack of a comprehensive review that brings all these findings together. In this systematic review, we collected and analyzed many studies that explore how to design good explanations in recommender systems based on user opinions. Our review covers various fields to reflect different user experiences. The paper starts by defining key concepts related to explanations in recommender systems, different types of explanations, and research directions. The PRISMA methodology, which was used to review the last 10 years of research reporting user perceptions of the explanations of recommendation systems, is presented. The systematic review results in essential factors that should be considered when designing explanations, as they affect user satisfaction and understanding. Finally, future research directions are suggested to improve the explanation design in recommender systems.
Emotion-aware recommender systems have received a lot of interest in recent years because of their ability to improve the user experience by adapting recommendations to users’ emotional states. This review article conducts a thorough examination of emotion-aware recommender systems, categorizing them according to the datasets used (video, text, audio, image, and physiological), the application domains (movie, music, social media, and news), the emotion categorization and the methodology used. We investigate multiple issues facing these systems, such as accurately detecting and interpreting emotions, integrating multimodal data, and ethical concerns about user privacy and emotional manipulation. In addition, we explore potential future directions for emotion-aware recommender systems.
News organizations must balance diverse journalistic exposure with user engagement, yet text similarity-based recommendation—widely deployed for privacy and cold-start advantages—remains poorly understood from a user experience perspective. We propose bisociative pivoting, which algorithmically constructs asymmetric similarity through dimensional decomposition: maintaining high similarity in one content dimension (anchor) while introducing dissimilarity in another (pivot). This creates cognitive scaffolding that may facilitate positive user responses to greater semantic distance. A user study (n=732) examining Dutch news recommendations confirms that semantic distance significantly reduces evaluation, engagement, and adoption attitude. However, bisociative pivoting substantially attenuates these penalties for item-level outcomes (36–59% reduction), though not for system-level attitude. By grounding algorithmic design in bisociation theory—how users cognitively process tension between connection and separation—we provide initial evidence that dimensional decomposition can substantially reduce the engagement costs of semantic distance. These findings suggest that bisociative pivoting warrants further investigation as an approach for managing the engagement-diversity tension in text similarity-based news recommendation.
Recommender systems exert a profound influence on digital information consumption. Although they excel at content personalisation, recommenders are increasingly investigated for disseminating irrelevant, unwanted, or harmful suggestions. Such exposure diminishes user satisfaction and exacerbates broader societal challenges, including the spread of misinformation, radicalisation, and the systemic erosion of trust. While existing platforms provide tools to limit the disclosure of problematic content, these mechanisms often lack efficacy and fail to adapt dynamically to user input. This article presents an intuitive, model-agnostic, and distribution-free framework that employs conformal risk control to provide statistical guarantees regarding unwanted content in personalised feeds. By leveraging simple feedback, our method ensures that the risk of encountering undesirable items is provably bounded. Furthermore, we address a common limitation of traditional conformal risk control, whereby risk mitigation often yields overly restrictive or smaller recommendation sets. We overcome this by incorporating implicit feedback from consumed items to expand the set of recommendations while maintaining robust risk constraints. Experimental evaluation conducted on datasets from two different real-world scenarios ( i.e., a short-video platform and a music streaming service) confirms that our approach achieves a controllable and effective reduction of unwanted recommendations in diverse domains with minimal computation overhead.
Very Large Online Platforms (VLOPs) use personalized recommender systems to optimize their main performance metric: attention-based user engagement. In doing so, these systems might however amplify systemic risks by promoting controversial or polarizing content, thereby exacerbating issues such as misinformation, societal polarization, and the manipulation of civic discourse. To mitigate these risks, regulations such as the European Union’s Digital Services Act (DSA) mandate increased data access and transparency, including for the auditing of personalized recommender systems. However, the data access provided by VLOPs remains limited—often restricted to specific user demographics, aggregate statistics, or curated datasets—hindering meaningful oversight. Consequently, new methods are needed to audit recommender systems effectively at the user level. In this paper, based on an analysis of the legal context and technical alternatives for data access, we present SOAP, the System for Observing and Analyzing Posts. SOAP is an open-source framework for auditing recommender systems using sock-puppet accounts. It enables fine-grained user-level analysis beyond the constrained data access typically provided by platforms. We detail SOAP’s technical implementation and evaluate its ability to scrutinize systemic risks. Additionally, we tested SOAP in a workshop with over 100 participants and observed a measurable increase in participants’ algorithmic literacy. This demonstrates SOAP’s potential not only for research and regulatory auditing, but also as an educational framework to foster public awareness of algorithmic influence.
Recent work on fairness-aware graph collaborative filtering (GCF) has shown the effectiveness of graph augmentation as a post-processing strategy for mitigating consumer unfairness. However, most studies remain confined to binary fairness setups and operate under limited experimental conditions, often relying on sparse and small-scale datasets. In this article, we extend our fairness-aware augmentation method to address intersectional unfairness across demographic subgroups, a setting where the intersection of multiple sensitive attributes leads to fine-grained subgroups. To this end, we reformulate the fairness objective to incorporate intersectional demographic groups and evaluate our extended method across interaction configurations that vary in density and scale. Our results reveal that the effectiveness of fair graph augmentations is model-dependent and sensitive to dataset properties. We show that the edges selected during augmentation tend to concentrate around interpretable structural patterns driven by the connected nodes’ characteristics. Furthermore, analyzing how these augmented edges differ across graph-level attributes offers actionable insights into the potential benefits of fairness-oriented graph modifications. Finally, we compare our method with recent fairness-aware baselines, explore the impact of augmenting different graph regions, and assess our mitigation strategy under scenarios with minimal unfairness. Source code publicly available at: https://github.com/jackmedda/Intersectional-FA4GCF .
In recent years, environmental sustainability has emerged as a critical issue for society, prompting governments, local authorities, and private organizations to promote initiatives that encourage more sustainable lifestyles. One such initiative is the AWorld application, which leverages gamification to educate, raise awareness, and inspire positive behavioral changes. The app engages users through personalized challenges, or missions, that require interaction with its features and reward participants with in-game points. In this article, we present a hierarchical Multi-Armed Bandit (MAB) framework, MABTree, designed to replace AWorld’s current random mission assignment with a targeted and adaptive approach guided by a custom reward function. Using real-world interaction data from AWorld, we evaluated MABTree in an offline setting across multiple policies and configurations. Results show that MABTree improves cumulative reward, user coverage, and mission diversity compared to classical MAB baselines, with particularly strong gains in balancing exploration of new features and exploitation of known user preferences. While these findings indicate the potential of MABTree to enhance personalization and engagement in sustainability-focused applications, further online testing is needed to validate its real-world impact on long-term user behavior change.
Personalized academic support plays a critical role in improving student engagement, reducing dropout rates, and enhancing the institutional reputation of universities. Although tutoring and mentoring services are available, students still lack access to an effective and timely course recommender system to guide their academic planning. This study aims to address that need by proposing a sequential recommender system trained on historical academic records from a prominent European university’s student portal. We develop and evaluate various state-of-the-art sequential recommendation models and enhance them by integrating exam grades as explicit feedback. Additionally, we introduce two re-ranking strategies that ( i ) increase recommendation diversity and ( ii ) amplify the influence of successful academic trajectories. Results from both offline and online experiments show that our best-performing method effectively anticipates relevant course choices, demonstrating the value of incorporating temporal patterns into the recommendation model.
Sequential recommender systems are an important and in-demand area of research. These systems aim at using the order of interactions in a user’s history to predict future interactions. The premise is that the order of interactions and sequential patterns play an essential role. Therefore, it is crucial to use datasets that exhibit a sequential structure to evaluate sequential recommenders properly. We apply several methods based on random shuffling of user interaction sequences to assess the strength of sequential structure across 19 datasets, most of which are frequently used for evaluating sequential recommenders in recent top-tier conference articles. Since shuffling explicitly breaks sequential dependencies, we estimate their strength by comparing metrics for shuffled and original versions of the dataset. We propose three approaches for this assessment: a sequential rule method and two more advanced model-based approaches. The proposed methods are easily reproducible and consistent with each other. We also examine the impact of common preprocessing and postprocessing techniques, showing that they may affect conclusions about the presence of sequential structure. Finally, we introduce a method for distinguishing between recency-based and order-sensitive sequential patterns and for assessing their complexity. Our findings show that several popular datasets exhibit rather weak sequential structure.
