Federated learning (FL) has emerged as a collaborative approach that allows multiple clients to jointly learn a machine learning model without sharing their private data. The concern about privacy leakage, albeit demonstrated under specific conditions [1], has triggered numerous follow-up research in designing powerful attacking methods and effective defending mechanisms aiming to thwart these attacking methods. Nevertheless, privacy-preserving mechanisms employed in these defending methods invariably lead to compromised model performances due to a fixed obfuscation applied to private data or gradients. In this article, we, therefore, propose a novel adaptive obfuscation mechanism, coined FedAdOb, to protect private data without yielding original model performances. Technically, FedAdOb utilizes passport-based adaptive obfuscation to ensure data privacy in both horizontal and vertical federated learning settings. The privacy-preserving capabilities of FedAdOb, specifically with regard to private features and labels, are theoretically proven through Theorems 1 and 2. Furthermore, extensive experimental evaluations conducted on various datasets and network architectures demonstrate the effectiveness of FedAdOb by manifesting its superior trade-off between privacy preservation and model performance, surpassing existing methods.

Recent developments have demonstrated the efficacy of enhancing the reliability and reasoning ability of Large Language Models (LLMs) through the integration of Knowledge Graphs (KGs). However, previous works have not fully considered the challenges present in expert domains, including the prevalence of professional terms with abbreviations and ambiguities, knowledge confusion stemming from the mixture of KGs with other retrieval methods, and others, which hinder their deployment in expert question-answering (QA) systems. In this work, we propose a QA framework for expert fields by Synergizing large Language models And knowledge Graphs (SLAG). SLAG is designed to improve the accuracy of KG retrieval and to intelligently coordinate KG retrievers with other retrieval methods through bidirectional enhancement between LLMs and KGs: (1) An LLM-Enhanced KG retriever is proposed to resolve entity ambiguities in queries and to accurately extract pertinent subgraphs. (2) A KG-Enhanced LLM reasoner is designed to distill subgraphs and to flexibly generate answers or explicit queries for cooperative retrievers. We compared SLAG with other retrieval techniques on multiple datasets across diverse expert domains, tasks, and languages. We observed that our approach achieved a relative improvement of 13% over state-of-the-art methods on public benchmarks in terms of the F1 score. Moreover, we successfully deployed SLAG in an industrial financial QA system and outperformed the latest online version. Finally, we provide an open source version of SLAG with code and datasets, which is available at https://anonymous.4open.science/r/SLAG-5611.

Recommender systems have become increasingly influential in shaping user behavior and decision-making, highlighting their growing impact in various domains. Meanwhile, the widespread adoption of machine learning models in recommender systems has raised significant concerns regarding user privacy and security. As compliance with privacy regulations becomes more critical, there is a pressing need to address the issue of recommendation unlearning, i.e., eliminating the memory of specific training data from the learned recommendation models. Despite its importance, traditional machine unlearning methods are ill-suited for recommendation unlearning due to the unique challenges posed by collaborative interactions and model parameters. This survey offers a comprehensive review of the latest advancements in recommendation unlearning, exploring the design principles, challenges, and methodologies associated with this emerging field. We provide a unified taxonomy that categorizes different recommendation unlearning approaches, followed by a summary of widely used benchmarks and metrics for evaluation. By reviewing the current state of research, this survey aims to guide the development of more efficient, scalable, and robust recommendation unlearning techniques. Furthermore, we identify open research questions in this field, which could pave the way for future innovations not only in recommendation unlearning but also in a broader range of unlearning tasks across different machine learning applications.

Federated Learning (FL) facilitates collaborative training of a global model whose performance is boosted by private data owned by distributed clients, without compromising data privacy. Yet the wide applicability of FL is hindered by the entanglement of data distributions across different clients. This paper demonstrates for the first time that by disentangling data distributions, FL can in principle achieve efficiencies comparable to those of distributed systems, requiring only one round of communication. To this end, we propose a novel FedDistr algorithm, which employs diffusion models to decouple and recover data distributions. Empirical results on the CIFAR100, DomainNet, OfficeHome, and ISIC2020 datasets show that FedDistr significantly enhances model utility and efficiency in both disentangled and near-disentangled scenarios while ensuring privacy, outperforming traditional federated learning methods.

Federated fine-tuning of Large Language Models (LLMs) is obstructed by a trilemma of challenges: protecting LLMs intellectual property (IP), ensuring client privacy, and mitigating performance loss on heterogeneous data. Existing methods like Offsite-Tuning (OT) secure the LLMs IP by having clients train only lightweight adapters, yet our analysis reveals they suffer from a fundamental performance bottleneck, leaving a significant gap compared to centralized training. To bridge this gap, we introduce FedProxy, a new federated adaptation framework. FedProxy replaces weak adapters with a unified, powerful Proxy Small Language Model (SLM), compressed from the proprietary LLM, to serve as a high-fidelity surrogate for collaborative fine-tuning. Our framework systematically resolves the trilemma through a three-stage architecture: (i) Efficient Representation via server-guided compression to create a resource-friendly proxy; (ii) Robust Optimization through an interference-mitigating aggregation strategy to handle data heterogeneity; and (iii) Effortless Fusion via a training-free "plug-in" mechanism to integrate learned knowledge back into the LLM. Experiments show FedProxy significantly outperforms OT methods and approaches centralized performance, establishing a new benchmark for secure and high-performance federated LLM adaptation.

Although deep research agents (DRAs) have emerged as a promising paradigm for complex information synthesis, their evaluation remains constrained by ad hoc empirical benchmarks. These heuristic approaches do not rigorously model agent behavior or adequately stress-test long-horizon synthesis and ambiguity resolution. To bridge this gap, we formalize DRA behavior through the lens of category theory, modeling deep research workflow as a composition of structure-preserving maps (functors). Grounded in this theoretical framework, we introduce a novel mechanism-aware benchmark with 296 questions designed to stress-test agents along four interpretable axes: traversing sequential connectivity chains, verifying intersections within V-structure pullbacks, imposing topological ordering on retrieved substructures, and performing ontological falsification via the Yoneda Probe. Our rigorous evaluation of 11 leading models establishes a persistently low baseline, with the state-of-the-art achieving only a 19.9\% average accuracy, exposing the difficulty of formal structural stress-testing. Furthermore, our findings reveal a stark dichotomy in the current AI capabilities. While advanced deep research pipelines successfully redefine dynamic topological re-ordering and exhibit robust ontological verification -- matching pure reasoning models in falsifying hallucinated premises -- they almost universally collapse on multi-hop structural synthesis. Crucially, massive performance variance across tasks exposes a lingering reliance on brittle heuristics rather than a systemic understanding. Ultimately, this work demonstrates that while top-tier autonomous agents can now organically unify search and reasoning, achieving a generalized mastery over complex structural information remains a formidable open challenge.\footnote{Our implementation will be available at https://github.com/tzq1999/CDR.

One important direction of Federated Foundation Models (FedFMs) is leveraging data from small client models to enhance the performance of a large server‑side foundation model. Existing methods based on model level or representation level knowledge transfer either require expensive local training or incur high communication costs and introduce unavoidable privacy risks. We reformulate this problem as a reinforcement learning style evaluation process and propose FedGRPO, a privacy preserving framework comprising two modules. The first module performs competence-based expert selection by building a lightweight confidence graph from auxiliary data to identify the most suitable clients for each question. The second module leverages the “Group Relative” concept from the Group Relative Policy Optimization (GRPO) framework by packaging each question together with its solution rationale into candidate policies, dispatching these policies to a selected subset of expert clients, and aggregating solely the resulting scalar reward signals via a federated group–relative loss function. By exchanging reward values instead of data or model updates, FedGRPO reduces privacy risk and communication overhead while enabling parallel evaluation across heterogeneous devices. Empirical results on diverse domain tasks demonstrate that FedGRPO achieves superior downstream accuracy and communication efficiency compared to conventional FedFMs baselines.

Large Language Models (LLMs) increasingly leverage Federated Learning (FL) to utilize private, task-specific datasets for fine-tuning while preserving data privacy. However, while federated LLM frameworks effectively enable collaborative training without raw data sharing, they critically lack built-in mechanisms for regulatory compliance like GDPR's right to be forgotten. Integrating private data heightens concerns over data quality and long-term governance, yet existing distributed training frameworks offer no principled way to selectively remove specific client contributions post-training. Due to distributed data silos, stringent privacy constraints, and the intricacies of interdependent model aggregation, federated LLM unlearning is significantly more complex than centralized LLM unlearning. To address this gap, we introduce Oblivionis, a lightweight learning and unlearning framework that enables clients to selectively remove specific private data during federated LLM training, enhancing trustworthiness and regulatory compliance. By unifying FL and unlearning as a dual optimization objective, we incorporate 6 FL and 5 unlearning algorithms for comprehensive evaluation and comparative analysis, establishing a robust pipeline for federated LLM unlearning. Extensive experiments demonstrate that Oblivionis outperforms local training, achieving a robust balance between forgetting efficacy and model utility, with cross-algorithm comparisons providing clear directions for future LLM development.

Applying large pre-trained Vision-Language Models to recommendation is a burgeoning field, a direction we term Vision-Language-Recommendation (VLR). Bringing VLR to user-oriented on-device intelligence within a federated learning framework is a crucial step for enhancing user privacy and delivering personalized experiences. This paper introduces FedVLR, a federated VLR framework specially designed for user-specific personalized fusion of vision-language representations. At its core is a novel bi-level fusion mechanism: The server-side multi-view fusion module first generates a diverse set of pre-fused multimodal views. Subsequently, each client employs a user-specific mixture-of-expert mechanism to adaptively integrate these views based on individual user interaction history. This designed lightweight personalized fusion module provides an efficient solution to implement a federated VLR system. The effectiveness of our proposed FedVLR has been validated on seven benchmark datasets.

The integration of Foundation Models (FMs) with Federated Learning (FL) presents a transformative paradigm in Artificial Intelligence (AI). This integration offers enhanced capabilities, while addressing concerns of privacy, data decentralization and computational efficiency. This paper provides a comprehensive survey of the emerging field of Federated Foundation Models (FedFM), elucidating their synergistic relationship and exploring novel methodologies, challenges, and future directions that the FL research field needs to focus on in order to thrive in the age of FMs. A systematic multi-tiered taxonomy is proposed, categorizing existing FedFM approaches for model training, aggregation, trustworthiness, and incentivization. Key challenges, including how to enable FL to deal with high complexity of computational demands, privacy considerations, contribution evaluation, and communication efficiency, are thoroughly discussed. Moreover, this paper explores the intricate challenges of communication, scalability and security inherent in training/fine-tuning FMs via FL. It highlights the potential of quantum computing to revolutionize the processes of training, inference, optimization and security. This survey also introduces the implementation requirement of FedFM and some practical FedFM applications. It highlights lessons learned with a clear understanding of our findings for FedFM. Finally, this survey not only provides insights into the current state and challenges of FedFM, but also offers a blueprint for future research directions, emphasizing the need for developing trustworthy solutions. It serves as a foundational guide for researchers and practitioners interested in contributing to this interdisciplinary and rapidly advancing field.

The ever-growing threat of deepfake technology has stimulated research in deepfake forensics to identify the genuineness of media materials on the Internet. The majority of existing techniques employ the convolutional neural network (CNN) framework to identify deepfake by capturing the image’s local features. However, due to the limited scope of receptive fields, relying solely on the feature extraction capabilities of CNNs is insufficient to meet the detection demands posed by increasingly diverse forged samples in real-world scenarios. To address these challenges, this study introduces an innovative model for deepfake identification, leveraging the strengths of EfficientNet and vision transformers to integrate local and global features. This integration allows for more comprehensive learning of deep image features, thereby enhancing detection accuracy. To further enhance detection capabilities and bolster the resilience of our approach against image compression, we designed a dual-stream architecture that incorporates noise features extracted from steganalysis rich model filters as an additional input. By integrating multiple information streams, the model’s performance is effectively improved. From the experiments, the proposed model achieved high accuracy rates of 0.845, 0.999, and 0.815 on the video test sets FaceForensics, Celeb-DF-V2, and Wild Data, respectively. It also attained the best results on the photo test sets FaceApp, FaceSwapApp, and FaceFuse, with scores of 0.998, 0.925, and 0.918. Notably, the model demonstrated strong performance across various deepfake datasets, especially those sourced from social networks. With 88M parameters compared to 101M in the next-best competing method, the proposed model confirms its effectiveness in real-world case studies while maintaining reduced computational complexity.

Transfer learning, which leverages knowledge from source domains to improve model performance in a target domain, has witnessed significant growth. Commonly, transfer learning methods assume identical feature and label spaces in both source and target domains—an approach known as homogeneous transfer learning. However, this assumption is often impractical, as domains frequently differ in their feature or label spaces, making precise data alignment both challenging and resource-intensive. Consequently, heterogeneous transfer learning (HTL), which addresses these disparities, has become an essential strategy in various tasks. In this paper, we provide an extensive review of over 60 HTL methods, covering both data-based and model-based approaches. We detail their key assumptions and algorithms, systematically categorizing them into instance-based, feature representation-based, parameter regularization, and parameter tuning techniques. We also conduct experiments to compare a few open-source HTL methods on four benchmarks and release the corresponding experimental scripts through GitHub. Furthermore, we examine applications in natural language processing, computer vision, multimodal learning, and biomedicine, with the aim of enhancing understanding and fostering further research in these areas. Our review incorporates recent advancements in HTL, including the adoption of transformer-based models and multimodal learning techniques, ensuring the discussion reflects the latest developments in the field. Additionally, we identify key limitations in current HTL research and provide systematic guidance for future investigations, highlighting areas that require further exploration and suggesting potential directions to advance the field.

Causal reasoning, which encompasses the discovery of causal structures and the inference of causal effects, is fundamental to data-driven decision making. In practice, data for reliable causal analysis are often distributed across institutions and cannot be centralized due to privacy regulations or communication constraints. Federated learning (FL) addresses this by enabling collaborative analysis without raw data sharing, giving rise to the rapidly growing field of federated causal discovery (FCD) and inference (FCI). However, the interdisciplinary nature of this field and the absence of a comprehensive survey present barriers to entry for researchers. This paper bridges that gap by providing a systematic review through multi-dimensional taxonomies. Grounded in the three core design decisions underlying any FCD solution, namely how structures are learned, how data are partitioned, and what structural knowledge each party obtains, we organize FCD along three axes: methodological paradigm, federation topology, and structural scope. We further examine key practical dimensions, including temporal dynamics, data heterogeneity, missing data, and non-identical variable sets. For FCI, we categorize methods by target estimand (average versus individualized/conditional treatment effects) and by estimation strategy, from classical weighting methods to modern deep generative architectures. Unlike prior works that treat FCD and FCI separately, we formalize their connection as complementary stages of a unified federated causal reasoning pipeline, where FCD supplies the structural knowledge required for valid effect estimation in FCI. Finally, we highlight their shared concerns regarding privacy, communication efficiency, theoretical guarantees, and application domains, and conclude by identifying open challenges for future research.

Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among clients. Unlike traditional parameter-based FL methods that exchange model weights or gradients during training, emerging logit-based FL approaches share model outputs (logits) on public data. This strategy promotes model heterogeneity, reduces communication overhead, and enhances clients' privacy. However, the potential privacy risks associated with these logit-based methods have been largely overlooked. This research presents the first theoretical and empirical analysis of a hidden privacy risk in logit-based FL methods - the risk that a semi-honest server (adversary) may learn clients' private models from logits. To quantify and address this threat, we develop the Adaptive Model Stealing Attack (AdaMSA) by leveraging historical logits during training. Notably, we observe that this inherent privacy risk persists even when public data is unrelated to private data, emphasizing the urgency to address privacy vulnerabilities in logit-based FL methods. Moreover, our theoretical analysis establishes the bounds of this privacy risk. We then propose a simple but effective defense strategy that perturbs the transmitted logits in the direction that minimizes the privacy risk while maximally preserving the training performance. The experimental results validate our analysis and demonstrate the effectiveness of AdaMSA and our defense strategy.

Federated Learning (FL) enables the collaborative training of a global model while protecting participants’ privacy. Evaluating each participant’s contribution is essential to providing a high-quality model, ensuring fairness, and mitigating potential biases. Most existing contribution evaluation approaches for FL assume that the server has a public validation dataset. However, it is almost impossible to obtain a validation dataset due to privacy concerns. In this article, we propose a Federated Privacy-preserving Representation Synthesis (FedPRS) framework to synthesize a validation dataset for contribution evaluation. The proposed FedPRS framework first transforms each participant’s private validation dataset into its representation. Then, a random-region desensitization strategy is developed to further desensitize the dataset without compromising its utility. The desensitized representation dataset of each participant is collected by the server to evaluate federated contribution, which considers both equity and privacy protection. Moreover, we instantiate and integrate three specific contribution evaluation approaches in this framework. We perform experiments on various FL settings, including independently identically distributed (IID) and non-IID data distributions. Experimental results demonstrate that the contribution evaluation results obtained using the validation dataset synthesized by the FedPRS framework are closely aligned with those obtained using a real, private validation dataset.

As a new paradigm that integrates clustering with federated learning, federated clustering (FC) has recently attracted increasing attention, as it addresses the practical issue of privacy protection in distributed data. In this paper, we provide a comprehensive survey of recent advances in FC. This survey is organized into four parts. First, since FC is often developed by extending existing clustering methods, we review several classical clustering paradigms. Meanwhile, the inherent challenges of FC are summarized, and common improvement strategies are categorized. Second, we summarize experimental setups and evaluation protocols used in FC studies. Third, from the perspectives of data partitioning schemes and whether deep representation learning is incorporated, FC methods are divided into four categories, and representative algorithms in each category are reviewed. Finally, we discuss the limitations of current FC approaches and highlight potential directions for future research.

Recommender systems is set up to address the issue of information overload in traditional information retrieval systems, which is focused on recommending information that is of most interest to users from massive information. Generally, there is a sequential nature and heterogeneity to the behavior of a person interacting with a system, leading to the proposal of multi-behavior sequential recommendation (MBSR). MBSR is a relatively new and worthy direction for in-depth research, which can achieve state-of-the-art recommendation through suitable modeling, and some related works have been proposed. This survey aims to shed light on the MBSR problem. Firstly, we introduce MBSR in detail, including its problem definition, application scenarios and challenges faced. Secondly, we detail the classification of MBSR, including neighborhood-based methods, matrix factorization-based methods and deep learning-based methods, where we further classify the deep learning-based methods into different learning architectures based on RNN, GNN, Transformer, and generic architectures as well as architectures that integrate hybrid techniques. In each method, we present related works based on the data perspective and the modeling perspective, as well as analyze the strengths, weaknesses and features of these works. Finally, we discuss some promising future research directions to address the challenges and improve the current status of MBSR.