Federated graph learning (FGL) enables multiple participants with distributed but connected graph data to collaboratively train a model in a privacy-preserving way. However, the high communication cost hinders the adoption of FGL in many resource-limited or delay-sensitive applications. In this work, we focus on reducing the communication cost incurred by the transmission of neighborhood information in FGL. We propose to search for local proxies that can play a substitute role as the external neighbors and develop a novel federated graph learning framework named Gproxy. Gproxy utilizes representation similarity and class correlation to select local proxies for external neighbors. Additionally, we propose to dynamically adjust the proxy strategy according to the changing representation of nodes during the iterative training process. We also design a proxy cache to accelerate the search process by reusing proxy search outcomes for similar external neighbors. Furthermore, we provide a theoretical analysis and show that using a proxy node has a similar influence on training when it is sufficiently similar to the external one. Extensive evaluations show that Gproxy significantly reduces communication cost while maintaining model performance compared to strong baselines.

Private set intersection (PSI) has emerged as a key cryptographic protocol, enabling secure data sharing and facilitating collaborative computing among distributed data providers in recent years. However, it remains challenging to achieve efficient multiparty PSI (MPSI) for large-scale data and numerous participants in an open environment. To this end, we propose EL-MPSI, an efficient and lightweight MPSI scheme based on vector oblivious linear evaluation (VOLE) and oblivious key-value store (OKVS), which enables secure data sharing in settings with millions of datasets and dozens of participants. By simplifying the interaction process among multiple participants, the proposed scheme achieves constant-level round complexity and provides resistance against malicious adversaries, as well as collusion attack. Through theoretical analysis and experiments, we demonstrate that the security, efficiency, and scalability of our scheme perform better than existing state-of-the-art (SOTA) works. For millions of datasets and dozens of participants, EL-MPSI achieves second-level latency while keeping client communication overhead to approximately 10 MB. Moreover, in scenarios of malicious adversary setting, the extra execution overhead is negligible, which effectively facilitates large-scale data sharing.

Private Set Union (PSU) protocol enables secure computation of set unions across multiple parties while preserving input privacy - a critical capability for large-scale data aggregation. To date, two-party PSU schemes predominantly adopt the "split-then-execute" paradigm, which as demonstrated by Jia et al. (2024), inherently suffers from during-execution leakage. While their proposed symmetric-key approach addresses this vulnerability, the dependency on private equality test (PET) protocol additionally introduces non-negligible computational and communication overheads. Meanwhile, research on circuit-PSU protocol capable of securely computing a function (merge, sort or federated learning) over the union set remains blank. In this paper, we focus on two main objectives: (1) We present an optimized two-party PSU protocol that eliminates during-execution leakage while achieving enhanced efficiency; (2) We propose the first circuit-PSU. Unlike conventional PSU where participant obtains plaintext result, circuit-PSU provides secret-shared outputs that inherently prevent both during-execution leakage and intersection cardinality exposure. Furthermore, circuit-PSU removes the requirement for predefined result recipient and thus can offer superior flexibility. Meanwhile, our technical innovations include an optimized Share Translation (ShareTrans) protocol and a novel Oblivious Filter (OF) primitive that synergistically enhance both PSU and circuit-PSU efficiency. The experimental results indicate that under constrained computational resources (8-core CPU, 16GB RAM), the proposed circuit-PSU achieves a comparatively better improvement in communication complexity compared to existing state-of-the-art schemes, and our two-party PSU also exhibits comparable performance, for example, achieving approximately 4x efficiency improvement over Jia et al. (2024).

As a key technology to protect data privacy, vertical federated learning has recently been developed fast in both academia and industry. Notable works such as secureboost and secureboost+, which are all additive ensemble models that use a gradient boosting tree have been widely used in many applications. This paper proposes VFBoost-MO, a novel vertical federated learning (VFL) framework for multi-class classification using gradient boosting decision trees (GBDTs). Existing VFL methods for GBDTs are primarily limited to binary classification, while multi-class methods suffer from high computational and communication complexity. VFBoost-MO addresses these challenges by introducing a class-pair selection algorithm that constructs a single vector tree per iteration, significantly reducing complexity. Additionally, the framework employs local differential privacy (LDP) to protect data privacy without relying on computationally expensive encryption techniques. We present a theoretical analysis and experimental evaluation on eight public datasets demonstrating that VFBoost-MO achieves comparable accuracy to state-of-the-art methods while offering 2-3x performance improvement in training speed and convergence rate.

The integration of privacy-preserving computation and blockchain have significantly advanced cross-institutional data sharing in financial regulation. However, challenges of transaction chain reconstruction and private information preservation remain unresolved, which causes illegal activities such as fraudulent trade due to the lack of inter-bank information interoperability. To address these issues, this paper proposes a collaborative supervision that combines Private Information Retrieval (PIR) and blockchain technology. Leveraging the PIR protocol, supervision authority can retrieve transactions associated with suspicious identity from various banks, without disclosing the retrieval target. Furthermore, blockchain is used to record and verify transactions, which helps to automatically correlate multi-account transaction chains and enables verifiable tracing of fund flows. Experimental results demonstrate that the transaction tracing scheme can achieve a single retrieval and batch retrieval with high execution efficiency. The running time only rises to 10.917 s (at 1 × 10^7 data size) for batch retrieval of 1000 data entries, which is within an acceptable range for financial regulatory scenarios

Private data alignment, as the prerequisite for multiparty collaborative computation, attracts more attention in recent years, and some existing researches achieve the intersection sharing through two-party private set intersection (PSI) protocol based on various cryptographic techniques. However, they focus on the correctness and confidentiality of the protocol in the two-party scenario, while ignoring the efficiency and scalability in multiparty scenario. Additionally, the multiparty PSI protocol is difficult to be compatible with two parties simultaneously. To this end, we propose an oblivious pseudorandom function-based PSI scheme to achieve the data alignment, which is suitable for two parties and multiple parties. Specifically, to avoid frequent interactions among multiple parties, an efficient filtering algorithm is designed with the assistance of a server. The security proof for semi-honest and corrupted parties is provided, meanwhile, the computation and communication overhead analysis is given in detail. To evaluate the performance, we deploy the proposed scheme in two-party and multiparty scenario, and compare it with the existing protocols to discuss the execution complexity and overhead, which shows the efficiency and scalability of the proposed scheme in the multiparty scenario.

Fully homomorphic encryption (FHE) has emerged as a prominent area of cryptographic research in recent years, offering the capability to perform computations on ciphertext without compromising data privacy. Among various FHE schemes, the Cheon–Kim–Kim–Song (CKKS) algorithm for approximate homomorphic encryption has gained prominence due to its efficient handling of floating-point operations. Bootstrapping, a critical technique that enables unlimited homomorphic operations by refreshing noisy ciphertexts, represents both the most complex and essential component of practical FHE implementations. This survey provides a comprehensive analysis of bootstrapping techniques in CKKS, examining their evolution from the original proposal to current state-of-the-art methods. Recent literature has witnessed a proliferation of novel bootstrapping schemes for CKKS, these diverse approaches often emphasize different performance aspects, leading to a lack of a unified quantitative framework for comparative analysis. To address this gap, we systematically categorize existing approaches into three main directions: optimization of homomorphic modular reduction, optimization of encoding/decoding operations, and development of alternative constructions using blind rotation techniques. Through detailed comparative analysis, we identify that current schemes can achieve either high throughput (processing over 1000 ciphertexts per second) or high precision (up to 400 bits), but exhibit limitations in concurrent optimization of both parameters. Furthermore, potential directions for future optimizations are explored and discussed, contributing to the ongoing development of efficient and practical FHE systems.

Background Decentralized federated learning (DFL) may serve as a useful framework for machine learning (ML) tasks in multicentered studies, maximizing the use of clinical data without data sharing. We aim to propose the first workflow of DFL for ML tasks in multicentered studies, which can be as powerful as those using centralized data. Methods A DFL workflow was developed with 4 steps: registration, local computation, model update, and inspection. A total of 598 participants with acromegaly from Peking Union Medical College Hospital, and 120 participants from Xuanwu Hospital were enrolled. The cohort from Peking Union Medical College Hospital was further split into 5 centers. Nine clinical features were incorporated into ML-based models trained based on 4 algorithms: logistic regression (LR), gradient boosted decision tree, support vector machine (SVM), and deep neural network (DNN). The area under the curve of receiver operating characteristic curves was used to evaluate the performance of the models. Results Models trained based on DFL workflow performed better than most models in LR (P < 0.05), all models in DNN, SVM, and gradient boosted decision tree (P < 0.05). Models trained on DFL workflow performed as powerful as models trained on centralized data in LR, DNN, and SVM (P > 0.05). Conclusions We demonstrate that the DFL workflow without data sharing should be a more appropriate method in ML tasks in multicentered studies. And the DFL workflow should be further exploited in clinical researches in other departments and it can encourage and facilitate multicentered studies.

Federated Learning (FL) has attracted increasing attention from both academia and industry due to its merit of securely constructing AI models across multiple entities while preserving the privacy of local training data. However, recent research shows two persisting problems in FL that have yet to be solved: (1) limited practical adaptation of federated learning because of time-consuming conventional privacy-preserving methods, and (2) the absence of quantum-computing resistance in these methods. To address these problems, we propose a novel vertical federated learning strategy, HQsFL, which relies on Fully Homomorphic Encryption (FHE) and Matrix Vector Product basing on Coefficient Encoding. The proposed method can be widely applied to FL algorithms such as logistic regression and XGBoost, etc. We fully implement our approach and evaluate its utility and efficiency through extensive experiments performed on four synthetic datasets. The experimental results demonstrate that our proposed methods for vertical LR and XGBoost achieve comparable levels of AUC to conventional methods, while significantly improving training efficiency and achieving security property of quantum-computing resistance.

A recommendation system is of vital importance in delivering personalization services, which often brings continuous dual improvement in user experience and organization revenue. However, the data of one single organization may not be enough to build an accurate recommendation model for inactive or new cold-start users. Moreover, due to the recent regulatory restrictions on user privacy and data security, as well as the commercial conflicts, the raw data in different organizations cannot be merged to alleviate the scarcity issue in training a model. In order to learn users' preferences from such cross-silo data of different organizations and then provide recommendations to the cold-start users, we propose a novel federated learning framework, i.e., federated cross-organization recommendation ecosystem (FedCORE). Specifically, we first focus on the ecosystem problem of cross-organization federated recommendation, including cooperation patterns and privacy protection. For the former, we propose a privacy-aware collaborative training and inference algorithm. For the latter, we define four levels of privacy leakage and propose some methods for protecting the privacy. We then conduct extensive experiments on three real-world datasets and two seminal recommendation models to study the impact of cooperation in our proposed ecosystem and the effectiveness of privacy protection.

Vertical federated learning (VFL) is a promising category of federated learning for the scenario where data is vertically partitioned and distributed among parties. VFL enriches the description of samples using features from different parties to improve model capacity. Compared with horizontal federated learning, in most cases, VFL is applied in the commercial cooperation scenario of companies. Therefore, VFL contains tremendous business values. In the past few years, VFL has attracted more and more attention in both academia and industry. In this paper, we systematically investigate the current work of VFL from a layered perspective. From the hardware layer to the vertical federated system layer, researchers contribute to various aspects of VFL. Moreover, the application of VFL has covered a wide range of areas, e.g., finance, healthcare, etc. At each layer, we categorize the existing work and explore the challenges for the convenience of further research and development of VFL. Especially, we design a novel MOSP tree taxonomy to analyze the core component of VFL, i.e., secure vertical federated machine learning algorithm. Our taxonomy considers four dimensions, i.e., machine learning model (M), protection object (O), security model (S), and privacy-preserving protocol (P), and provides a comprehensive investigation.

Artificial intelligence (AI) is a promising technology to transform the healthcare industry. Due to the highly sensitive nature of patient data, federated learning (FL) is often leveraged to build models for smart healthcare applications. Existing deployed FL frameworks cannot address the key issues of varying data quality and heterogeneous data distributions across multiple institutions in this sector. In this paper, we report our experience developing and deploying the Contribution-Aware Federated Learning (CAFL) framework for smart healthcare. It provides an efficient and accurate approach to fairly evaluate FL participants' contribution to model performance without exposing their private data, and improves the FL model training protocol to allow the best performing intermediate models to be distributed to participants for FL training. Since its deployment in Yidu Cloud Technology Inc. in March 2021, CAFL has served 8 well-established medical institutions in China to build healthcare decision support models. It can perform contribution evaluations 2.84 times faster than the best existing approach, and has improved the average accuracy of the resulting models by 2.62% compared to the previous system (which is significant in industrial settings). To our knowledge, it is the first contribution-aware federated learning successfully deployed in the healthcare industry.

Sentiment analysis is a complex process that involves multiple modalities, which can provide more accurate and informative results than using a single modality. Although existing multimodal approaches have shown to be superior to mono-modal sentiment classification, they are not always practical in real-world scenarios where only mono-modal input is available, or where multimodal data is limited due to data scarcity or privacy concerns. To address this issue, we propose a novel approach that enhances mono-modal sentiment classification through federated transfer learning. Specifically, we focus on a practical industrial problem where text and speech data are owned by different affiliations, and we aim to bridge these modalities by sharing a cross-modal feature generator and phone classifier. Our proposed framework also incorporates differential privacy techniques to ensure privacy-preserving cross-modal transfer. Our experimental results on real-world spoken language sentiment classification corpora demonstrate the effectiveness of our proposed framework. We show that our approach can significantly improve the accuracy of mono-modal sentiment classification, even when only a limited amount of data is available.

Traditional federated learning (FL) algorithms, such as FedAvg, fail to handle non-i.i.d data because they learn a global model by simply averaging biased local models that are trained on non-i.i.d local data, therefore failing to model the global data distribution. In this paper, we present a novel Bayesian FL algorithm that successfully handles such a non-i.i.d FL setting by enhancing the local training task with an auxiliary task that explicitly estimates the global data distribution. One key challenge in estimating the global data distribution is that the data are partitioned in FL, and therefore the ground-truth global data distribution is inaccessible. To address this challenge, we propose an expectation-propagation-inspired probabilistic neural network, dubbed federated neural propagation (FedNP), which efficiently estimates the global data distribution given non-i.i.d data partitions. Our algorithm is sampling-free and end-to-end differentiable, can be applied with any conventional FL frameworks and learns richer global data representation. Experiments on both image classification tasks with synthetic non-i.i.d image data partitions and real-world non-i.i.d speech recognition tasks demonstrate that our framework effectively alleviates the performance deterioration caused by non-i.i.d data.

Recently, the artificial intelligence of things (AIoT) has been gaining increasing attention, with an intriguing vision of providing highly intelligent services through the network connection of things, leading to an advanced AI-driven ecology. However, recent regulatory restrictions on data privacy preclude uploading sensitive local data to data centers and utilizing them in a centralized approach. Directly applying federated learning algorithms in this scenario could hardly meet the industrial requirements of both efficiency and accuracy. Therefore, we propose an efficient industrial federated learning framework for AIoT in terms of a face recognition application. Specifically, we propose to utilize the concept of transfer learning to speed up federated training on devices and further present a novel design of a private projector that helps protect shared gradients without incurring additional memory consumption or computational cost. Empirical studies on a private Asian face dataset show that our approach can achieve high recognition accuracy in only 20 communication rounds, demonstrating its effectiveness in prediction and its efficiency in training.

The issue of data privacy is receiving increasing attention nowadays. Federated learning, which is a distributed machine learning setting where many clients (e.g. mobile devices or organizations) train a model collaboratively while keeping the training data local, was born as a result. And it has been applied to an increasing number of business scenarios. Like traditional machine learning, federated learning methods are also very sensitive to hyperparameters. However, data privacy will be protected using cryptographic techniques such as homomorphic encryption and secret sharing in federated learning, resulting in significant data amounts transferred among clients. Given this, we should find a more efficient method to optimize hyperparameters, which means utilizing fewer budgets (time, iterations, etc.). In this paper, we proposed a new efficient method FedTLBOHB, which combines both Hyperband and knowledge transfer for model-based optimization, to optimize hyperparameters of vertical federated learning. FedTLBOHB achieve high security level by avoiding a meta-feature based transfer learning paradigm, which will bring about security concerns in federated learning. Instead, a method similar to the model ensemble in Adaboost is designed for knowledge transfer. Meanwhile, according to our analysis, FedTLBOHB outperforms BOHB. The proposed method has been tested extensively on OpenML datasets and has been shown to be more effective than conventional methods at finding better hyperparameter configurations.

Generative Adversarial Networks (GANs) are overwhelming in unsupervised learning tasks due to their expressive power in modeling fine-grained data distributions. However, it is challenging for GANs to model distributions of separate non-i.i.d. data partitions as it usually adopts an over-general prior, limiting its capability in capturing the latent structure of multiple data partitions and thus leading to mode collapse. In this paper, we present a new Bayesian GAN, dubbed expectation propagation prior GAN (EP-GAN), which addresses the above challenge of modeling non-i.i.d. federated data through imposing a partition-invariant prior distribution on a Bayesian GAN. Furthermore, unlike most existing algorithms for deep-learning-based EP inference that require numerical quadrature, here we propose a closed-form solution for each update step of EP, leading to a more efficient solution for federated data modeling. Experiments on both synthetic extremely non-i.i.d. image data partitions and realistic non-i.i.d. speech recognition tasks demonstrate that our framework effectively alleviates the performance deterioration caused by non-i.i.d. data.

Since data regulations such as the European Union's General Data Protection Regulation (GDPR) have taken effect, the traditional two-step Automatic Speech Recognition (ASR) optimization strategy (i.e., training a one-size-fits-all model with vendor's centralized data and fine-tuning the model with clients' private data) has become infeasible. To meet these privacy requirements, TFE, a novel GDPR-compliant ASR ecosystem, has been proposed by us to incorporate transfer learning, federated learning, and evolutionary learning towards effective ASR model optimization. In this demonstration, we further design and implement a novel platform to promote the deployment and applicability of TFE. Our proposed platform allows enterprises to easily conduct the ASR optimization task using TFE across organizations.

Robustness is a long-standing challenge for automatic speech recognition (ASR) as the applied environment of any ASR system faces much noisier speech samples than clean training corpora. However, it is impractical to annotate every types of noisy environments. In this work, we propose a novel phonetic-semantic pre-training (PSP) framework that allows a model to effectively improve the performance of ASR against practical noisy environments via seamlessly integrating pre-training, self-supervised learning, and fine-tuning. In particular, there are three fundamental stages in PSP. First, pre-train the phone-to-word transducer (PWT) to map the generated phone sequence to the target text using only unpaired text data; second, continue training the PWT on more complex data generated from an empirical phone-perturbation heuristic, in additional to self-supervised signals by recovering the tainted phones; and third, fine-tune the resultant PWT with real world speech data. We perform experiments on two real-life datasets collected from industrial scenarios and synthetic noisy datasets, which show that the PSP effectively improves the traditional ASR pipeline with relative character error rate (CER) reductions of 28.63% and 26.38%, respectively, in two real-life datasets. It also demonstrates its robustness against synthetic highly noisy speech datasets.

EDITORIAL article Front. Genet., 25 April 2022Sec. Statistical Genetics and Methodology Volume 13 - 2022 | https://doi.org/10.3389/fgene.2022.897383