The removal of learned data from Machine Learning models through Machine Unlearning (MU) has been widely studied; however, there has yet to be an agreed-upon scheme for auditing MU. Existing work has shown that a dishonest model owner can falsify evidence to avoid executing MU, while curious auditors (and adversaries) can infer the privacy-sensitive properties of the model and its training data even with limited access. Yet auditing of MU under mutual distrust between the model owner and the auditor remains unexplored. We provide an information-theoretic proof for this scenario: for convex ML models, a generic audit scheme that relies solely on querying the model for behavioral signals cannot identify insufficiently unlearned models without revealing membership information of the retained set. Therefore, auditing MU under the assumption of a dishonest model owner and an honest-but-curious auditor faces an inherent privacy-audit tradeoff. Our empirical results on convex models strongly supports this result, while further experiments demonstrate that this privacy-audit tension persists in non-convex models. Our results call for a more careful consideration of the privacy-audit tension under a realistic auditor threat model, and serve as a foundation for more scrutiny of designs of privacy-preserving audit schemes for the MU pipeline. We also release our code implementation at https://github.com/LiouTang/Behavioral-Unlearn-Audit.

Machine Unlearning (MU) aims to update Machine Learning (ML) models following requests to remove training samples and their influences on a trained model efficiently without retraining the original ML model from scratch. While MU itself has been employed to provide privacy protection and regulatory compliance, it can also increase the attack surface of the model. Existing privacy inference attacks towards MU that aim to infer properties of the unlearned set rely on the weaker threat model that assumes the attacker has access to both the unlearned model and the original model, limiting their feasibility toward real-life scenarios. We propose a novel privacy attack, A Posteriori Label-Only Membership Inference Attack towards MU, Apollo, that infers whether a data sample has been unlearned, following a strict threat model where an adversary has access to the label-output of the unlearned model only. We demonstrate that our proposed attack, while requiring less access to the target model compared to previous attacks, can achieve relatively high precision on the membership status of the unlearned samples.

Federated learning (FL) is inherently susceptible to privacy breaches and poisoning attacks. To tackle these challenges, researchers have separately devised secure aggregation mechanisms to protect data privacy and robust aggregation methods that withstand poisoning attacks. However, simultaneously addressing both concerns is challenging; secure aggregation facilitates poisoning attacks as most anomaly detection techniques require access to unencrypted local model updates, which are obscured by secure aggregation. Few recent efforts to simultaneously tackle both challenges offen depend on impractical assumption of non-colluding two-server setups that disrupt FL's topology, or three-party computation which introduces scalability issues, complicating deployment and application. To overcome this dilemma, this paper introduce a Dual Defense Federated learning (DDFed) framework. DDFed simultaneously boosts privacy protection and mitigates poisoning attacks, without introducing new participant roles or disrupting the existing FL topology. DDFed initially leverages cutting-edge fully homomorphic encryption (FHE) to securely aggregate model updates, without the impractical requirement for non-colluding two-server setups and ensures strong privacy protection. Additionally, we proposes a unique two-phase anomaly detection mechanism for encrypted model updates, featuring secure similarity computation and feedback-driven collaborative selection, with additional measures to prevent potential privacy breaches from Byzantine clients incorporated into the detection process. We conducted extensive experiments on various model poisoning attacks and FL scenarios, including both cross-device and cross-silo FL. Experiments on publicly available datasets demonstrate that DDFed successfully protects model privacy and effectively defends against model poisoning threats.

Ongoing intense research in and rapid adoption of machine learning (ML) have raised the critical need to study both the privacy and fairness issues in ML models together because of the potential harm from their use. Research has shown that ML models can leak or allow inference of privacy-sensitive information. Similarly, research has also focused on fairness issues related to ML systems that are used for decision-making tasks that rely on sensitive attributes such as gender and race. Recent research includes several proposed approaches to mitigate privacy and fairness issues in ML models. In this paper, we explore the trade-offs between using differential privacy (DP) and fairness mitigation mechanisms in ML, particularly supervised learning. We examine the existing ML fairness mitigation mechanisms and their ability to mitigate bias in privacy-preserving ML approaches. DP and fairness mechanisms can be employed in different stages within the ML process: pre-, in-/during, and post-processing. We systematically compare the performance and fairness scores for fairness mechanisms in supervised ML models and those scores in their privacy-preserving ML versions. Our findings show the best approaches to achieving a balance between fairness and privacy. We show that it is possible to maintain good accuracy using DP and in-processing fairness mechanisms. We discuss the best ways to avoid the impact of privacy loss budget, epsilon, when building a privacy- and fairness-preserving model.

Federated learning is a computing paradigm that enhances privacy by enabling multiple parties to collaboratively train a machine learning model without revealing personal data. However, current research indicates that traditional federated learning platforms are unable to ensure privacy due to privacy leaks caused by the interchange of gradients. To achieve privacy-preserving federated learning, integrating secure aggregation mechanisms is essential. Unfortunately, existing solutions are vulnerable to recently demonstrated inference attacks such as the disaggregation attack. This paper proposes TAPFed , an approach for achieving privacy-preserving federated learning in the context of multiple decentralized aggregators with malicious actors. TAPFed uses a proposed threshold functional encryption scheme and allows for a certain number of malicious aggregators while maintaining security and privacy. We provide formal security and privacy analyses of TAPFed and compare it to various baselines through experimental evaluation. Our results show that TAPFed offers equivalent performance in terms of model quality compared to state-of-the-art approaches while reducing transmission overhead by 29%-45% across different model training scenarios. Most importantly, TAPFed can defend against recently demonstrated inference attacks caused by curious aggregators, which the majority of existing approaches are susceptible to.

Increasingly, information systems rely on computational, storage, and network resources deployed in third-party facilities such as cloud centers and edge nodes. Such an approach further exacerbates cybersecurity concerns constantly raised by numerous incidents of security and privacy attacks resulting in data leakage and identity theft, among others. These have, in turn, forced the creation of stricter security and privacy-related regulations and have eroded the trust in cyberspace. In particular, security-related services and infrastructures, such as Certificate Authorities (CAs) that provide digital certificate services and Third-Party Authorities (TPAs) that provide cryptographic key services, are critical components for establishing trust in crypto-based privacy-preserving applications and services. To address such trust issues, various transparency frameworks and approaches have been recently proposed in the literature. This article proposes TAB framework that provides transparency and trustworthiness of third-party authority and third-party facilities using blockchain techniques for emerging crypto-based privacy-preserving applications. TAB employs the Ethereum blockchain as the underlying public ledger and also includes a novel smart contract to automate accountability with an incentive mechanism that motivates users to participate in auditing, and punishes unintentional or malicious behaviors. We implement TAB and show through experimental evaluation in the Ethereum official test network, Rinkeby, that the framework is efficient. We also formally show the security guarantee provided by TAB, and analyze the privacy guarantee and trustworthiness it provides.

With the rapid advances in computing and information technologies, traditional access control models have become inadequate in terms of capturing fine-grained, and expressive security requirements of newly emerging applications. An attribute-based access control (ABAC) model provides a more flexible approach to addressing the authorization needs of complex and dynamic systems. While organizations are interested in employing newer authorization models, migrating to such models pose as a significant challenge. Many large-scale businesses need to grant authorizations to their user populations that are potentially distributed across disparate and heterogeneous computing environments. Each of these computing environments may have its own access control model. The manual development of a single policy framework for an entire organization is tedious, costly, and error-prone. In this article, we present a methodology for automatically learning ABAC policy rules from access logs of a system to simplify the policy development process. The proposed approach employs an unsupervised learning-based algorithm for detecting patterns in access logs and extracting ABAC authorization rules from these patterns. In addition, we present two policy improvement algorithms, including rule pruning and policy refinement algorithms to generate a higher quality mined policy. Finally, we implement a prototype of the proposed approach to demonstrate its feasibility.

Training complex neural network models using third-party cloud-based infrastructure among multiple data sources is a promising approach among existing machine learning solutions. However, privacy concerns of large-scale data collections and recent regulations have restricted the availability and use of privacy sensitive data in the third-party infrastructure. To address such privacy issues, a promising emerging approach is to train a neural network model over an encrypted dataset. Specifically, the model training process can be outsourced to a third party such as a cloud service that is backed by significant computing power, while the encrypted training data keeps the data confidential from the third party. Compared to training a traditional machine learning model over encrypted data, however, it is extremely challenging to train a deep neural network (DNN) model over encrypted data for two reasons: first, it requires large-scale computation over huge datasets; second, the existing solutions for computation over encrypted data, such as using homomorphic encryption, is inefficient. Further, for enhanced performance of a DNN model, we also need to use huge training datasets composed of data from multiple data sources that may not have pre-established trust relationships among each other. We propose a novel framework, NN-EMD , to train DNN over encrypted multiple datasets collected from multiple sources. Toward this, we propose a set of secure computation protocols using hybrid functional encryption schemes. We evaluate our framework for performance with regards to the training time and model accuracy on the MNIST datasets. We show that compared to other existing frameworks, our proposed NN-EMD framework can significantly reduce the training time, while providing comparable model accuracy and privacy guarantees as well as supporting multiple data sources. Furthermore, the depth and complexity of neural networks do not affect the training time despite introducing a privacy-preserving NN-EMD setting.

Federated learning (FL) has been proposed to allow collaborative training of machine learning (ML) models among multiple parties to keep their data private and only model updates are shared. Most existing approaches have focused on horizontal FL, while many real scenarios follow a vertically-partitioned FL setup, where a complete feature set is formed only when all the datasets from the parties are combined, and the labels are only available to a single party. Privacy-preserving vertical FL is challenging because complete sets of labels and features are not owned by one entity. Existing approaches for vertical FL require multiple peer-to-peer communications among parties, leading to lengthy training times, and are restricted to (approximated) linear models and just two parties. To close this gap, we propose FedV, a framework for secure gradient computation in vertical settings for several widely used ML models such as linear models, logistic regression, and support vector machines. FedV removes the need for peer-to-peer communication among parties by using functional encryption schemes and works for larger and changing sets of parties. We empirically demonstrate the applicability for multiple ML models and show a reduction of 10%-70% of training time and 80% to 90% in data transfer to the comparable state-of-the-art approaches.

The 2019 Federal Cybersecurity Research and Development Strategic Plan highlighted the mutual needs and benefits of artificial intelligence (AI) and cybersecurity. AI techniques are expected to enhance cybersecurity by assisting human system managers with automated monitoring, analysis, and responses to cybersecurity attacks. Conversely, it is essential to guard AI technologies from unintended uses and hostile exploitation by leveraging cybersecurity practices. Research results at the intersection of AI and cybersecurity can help us to be better equipped with tools and techniques to tackle the growing cybersecurity challenges, while also presenting an opportunity to devise fundamentally new ways to motivate and educate students about cybersecurity in the age of AI. Likewise, a June 2019 technical workshop on 'Artificial Intelligence and Cybersecurity: Opportunities and Challenges' noted how the interplay between AI, machine learning, and cybersecurity will continue to introduce new opportunities and challenges in the security of AI as well as AI for cybersecurity. Basic research at the intersection of AI, cybersecurity, and education has the potential to expand existing AI opportunities and resources in cybersecurity education and workforce development. Education efforts are needed to foster workforce knowledge and skills about applying AI expertise to cybersecurity as well as building robust and trustworthy AI. This BOF session will bring together researchers who are interested in these collaborative explorations.

Recent advances in information technologies have facilitated applications to generate, collect or process large amounts of sensitive personal data. Emerging cloud storage services provide a better paradigm to support the needs of such applications. Such cloud based solutions introduce additional security and privacy challenges when dealing with outsourced data including that of supporting fine-grained access control over such data stored in the cloud. In this paper, we propose an integrated, privacy-preserving user-centric attribute based access control framework to ensure the security and privacy of users’ data outsourced and stored by a cloud service provider (CSP). The core component of the proposed framework is a novel privacy-preserving, revocable ciphertext policy attribute-based encryption (PR-CP-ABE) scheme. To support advanced access control features like write access on encrypted data and privacy-preserving access policy updates, we propose extended Path-ORAM access protocol that can also prevent privacy disclosure of access patterns. We also propose an integrated secure deduplication approach to improve the storage efficiency of CSPs while protecting data privacy. Finally, we evaluate the proposed framework and compare it with other existing solutions with regards to the security and performance issues.

With rapid advances in computing systems, there is an increasing demand for more effective and efficient access control (AC) approaches. Recently, Attribute Based Access Control (ABAC) approaches have been shown to be promising in fulfilling the AC needs of such emerging complex computing environments. An ABAC model grants access to a requester based on attributes of entities in a system and an authorization policy; however, its generality and flexibility come with a higher cost. Further, increasing complexities of organizational systems and the need for federated accesses to their resources make the task of AC enforcement and management much more challenging. In this paper, we propose an adaptive ABAC policy learning approach to automate the authorization management task. We model ABAC policy learning as a reinforcement learning problem. In particular, we propose a contextual bandit system, in which an authorization engine adapts an ABAC model through a feedback control loop; it relies on interacting with users/administrators of the system to receive their feedback that assists the model in making authorization decisions. We propose four methods for initializing the learning model and a planning approach based on attribute value hierarchy to accelerate the learning process. We focus on developing an adaptive ABAC policy learning model for a home IoT environment as a running example. We evaluate our proposed approach over real and synthetic data. We consider both complete and sparse datasets in our evaluations. Our experimental results show that the proposed approach achieves performance that is comparable to ones based on supervised learning in many scenarios and even outperforms them in several situations.

Machine learning (ML) is increasingly being adopted in a wide variety of application domains. Usually, a well-performing ML model relies on a large volume of training data and high-powered computational resources. Such a need for and the use of huge volumes of data raise serious privacy concerns because of the potential risks of leakage of highly privacy-sensitive information; further, the evolving regulatory environments that increasingly restrict access to and use of privacy-sensitive data add significant challenges to fully benefiting from the power of ML for data-driven applications. A trained ML model may also be vulnerable to adversarial attacks such as membership, attribute, or property inference attacks and model inversion attacks. Hence, well-designed privacy-preserving ML (PPML) solutions are critically needed for many emerging applications. Increasingly, significant research efforts from both academia and industry can be seen in PPML areas that aim toward integrating privacy-preserving techniques into ML pipeline or specific algorithms, or designing various PPML architectures. In particular, existing PPML research cross-cut ML, systems and applications design, as well as security and privacy areas; hence, there is a critical need to understand state-of-the-art research, related challenges and a research roadmap for future research in PPML area. In this paper, we systematically review and summarize existing privacy-preserving approaches and propose a Phase, Guarantee, and Utility (PGU) triad based model to understand and guide the evaluation of various PPML solutions by decomposing their privacy-preserving functionalities. We discuss the unique characteristics and challenges of PPML and outline possible research directions that leverage as well as benefit multiple research communities such as ML, distributed systems, security and privacy.

Increasing incidents of security compromises and privacy leakage have raised serious privacy concerns related to cyberspace. Such privacy concerns have been instrumental in the creation of several regulations and acts to restrict the availability and use of privacy-sensitive data. The secure computation problem, initially and formally introduced as secure two-party computation by Andrew Yao in 1986, has been the focus of intense research in academia because of its fundamental role in building many of the existing privacy-preserving approaches. Most of the existing secure computation solutions rely on garbled-circuits and homomorphic encryption techniques to tackle secure computation issues, including efficiency and security guarantees. However, it is still challenging to adopt these secure computation approaches in emerging compute-intensive and data-intensive applications such as emerging machine learning solutions. Recently proposed functional encryption scheme has shown its promise as an underlying secure computation foundation in recent privacy-preserving machine learning approaches proposed. This paper revisits the secure computation problem using emerging and promising functional encryption techniques and presents a comprehensive study. We first briefly summarize existing conventional secure computation approaches built on garbled-circuits, oblivious transfer, and homomorphic encryption techniques. Then, we elaborate on the unique characteristics and challenges of emerging functional encryption based secure computation approaches and outline several research directions.

Recent advances in cryptographic approaches, such as Functional Encryption and Attribute-based Encryption and their variants, have shown significant promise for enabling public clouds to provide secure computation and storage services for users’ sensitive data. A crucial component of these approaches is a third-party authority (TPA) that must be trusted to set up public parameters, provide private key service, and so on. Components of deployed cryptographic mechanisms such as the certificate authorities (CAs) , which are the TPAs of the underlying PKI for the SSL/TLS protocol, have faced several types of attacks (e.g., stealthy targeted and censorship attacks), and certificate mis-issuance problems. Such practical challenges indicate that the successful deployment of newer emerging cryptographic schemes will also significantly depend on the trustworthiness of the TPAs. Furthermore, recently proposed decentralized TPA approaches that lower the threshold on the conditions required for an entity to become an authority can make the trust issue much worse. To address this issue, we propose an authority transparency framework to ensure the trustworthiness of TPAs of recent and emerging advanced cryptographic schemes. The framework includes a formal model and a secure logging -based approach to implement the framework. Further, to address the issues related to privacy, we also present a privacy-preserving authority transparency approach. We present security analysis and performance evaluation to show that authority transparency achieves the security and performance goals.

Cyber-physical systems (CPSs) integrate cyber components and physical processes. This integration enhances the capabilities of physical systems by incorporating intelligence into objects and services. On the other hand, the integration of cyber and physical components and the interaction between them introduce new security threats. Since CPSs are mostly safety-critical systems, data stored and communicated in them are highly critical. Hence, there is a crucial need for protecting the data and resources in CPSs against unauthorized accesses. In this paper, we propose an access control (AC) framework to address CPS related security issues. The proposed framework consists of two parts: a Cyber-Physical Access Control model (CPAC) and a Generalized Action Generation Model (GAGM). CPAC utilizes an attribute-based approach and extends it with cyber-physical components and cyber-physical interactions. In addition, we incorporate Separation of Duty (SoD) constraints into the CPAC model. GAGM is used to augment the enforcement of access policies. We present formal representations of CPAC and GAGM and demonstrate their use in a sample scenario for a medical CPS. We propose an algorithm for enforcing authorization policies. We implement the CPAC model and compare its performance against the core attribute-based access control model. We present an authorization enforcement approach and show through our experimental results its feasibility. (C) 2020 Elsevier Ltd. All rights reserved.