Privacy models were introduced in privacy-preserving data publishing and statistical disclosure control with the promise to end the need for costly empirical assessment of disclosure risk. We examine how well this promise is kept by the main privacy models. We find they may fail to provide adequate protection guarantees because of problems in their definition or incur unacceptable trade-offs between privacy protection and utility preservation. Specifically, k-anonymity may not entirely exclude disclosure if enforced with deterministic mechanisms or without constraints on the confidential values. On the other hand, differential privacy (DP) incurs unacceptable utility loss for small budgets and its privacy guarantee becomes meaningless for large budgets. In the latter case, an ex post empirical assessment of disclosure risk becomes necessary, undermining the main appeal of privacy models. Whereas the utility preservation of DP can only be improved by relaxing its privacy guarantees, we argue that a semantic reformulation of k-anonymity can offer more robust privacy without losing utility with respect to traditional syntactic k-anonymity.

Large language models (LLMs) have recently revolutionized language processing tasks but have also brought ethical and legal issues. LLMs have a tendency to memorize potentially private or copyrighted information present in the training data, which might then be delivered to end users at inference time. When this happens, a naive solution is to retrain the model from scratch after excluding the undesired data. Although this guarantees that the target data have been forgotten, it is also prohibitively expensive for LLMs. Approximate unlearning offers a more efficient alternative, as it consists of ex post modifications of the trained model itself to prevent undesirable results, but it lacks forgetting guarantees because it relies solely on empirical evidence. In this work, we present DP2Unlearning, a novel LLM unlearning framework that offers formal forgetting guarantees at a significantly lower cost than retraining from scratch on the data to be retained. DP2Unlearning involves training LLMs on textual data protected using ϵ-differential privacy (DP), which later enables efficient unlearning with the guarantees against disclosure associated with the chosen ϵ. Our experiments demonstrate that DP2Unlearning achieves similar model performance post-unlearning, compared to an LLM retraining from scratch on retained data -the gold standard exact unlearning- but at approximately half the unlearning cost. In addition, with a reasonable computational cost, it outperforms approximate unlearning methods at both preserving the utility of the model post-unlearning and effectively forgetting the targeted information. The code of our experiments is available at https://github.com/tamimalmahmud/LLM-Unlearning/tree/main/DP2Unlearning.

The massive adoption of large language models has prompted concerns about how to align them with human ethics and the rule of law. Digital forgetting of undesirable knowledge via machine unlearning is a promising strategy we survey here.

Privacy protection laws, such as the GDPR, grant individuals the right to request the forgetting of their personal data not only from databases but also from machine learning (ML) models trained on them. Machine unlearning has emerged as a practical means to facilitate model forgetting of data instances seen during training. Although some existing machine unlearning methods guarantee exact forgetting, they are typically costly in computational terms. On the other hand, more affordable methods do not offer forgetting guarantees and are applicable only to specific ML models. In this paper, we present efficient unlearning with privacy guarantees (EUPG), a novel machine unlearning framework that offers formal privacy guarantees to individuals whose data are being unlearned. EUPG involves pre-training ML models on data protected using privacy models, and it enables efficient unlearning with the privacy guarantees offered by the privacy models in use. Through empirical evaluation on four heterogeneous data sets protected with k-anonymity and ε-differential privacy as privacy models, our approach demonstrates utility and forgetting effectiveness comparable to those of exact unlearning methods, while significantly reducing computational and storage costs. Our code is available at https://github.com/najeebjebreel/EUPG.

Text anonymization is a challenging task usually carried out by human annotators, thereby incurring significant economic and temporal costs. Even though automated approaches have been proposed to mitigate those costs, practical mechanisms for text anonymization mostly rely on named entity recognition (NER), which is acknowledged to offer insufficient privacy protection. To tackle this issue, we propose a methodology to enhance the privacy protection attained by any text anonymization mechanism –with a focus on NER-based ones–, while providing empirical guarantees against re-identification rooted on the k-anonymity privacy model. Our method relies on a neural language model trained on the aggregated background knowledge that can be leveraged to conduct re-identification attacks. Then, it employs explainability techniques to detect and iteratively mask the unprotected terms that caused the greatest re-identification risk until a user-defined k-anonymity level is reached. On the contrary to most existing methods in the text anonymization literature, our approach allows to intuitively configure the desired level of protection, and to tune the trade-off between privacy and data utility preservation. Experiments show that our method is able to significantly and consistently lower the re-identification risk of NER-based anonymizations, and to compete against more sophisticated state-of-the-art text anonymization methods while being free of their costs and external dependencies.

Machine unlearning is a discipline that seeks to make a machine learning model forget some of the information items it was trained on. This is a means to enforce the fundamental right-to-beforgotten (RTBF), among others. However, unlearning may paradoxically increase the vulnerability of models to membership inference attacks (MIAs) on precisely the unlearned data, by creating distinguishable patterns between forgotten and non-forgotten data. This is a manifestation of the "Streisand effect", where attempts to remove or forget information make it more noticeable. In this paper, we investigate this critical challenge and propose defense mechanisms designed to seamlessly integrate with existing machine unlearning methods. These defenses aim to obscure the distinguishability of forgotten data, thus mitigating MIA risks without compromising model performance or computational efficiency. In addition, we define rigorous evaluation criteria to assess the effectiveness of such defenses.

Longitudinal data consist of observations collected over time from a set of individuals. The accumulation of information on each individual over time makes longitudinal data particularly privacy-sensitive. However, existing anonymization methods are often inadequate for ensuring privacy-preserving publication of such data, as current privacy models assume unrealistic levels of attacker knowledge. To address this, we propose the (k, beta)L-privacy model, which assumes that an attacker's knowledge is limited to a subsequence of L quasi-identifiers. This provides a more realistic representation of the information an attacker might actually possess. Our model guarantees that every subsequence of L quasi-identifier values appears in either zero or at least k records within the longitudinal database. Additionally, it ensures that the confidence of any sensitive value within these k records is at most beta times higher than its confidence in the entire dataset. This not only strengthens privacy protection but also enhances data utility. Furthermore, we introduce FCLA, an anonymization algorithm designed to enforce our privacy model while prioritizing data utility. FCLA effectively mitigates identity and attribute disclosures, as well as skewness attacks in longitudinal data. It achieves this by partitioning sequences into groups and anonymizing them independently-a process that can be efficiently parallelized. Experimental results show that FCLA outperforms existing methods in preserving data utility while adhering to strict privacy constraints. Additionally, time complexity analysis and execution time measurements demonstrate that FCLA is more efficient and scalable than alternative approaches.

This paper offers a comprehensive analysis of the statistical disclosure limitation (SDL) methodologies employed by the U.S. Census Bureau on the 2010 and 2020 Decennial Census releases under the perspective of the disclosure risk of the most vulnerable respondents. We first review the SDL methodology used up to the Decennial Census 2010, which was based on targeted swapping. Second, we examine recently reported reconstruction and reidentification results on the Decennial Census 2010 outputs, which form the foundation for the U.S. Census Bureau’s decision to switch to a differentially private (DP) method for the 2020 release. Third, we examine the actual privacy and data accuracy achieved by the DP method and compare with the privacy and accuracy offered by the formerly employed swapping mechanism. We conclude that the DP method is not an adequate solution to protect the typically sparse tables present in the Decennial Censuses because it does not offer meaningful privacy guarantees in general, it poorly protects the privacy of the most vulnerable respondents in particular, and it significantly degrades the quality of the released data. We also argue that the claimed disclosure risks of previous Census releases were overstated because of a flawed reidentification procedure. Therefore, the U.S. Census Bureau’s decision to change the SDL methodology to a DP-based one for the 2020 release was not only unwarranted, but it also reduced privacy and data quality compared to former releases.

Human facial data hold tremendous potential to address a variety of classification problems, including face recognition, age estimation, gender identification, emotion analysis, and race classification. However, recent privacy regulations, such as the EU General Data Protection Regulation and others, have restricted the ways in which human images may be collected and used for research. As a result, several previously published data sets containing human faces have been removed from the internet due to inadequate data collection methods that failed to meet privacy regulations. Data sets consisting of synthetic data have been proposed as an alternative, but they fall short of accurately representing the real data distribution. On the other hand, most available data sets are labeled for just a single task, which limits their applicability. To address these issues, we present the Multi-Task Faces (MTF) image data set, a meticulously curated collection of face images designed for various classification tasks, including face recognition, as well as race, gender, and age classification. The MTF data set has been ethically gathered by leveraging publicly available images of celebrities and strictly adhering to copyright regulations. In this paper, we present this data set and provide detailed descriptions of the followed data collection and processing procedures. Furthermore, we evaluate the performance of five deep learning (DL) models on the MTF data set across the aforementioned classification tasks. Additionally, we compare the performance of DL models over the processed MTF data and over raw data crawled from the internet. The reported results constitute a baseline for further research employing these data. The MTF data set can be accessed through the following link (please cite the present paper if you use the data set): https://github.com/RamiHaf/MTF_data_set

Differential privacy (DP) is a widespread data protection mechanism. However, its application in real-world scenarios has been challenging. To shed some light on this, we offer a critical analysis of 21 DP deployments by top-tier companies and institutions over the past decade.

Text sanitization aims to rewrite parts of a document to prevent disclosure of personal information. The central challenge of text sanitization is to strike a balance between privacy protection (avoiding the leakage of personal information) and utility preservation (retaining as much as possible of the document's original content). To this end, we introduce a novel text sanitization method based on generalizations, that is, broader but still informative terms that subsume the semantic content of the original text spans. The approach relies on the use of instruction-tuned large language models (LLMs) and is divided into two stages. Given a document including text spans expressing personally identifiable information (PII), the LLM is first applied to obtain truth-preserving replacement candidates for each text span and rank them according to their abstraction level. Those candidates are then evaluated for their ability to protect privacy by conducting inference attacks with the LLM. Finally, the system selects the most informative replacement candidate shown to be resistant to those attacks. This two-stage process produces replacements that effectively balance privacy and utility. We also present novel metrics to evaluate these two aspects without needing to manually annotate documents. Results on the Text Anonymization Benchmark show that the proposed approach, implemented with Mistral 7B Instruct, leads to enhanced utility, with only a marginal ( < 1 p.p.) increase in re-identification risk compared to fully suppressing the original spans. Furthermore, our approach is shown to be more truth-preserving than existing methods such as Microsoft Presidio's synthetic replacements.

Accurate diagnosis of breast cancer in dense breasts requires expert radiologists to examine multiple ultrasound images per patient. This diagnosis procedure is tedious, time-consuming and prone to misdiagnosis due to human fatigue. AI -aided diagnosis systems can help alleviate this burden. However, vast amounts of data from multiple hospitals, diverse patient demographics, imaging scanners, and protocols are required to develop accurate, robust, and generalizable AI models. Obtaining such a mixture of data is quite challenging due to privacy concerns, data ownership issues, and regulatory constraints. To address this problem, we introduce a privacy-preserving and regulatory-compliant method for breast cancer diagnosis from ultrasound images using federated learning. This allows hospitals to keep their data on their premises while collaboratively training the AI model, only exchanging the model parameters trained on their private data. Experimental evaluation within a real federated data setting shows that our method achieved higher diagnostic accuracy than standardized centralized training while ensuring compliance with data privacy regulations.

Large language models (LLMs) have become the state of the art in natural language processing. The massive adoption of generative LLMs and the capabilities they have shown have prompted public concerns regarding their impact on the labor market, privacy, the use of copyrighted work, and how these models align with human ethics and the rule of law. As a response, new regulations are being pushed, which require developers and service providers to evaluate, monitor, and forestall or at least mitigate the risks posed by their models. One mitigation strategy is digital forgetting: given a model with undesirable knowledge or behavior, the goal is to obtain a new model where the detected issues are no longer present. Digital forgetting is usually enforced via machine unlearning techniques, which modify trained machine learning models for them to behave as models trained on a subset of the original training data. In this work, we describe the motivations and desirable properties of digital forgetting when applied to LLMs, and we survey recent works on machine unlearning. Specifically, we propose a taxonomy of unlearning methods based on the reach and depth of the modifications done on the models, we discuss and compare the effectiveness of machine unlearning methods for LLMs proposed so far, and we survey their evaluation. Finally, we describe open problems of machine unlearning applied to LLMs and we put forward recommendations for developers and practitioners.

$\epsilon$-Differential privacy (DP) is a well-known privacy model that offers strong privacy guarantees. However, when applied to data releases, DP significantly deteriorates the analytical utility of the protected outcomes. To keep data utility at reasonable levels, practical applications of DP to data releases have used weak privacy parameters (large $\epsilon$), which dilute the privacy guarantees of DP. In this work, we tackle this issue by using an alternative formulation of the DP privacy guarantees, named $\epsilon$-individual differential privacy (iDP), which causes less data distortion while providing the same protection as DP to subjects. We enforce iDP in data releases by relying on attribute masking plus a pre-processing step based on data microaggregation. The goal of this step is to reduce the sensitivity to record changes, which determines the amount of noise required to enforce iDP (and DP). Specifically, we propose data microaggregation strategies designed for iDP whose sensitivities are significantly lower than those used in DP. As a result, we obtain iDP-protected data with significantly better utility than with DP. We report on experiments that show how our approach can provide strong privacy (small $\epsilon$) while yielding protected data that do not significantly degrade the accuracy of secondary data analysis.

Several authors have claimed the “failure of anonymization,” despite over 50 years of research. We review privacy leaks reported over the past decades and conclude they were due to nonexistent or inadequate anonymization, rather than a lack of robust anonymization methods.

The availability of textual data depicting human-centered features and behaviors is crucial for many data mining and machine learning tasks. However, data containing personal information should be anonymized prior making them available for secondary use. A variety of text anonymization methods have been proposed in the last years, which are standardly evaluated by comparing their outputs with human-based anonymizations. The residual disclosure risk is estimated with the recall metric, which quantifies the proportion of manually annotated re-identifying terms successfully detected by the anonymization algorithm. Nevertheless, recall is not a risk metric, which leads to several drawbacks. First, it requires a unique ground truth, and this does not hold for text anonymization, where several masking choices could be equally valid to prevent re-identification. Second, it relies on human judgements, which are inherently subjective and prone to errors. Finally, the recall metric weights terms uniformly, thereby ignoring the fact that the influence on the disclosure risk of some missed terms may be much larger than of others. To overcome these drawbacks, in this paper we propose a novel method to evaluate the disclosure risk of anonymized texts by means of an automated re-identification attack. We formalize the attack as a multi-class classification task and leverage state-of-the-art neural language models to aggregate the data sources that attackers may use to build the classifier. We illustrate the effectiveness of our method by assessing the disclosure risk of several methods for text anonymization under different attack configurations. Empirical results show substantial privacy risks for most existing anonymization methods.

In federated learning (FL), a set of participants share updates computed on their local data with an aggregator server that combines updates into a global model. However, reconciling accuracy with privacy and security is a challenge to FL. On the one hand, good updates sent by honest participants may reveal their private local information, whereas poisoned updates sent by malicious participants may compromise the model's availability and/or integrity. On the other hand, enhancing privacy via update distortion damages accuracy, whereas doing so via update aggregation damages security because it does not allow the server to filter out individual poisoned updates. To tackle the accuracy-privacy-security conflict, we propose fragmented FL (FFL), in which participants randomly exchange and mix fragments of their updates before sending them to the server. To achieve privacy, we design a lightweight protocol that allows participants to privately exchange and mix encrypted fragments of their updates so that the server can neither obtain individual updates nor link them to their originators. To achieve security, we design a reputation-based defense tailored for FFL that builds trust in participants and their mixed updates based on the quality of the fragments they exchange and the mixed updates they send. Since the exchanged fragments' parameters keep their original coordinates and attackers can be neutralized, the server can correctly reconstruct a global model from the received mixed updates without accuracy loss. Experiments on four real data sets show that FFL can prevent semi-honest servers from mounting privacy attacks, can effectively counter-poisoning attacks, and can keep the accuracy of the global model.

Federated learning (FL) is a decentralized machine learning (ML) framework that allows models to be trained without sharing the participants’ local data. FL thus preserves privacy better than centralized machine learning. Since textual data (such as clinical records, posts in social networks, or search queries) often contain personal information, many natural language processing (NLP) tasks dealing with such data have shifted from the centralized to the FL setting. However, FL is not free from issues, including convergence and security vulnerabilities (due to unreliable or poisoned data introduced into the model), communication and computation bottlenecks, and even privacy attacks orchestrated by honest-but-curious servers. In this paper, we present a systematic literature review (SLR) of NLP applications in FL with a special focus on FL issues and the solutions proposed so far. Our review surveys 36 recent papers published in relevant venues, which are systematically analyzed and compared from multiple perspectives. As a result of the survey, we also identify the most outstanding challenges in the area.

Due to its small size and lifelong optical transparency, the fish Danionella cerebrum is an emerging model organism in biomedical research. How can this small vertebrate under 12 mm length produce sounds over 140 dB? We found that it possesses ...Motion is the basis of nearly all animal behavior. Evolution has led to some extraordinary specializations of propulsion mechanisms among invertebrates, including the mandibles of the dracula ant and the claw of the pistol shrimp. In contrast, vertebrate ...