This paper introduces a paradigm shift in the way privacy is defined, driven by a novel interpretation of the fundamental result of Dwork and Naor about the impossibility of absolute disclosure prevention. We propose a general model of utility and privacy in which utility is achieved by disclosing the value of low-entropy features of a secret X, while privacy is maintained by hiding the value of high-entropy features of X. Adopting this model, we prove that contrary to popular opinion, it is possible to provide meaningful inferential privacy guarantees. These guarantees are given in terms of an operationally-meaningful information measure called pointwise maximal leakage (PML) and prevent privacy breaches against a large class of adversaries regardless of their prior beliefs about X. We show that PML-based privacy is compatible with and provides insights into existing notions such as differential privacy. We also argue that our new framework enables highly flexible mechanism designs, where the randomness of a mechanism can be adjusted to the entropy of the data, ultimately, leading to higher utility.

We investigate the possibility of guaranteeing inferential privacy for mechanisms that release useful information about some data containing sensitive information, denoted by $X$. We describe a general model of utility and privacy in which utility is achieved by disclosing the value of low-entropy features of $X$, while privacy is maintained by keeping high-entropy features of $X$ secret. Adopting this model, we prove that meaningful inferential privacy guarantees can be obtained, even though this is commonly considered to be impossible by the well-known result of Dwork and Naor. Then, we specifically discuss a privacy measure called pointwise maximal leakage (PML) whose guarantees are of the inferential type. We use PML to show that differential privacy admits an inferential formulation: it describes the information leaking about a single entry in a database assuming that every other entry is known, and considering the worst-case distribution on the data. Moreover, we define inferential instance privacy (IIP) as a bound on the (non-conditional) information leaking about a single entry in the database under the worst-case distribution, and show that it is equivalent to free-lunch privacy. Overall, our approach to privacy unifies, formalizes, and explains many existing ideas, e.g., why the informed adversary assumption may lead to underestimating the information leaking about each entry in the database. Furthermore, insights obtained from our results suggest general methods for improving privacy analyses; for example, we argue that smaller privacy parameters can be obtained by excluding low-entropy prior distributions from protection.

We introduce a privacy measure called pointwise maximal leakage, generalizing the pre-existing notion of maximal leakage, which quantifies the amount of information leaking about a secret $X$ by disclosing a single outcome of a (randomized) function calculated on $X$ . Pointwise maximal leakage is a robust and operationally meaningful privacy measure that captures the largest amount of information leaking about $X$ to adversaries seeking to guess arbitrary (possibly randomized) functions of $X$ , or equivalently, aiming to maximize arbitrary gain functions. We study several properties of pointwise maximal leakage, e.g., how it composes over multiple outcomes, how it is affected by pre- and post-processing, etc. Furthermore, we propose to view information leakage as a random variable which, in turn, allows us to regard privacy guarantees as requirements imposed on different statistical properties of the information leakage random variable. We define several privacy guarantees and study how they behave under pre-processing, post-processing and composition. Finally, we examine the relationship between pointwise maximal leakage and other privacy notions such as local differential privacy, local information privacy, $f$ -information, and so on. Overall, our paper constructs a robust and flexible framework for privacy risk assessment whose central notion has a strong operational meaning which can be adapted to a variety of applications and practical scenarios.

Pointwise maximal leakage (PML) is an operationally meaningful privacy measure that quantifies the amount of information leaking about a secret X to a single outcome of a related random variable Y. In this paper, we extend the notion of PML to random variables on arbitrary probability spaces. We develop two new definitions: First, we extend PML to countably infinite random variables by considering adversaries who aim to guess the value of discrete (finite or countably infinite) functions of X. Then, we consider adversaries who construct estimates of X that maximize the expected value of their corresponding gain functions. We use this latter setup to introduce a highly versatile form of PML that captures many scenarios of practical interest whose definition requires no assumptions about the underlying probability spaces.

This paper investigates the problem of synthesizing joint distributions in the finite-length regime. For a fixed blocklength $n$ and an upper bound on the distribution approximation $\epsilon$, we prove a capacity result for fixed-length strong coordination. It is shown analytically that the rate conditions for the fixed-length regime are lower-bounded by the mutual information that appears in the asymptotical condition plus $Q^{-1} \left(\epsilon \right) \sqrt{ V/n}$, where $V$ is the channel dispersion, and $Q^{-1}$ is the inverse of the Gaussian cumulative distribution function.

Most methods for publishing data with privacy guarantees introduce randomness into datasets which reduces the utility of the published data. In this paper, we study the privacy-utility tradeoff by taking maximal leakage as the privacy measure and the expected Hamming distortion as the utility measure. We study three different but related problems. First, we assume that the data-generating distribution (i.e., the prior) is known, and we find the optimal privacy mechanism that achieves the smallest distortion subject to a constraint on maximal leakage. Then, we assume that the prior belongs to some set of distributions, and we formulate a min-max problem for finding the smallest distortion achievable for the worst-case prior in the set, subject to a maximal leakage constraint. Lastly, we define a partial order on privacy mechanisms based on the largest distortion they generate. Our results show that when the prior distribution is known, the optimal privacy mechanism fully discloses symbols with the largest prior probabilities, and suppresses symbols with the smallest prior probabilities. Furthermore, we show that sets of priors that contain more uniform distributions lead to larger distortion, while privacy mechanisms that distribute the privacy budget more uniformly over the symbols create smaller worst-case distortion. A full version of this paper is accessible at: https://arxiv.org/pdf/2105.01033.pdf

Machine learning models are known to memorize the unique properties of individual data points in a training set. This memorization capability can be exploited by several types of attacks to infer information about the training data, most notably, membership inference attacks. In this paper, we propose an approach based on information leakage for guaranteeing membership privacy. Specifically, we propose to use a conditional form of the notion of maximal leakage to quantify the information leaking about individual data entries in a dataset, i.e., the entrywise information leakage. We apply our privacy analysis to the Private Aggregation of Teacher Ensembles (PATE) framework for privacy-preserving classification of sensitive data and prove that the entrywise information leakage of its aggregation mechanism is Schur-concave when the injected noise has a log-concave probability density. The Schur-concavity of this leakage implies that increased consensus among teachers in labeling a query reduces its associated privacy cost. Finally, we derive upper bounds on the entrywise information leakage when the aggregation mechanism uses Laplace distributed noise.

We consider a network of two nodes separated by a noisy channel with two-sided state information, in which the input and output signals have to be coordinated with the source and its reconstruction. In the case of non-causal encoding and decoding, we propose a joint source-channel coding scheme and we develop inner and outer bounds for the strong coordination region. While the inner and outer bounds do not match in general, we provide a complete characterization of the strong coordination region in three particular cases: i) when the channel is perfect; ii) when the decoder is lossless; and iii) when the random variables of the channel are independent from the random variables of the source. Through the study of these special cases, we prove that the separation principle does not hold for the joint source-channel strong coordination. Finally, in the absence of state information, we show that polar codes achieve a subset of the best known inner bound for the strong coordination region, therefore offering a constructive alternative to random binning and coding proofs.

We consider a three-node network, in which two agents wish to communicate over a noisy channel, while controlling the distribution observed by a third external agent. We use strong coordination to constrain the distribution, and we provide a complete characterization of the "remote strong coordination and reliable communication" region.

We consider a network of two nodes separated by a noisy channel, in which the source and its reconstruction have to be strongly coordinated, while simultaneously satisfying the strong secrecy condition with respect to an outside observer of the noisy channel. In the case of non-causal encoding and decoding, we propose a joint source-channel coding scheme for the secure strong coordination region. Furthermore, we provide a complete characterization of the secure strong coordination region when the decoder has to reliably reconstruct the source sequence and the legitimate channel is more capable than the channel of the eavesdropper.

We consider the problem of synthesizing joint distributions of signals and actions over noisy channels in the finite-length regime. For a fixed blocklength n and an upper bound on the distance ε, a coding scheme is proposed such that the induced joint distribution is ε-close in L^1 distance to a target i.i.d. distribution. The set of achievable target distributions and rate for asymptotic strong coordination can be recovered from the main result of this paper by having n that tends to infinity.

In this paper, we propose a coding scheme based on polar codes for empirical coordination of autonomous devices. We consider a two-node network with a noisy link in which the input and output signals have to be coordinated with the source and the reconstruction. In the case of strictly causal encoding, we show that polar codes achieve the empirical coordination region, provided that a vanishing rate of common randomness is available.

We consider a network of two nodes separated by a noisy channel, in which the input and output signals have to be coordinated with the source and its reconstruction. In the case of strictly causal encoding and non-causal decoding, we prove inner and outer bounds for the strong coordination region and show that the inner bound is achievable with polar codes.

We develop a random binning scheme for strong coordination in a network of two nodes separated by a noisy channel, in which the input and output signals have to be coordinated with the source and its reconstruction. In the case of non-causal encoding and decoding, we propose a joint source-channel coding scheme and develop inner and outer bounds for the strong coordination region. While the set of achievable target distributions is the same as for empirical coordination, we characterize the rate of common randomness required for strong coordination.

We develop a polar coding scheme for empirical coordination in a two-node network with a noisy link in which the input and output signals have to be coordinated with the source and the reconstruction. In the case of non-causal encoding and decoding, we show that polar codes achieve the best known inner bound for the empirical coordination region, provided that a vanishing rate of common randomness is available. This scheme provides a constructive alternative to random binning and coding proofs.