We present a simplified alternating direction method of multipliers with linear programming (ADMM-LP) decoder for LDPC codes by developing a method of approximate projection onto the parity polytope. The algorithm projects onto the affine hull of the $X$ closest local codewords for each check of degree d, where $X$ can be much smaller than d. We name this SAPA, the “sparse affine projection algorithm”. In contrast to exact projection, SAPA does not require a water-filling step and thus can be implemented with lower per-iteration complexity. We present numerical results which demonstrate not only that SAPA's performance, when used as part of an overall decoder, is close to that of exact projection, but also that the use of SAPA does not incur many additional iterations. This is in contrast to other approximate algorithms. Thus, SAPA is appropriate for use in limited-iteration decoders in high-throughput applications. Moreover, we analyze sparsity of polytope projections in the exact ADMM-LP decoding to explain the suitability of the proposed sparse approximation.
The Rényi cross-entropy measure between two distributions, a generalization of the Shannon cross-entropy, was recently used as a loss function for the improved design of deep learning generative adversarial networks. In this work, we examine the properties of this measure and derive closed-form expressions for it when one of the distributions is fixed and when both distributions belong to the exponential family. We also analytically determine a formula for the cross-entropy rate for stationary Gaussian processes and for finite-alphabet Markov sources.
We investigate the problem of distribution simu-lation under local differential privacy: Alice and Bob observe sequences $X^{n}$ and $Y^{n}$ respectively, where $Y^{n}$ is generated by a non-interactive $\varepsilon$ -Iocally differentially private (LDP) mechanism from $X^{n}$. The goal is for Alice and Bob to output $U$ and $V$ from a joint distribution that is close in total variation distance to a target distribution $P_{UV}$. As the main result, we show that such task is impossible if the hynercontractivity coefficient of $P_{UV}$ is strictly bigger than $\left(\frac{e^{\varepsilon}-1}{e^{\varepsilon}+1}\right)^{2}$ . The proof of this result also leads to a new operational interpretation of LDP mechanisms: if $Y$ is an output of an $\varepsilon$ -LDP mechanism with input $X$, then the probability of correctly guessing $f(X)$ given $Y$ is bigger than the probability of blind guessing only by $\frac{e^{\varepsilon}-1}{e^{\varepsilon}+1}$, for any deterministic finitely-supported function $f$ • If $f(X)$ is continuous, then a similar result holds for the minimum mean-squared error in estimating $f(X)$ given $Y$.
Separate source-channel (SSC) coding is known to be sub-optimal for communicating correlated Gaussian sources over fading Gaussian multiple-access channel (GMAC). Consid-ering the two-to-one GMAC with Rayleigh block fading and no transmitter-side channel-state information (CSI), a recent work demonstrated a simple fixed-rate joint source-channel (JSC) coding scheme referred to as source-channel trellis coded vector quantization (SC- TCVQ). This paper presents a new extension to SC- TCVQ which is shown to further improve performance with only a minor increase in encoder complexity. New results are presented which show that this finite block-length JSC coding scheme can beat the asymptotic performance upper bound of SSC coding.
Coding schemes for the lossy source coding problem are developed starting from point-to-point channel codes that are designed for symmetric channels. Bounds on the attained average distortion of the coding schemes are derived in terms of the parameters of the point-to-point channel codes. Assuming that the constituent channel codes satisfy some properties on the rate, the error probability and the input-output distribution of the decoding functions, the performance guarantees hold irrespective of other properties of the codes. Moreover, it is shown that the proposed coding schemes achieve the rate-distortion bound, provided that the constituent codes are rate-optimal. This would allow one to leverage commercial off-the-shelf codes for point-to-point symmetric channels in the practical implementation of lossy source codes.
