Entropy Under Additive Bernoulli And Spherical Noises

Let Z(n) be iid Bernoulli(delta) and U-n be uniform on the set of all binary vectors of weight delta n (Hamming sphere). As is well known, the entropies of Z(n) and U-n are within O(log n). However, if X-n is another binary random variable independent of Z(n) and U-n, we show that H (X-n + U-n) and H (X-n + Z(n)) are within O (root n) and this estimate is tight. The bound is shown via coupling method. Tightness follows from the observation that the channels x(n) -> x(n) + U-n and x(n) -> x(n) + Z(n) have similar capacities, but the former has zero dispersion. Finally, we show that despite the root n slack in general, the Mrs. Gerber Lemma for H (X-n + U-n) holds with only an O(log n) correction compared to its brethren for H (X-n + Z(n)).