Kullback-Leibler Divergence and the Central Limit Theorem

This paper investigates the asymptotics of Kullback-Leibler divergence between two probability distributions satisfying a Central Limit Theorem property. The basic problem is as follows. Let Xi, i ∈ N, be a sequence of independent random variables such that the sum Sn = ∑n i=1 Xi has the same expected value and satisfies the CLT under each probability distribution. Then what are the asymptotics of the KL divergence between the two distributions on Sn? Under regularity assumptions, we show that this KL divergence tends to a constant which is explicitly identified, along with the rate of convergence. This result holds for several variations of the basic problem.