Urns And Entropies Revisited

An urn containing colored balls is sampled sequentially without replacement. New lower and upper bounds on the conditional and unconditional mutual information, and multi information are presented. They estimate dependence between drawings in terms of the colored ball configuration. Asymptotics are worked out when the number of balls increases and the proportion of the halls of each color stabilizes. Inequalities by Stam and by Diaconis and Freedman are compared and improved. Distances between the sampling with and without replacement, and between the multinomial and multivariate hypergeometric distributions are discussed.