Weak approximation of transformed stochastic gradient MCMC
MACHINE LEARNING(2020)
摘要
Stochastic gradient Langevin dynamics (SGLD) is a computationally efficient sampler for Bayesian posterior inference given a large scale dataset and a complex model. Although SGLD is designed for unbounded random variables, practical models often incorporate variables within a bounded domain, such as non-negative or a finite interval. The use of variable transformation is a typical way to handle such a bounded variable. This paper reveals that several mapping approaches commonly used in the literature produce erroneous samples from theoretical and empirical perspectives. We show that the change of random variable in discretization using an invertible Lipschitz mapping function overcomes the pitfall as well as attains the weak convergence, while the other methods are numerically unstable or cannot be justified theoretically. Experiments demonstrate its efficacy for widely-used models with bounded latent variables, including Bayesian non-negative matrix factorization and binary neural networks.
更多查看译文
关键词
Stochastic gradient MCMC,Transform,Convergence analysis,Itô process
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络