Variational Inference with Orthogonal Normalizing Flows

Normalizing flows Variational inference relies on flexible approximate posterior distributions. In many settings very simple posteriors such as diagonal covariance Gaussians are used. Rezende and Mohamed [2015] propose a way to construct more flexible posteriors by transforming a simple base distribution with a series of invertible transformations with easily computable Jacobians. The resulting transformed density after one such transformation is given by: