Statistical Distance Latent Regulation Loss for Latent Code Recovery

Finding a latent code that can generate specific data by inverting a generative model is called latent code recovery (or latent vector recovery). When performing gradient descent based latent recovery, the probability that the recovered latent code was sampled from a latent random variable can be very low. To prevent this, latent regulation losses or element resampling methods have been used in some papers. In this paper, when the latent random variable is an IID (Independent and Identically Distributed) random variable and performing gradient descent-based latent code recovery, we propose statistical distance latent regulation loss to maximize the probability that the latent code was sampled from the latent random variable. The statistical distance latent regulation loss is the distance between the discrete uniform distribution, assuming each latent code element has the same probability and one-dimensional distribution that each element of the latent random variable follows in common. Since the statistical distance latent regulation loss considers all elements simultaneously, it maximizes the probability that the latent code was sampled from a latent random variable. Also, we propose the latent distribution goodness of fit test, an additional test that verifies whether the latent code is sampled from the latent random variable. This additional test verifies whether all recovered latent codes’ elements’ distribution follows one-dimensional distribution that each element of the latent random variable follows in common when the latent random variable is an IID random variable. Passing the latent distribution goodness of fit test does not mean that the latent codes are recovered correctly, but when the latent codes are recovered correctly, the latent distribution goodness of fit test should be passed. Compared with other latent regulation losses or element resampling methods, only latent code recovery using the statistical distance latent regulation loss could recover the correct latent code with high performance in the gradient descent-based latent code recovery.