Geometry Aware Constrained Optimization Techniques for Deep Learning.
CVPR, pp.4460-4469, (2018)
In this paper, we generalize the Stochastic Gradient Descent (SGD) and RMSProp algorithms to the setting of Riemannian optimization. SGD is a popular method for large scale optimization. In particular, it is widely used to train the weights of Deep Neural Networks. However, gradients computed using standard SGD can have large variance, wh...More
PPT (Upload PPT)