Comparing Dynamics: Deep Neural Networks versus Glassy Systems
international conference on machine learning, 2018.
We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems. The two main issues we address are the complexity of the loss-landscape and of the dynamics within it, and to what extent DNNs share similarities with glassy systems. Our findings, obtained for di...More
PPT (Upload PPT)