Piecewise linear activations substantially shape the loss surfaces of neural networks

ICLR, 2020.

Cited by: 0|Views5
EI

Abstract:

Understanding the loss surfaces of neural networks is fundamentally important to understanding deep learning. This paper presents how the nonlinearities in activations substantially shape the loss surfaces of neural networks. We first prove that the loss surface of every neural network has infinite spurious local minima, which are defined...More

Code:

Data:

Your rating :
0

 

Tags
Comments