Revise Saturated Activation Functions

arXiv: Learning, Volume abs/1602.05980, 2016.

Cited by: 50|Views60
EI

Abstract:

In this paper, we revise two commonly used saturated functions, the logistic sigmoid and the hyperbolic tangent (tanh). point out that, besides the well-known non-zero centered property, slope of the activation function near the origin is another possible reason making training deep networks with the logistic function difficult to train....More

Code:

Data:

Your rating :
0

 

Tags
Comments