A New Activation Function Validated on Function Approximation Tasks

In this paper, a bounded, continuous, differentiable and non-monotonically increasing function is proposed (σ05). The efficiency and efficacy of using this function as an activation function for hidden layer nodes of FFANNs is demonstrated on five function approximation problems. The proposed function is compared against four bounded, continuous, differentiable, and non-monotonically increasing function (also called sigmoid functions). The results demonstrate that the usage of the proposed activation function as an activation function leads to creation of networks that generalize better than FFANNs using other (sigmoidal) activation function(s).