FFANN Weight Initialization: A New Method

In this paper a new weight initialization scheme is proposed. The proposed scheme, distributes the biases / thresholds at a equal intervals in an interval ( - λ, λ); while the weights are distributed uniformly in the same interval. The value of the λ is chosen such that the expected value of the net inputs is 0 while the variance is 1. On a set of 10 tasks (5 function approximation and 5 real life benchmark regression tasks), the proposed weight initialization routine was compared to three existing weight initialization routine. From the results obtained we may infer that the proposed weight initialization is almost always better than these three existing routines and is never worse on the basis of generalization performance (result over the test data set).