A New Method for Improving Prediction Performance in Neural Networks with Insufficient Data

This paper proposes Simultaneous Trainings of Identical Neural Networks (STNN) that aims to predict when sufficient data is not available for training neural networks (NN). While predictive applications of neural networks are growing, a common assumption in the NN algorithms is to have a training dataset that is large enough to sufficiently represent the population. However, in practice, this is difficult or expensive where the size of datasets is limited by the complexity and cost of large-scale experiments or data collections. Lacking sufficient data commits the NN training to two issues; namely parameter initialization and training sequence. STNN selects the outperforming NN out of several training episodes of the selected identical NN design by changing parameter initialization and training sequence. STNN has been evaluated by comparing with alternative methods in the literature. The results demonstrate improvement in prediction of STNN compared to other alternatives.