A study on training fine-tuning of convolutional neural networks

As a subdiscipline of deep learning, Convolutional Neural Network (CNN) has been increasingly concerned about by people recently and widely applied in various fields related to computer vision. Although various algorithms that can improve CNN model have been proposed continuously, there is no consensus on utilizing which techniques to obtain better results for a specific task. This paper studies the fundamental deep CNN model's training fine-tuning, which can be classified as follows: network depth, network width, nonlinear activation function, pooling method, parameter initialization method, and learning rate strategy. According to each fine-tuning class's different schemes, the relevant comparison CNN model is designed by controlling variables. We trained and tested the models on the benchmark data set CIFAR-10. After comparing the advantages and disadvantages of the different fine-tuning schemes, we investigated their characteristics, explored the reasons, and discussed the general rules among them.