Cropout: A General Mechanism For Reducing Overfitting On Convolutional Neural Networks

Recently, a lot of Convolutional Neural Networks (CNNs) have been proposed for computer vision applications. However, how to improve the generalization ability of them remains challenging. In this paper, we propose a novel mechanism, namely, Cropout, to further reduce overfitting on Convolutional Neural Networks. The proposed Cropout is able to enlarge the diversity of the feature-map produced by convolutional layer, and further improve the generalization ability of deep CNNs. It mainly consists of three operations: grouping, cropping, and concatenating. Specifically, we first divide the feature-map produced by convolutional layer into different groups, and each group is considered as one transformation path. Next, each transformation path is assigned with a random crop transformation. Finally, all the transformation paths are concatenated into a new feature-map for further training. Extensive experiments on two benchmark datasets CIFAR-10/100 validate the effectiveness and generality of Cropout. Specially, the ResNeXt-29, 8x64d (with 34M parameters) with our proposed Cropout achieve the error rate of 3.38% and 16.89% on CIFAR-10/100 dataset, and surpass the standard ResNeXt-29, 16x64d (with 68M parameters) which is twice larger in model size. In addition, the proposed Cropout is able to be applied to different modern deep networks (e.g., ResNeXt, ResNet and DenseNet) to further boost the performance on image classification tasks.