Dynamic Network Pruning Based on Expectation Scaling Factor

Abstract Convolutional neural networks are limited by the large number of floating-point operations (FLOPs) and cannot be easily ported to edge devices. Popular pruning methods formulate a fixed pruning strategy with the assistance of the intrinsic attributes of the model, ignoring of the effect of the dataset on the pruning results. This study introduces the expectation scaling factor (δE) as the zero-order attribute of the channel, which reflects the expectation changing of the channel on the pixel distribution of the image dataset. This attribute establishes a connection between the input and output of the convolutional layer and reflects the overall information of the channels. Then, during the fine-tuning, we formulate a pruning strategy by observing the changes in δE to improve the adaptability of the pruned model to a specific dataset, and call this method DExp. Extensive experiments on representative image classification tasks reveal that DExp outper-forms mainstream methods. For example, with DExp, on CIFAR-10, the FLOPs of ResNet-56 are reduce by 53.99%, and the accuracy improved by 0.38%; on ILSVRC-2012, the FLOPs of ResNet-50 are reduced by 58.43%, and the Top-1 accuracy only loses by 1.15%. The code is available at: https://github.com/EXP-Pruning/DExp pruning.