Efficient Network Compression Via Gradient-Score Aware Pruning

Convolutional neural networks (CNNs) have demonstrated significant achievements in the field of computer vision, yet their high computational demands restrict practical applications. Current pruning methods seek to mitigate this issue, which however often rely on heuristic manual approaches, encountering challenges in maintaining both significant model compression and accuracy. To address the above issues, a fast neural architecture search pruning (FNP) technique is proposed in this paper. Firstly, an importance matrix (IM) based preprocessing stage efficiently removes redundant structures by considering both weight importance and computational complexity, providing a compact baseline for subsequent pruning. Secondly, we adapt fast genetic algorithms (FGA) to identify optimally pruned model configurations. Furthermore, to accelerate the search process, we utilize a zero-shot learning approach to estimate model performance with the score of the frame (SoF), which is a gradient-based score. Compared with state-of-the-art (SOTA) pruning techniques, FNP demonstrates superior performance in terms of search duration and compression ratio. On the CIFAR-10 dataset, our method removes 95.24 % of the parameters in VGG-16 while achieving a 0.72 % accuracy improvement compared with the baseline. On the ImageNet dataset, we prune 68.98 % of the parameters in ResNet-50 and obtain a 1.2 % accuracy improvement compared with state-of-the-art (SOTA) approaches, while reducing the search time by 98.94 %. The code is available at https://github.com/aqiu1222/FNP.git