Compression Repair for Feedforward Neural Networks Based on Model Equivalence Evaluation

In this paper, we propose a method of repairing compressed Feedforward NeuralNetworks (FNNs) based on equivalence evaluation of two neural networks. In therepairing framework, a novel neural network equivalence evaluation method isdeveloped to compute the output discrepancy between two neural networks. Theoutput discrepancy can quantitatively characterize the output differenceproduced by compression procedures. Based on the computed output discrepancy,the repairing method first initializes a new training set for the compressednetworks to narrow down the discrepancy between the two neural networks andimprove the performance of the compressed network. Then, we repair thecompressed FNN by re-training based on the training set. We apply our developedmethod to the MNIST dataset to demonstrate the effectiveness and advantages ofour proposed repair method.