AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
We assert the problem of attribution preservation in compressed deep neural networks based on the observation that compression techniques significantly alters the generated attributions

Attribution Preservation in Network Compression for Reliable Network Interpretation

NIPS 2020, (2020)

Cited by: 2|Views118
EI
Full Text
Bibtex
Weibo

Abstract

Neural networks embedded in safety-sensitive applications such as self-driving cars and wearable health monitors rely on two important techniques: input attribution for hindsight analysis and network compression to reduce its size for edge-computing. In this paper, we show that these seemingly unrelated techniques conflict with each oth...More

Code:

Data:

0
Introduction
  • Riding on the recent success of deep learning in numerous fields, there is an emergent trend to utilize deep neural networks (DNNs) even for safety-critical applications such as self-driving cars and wearable health monitors.
  • To guarantee reliable service, the DNNs must be embedded on the edge device
  • To this end, network compression techniques such as pruning [1, 2] and distillation [3, 4] are commonly employed - as a compressed network would require less computational time and memory but maintain its prediction performance to a certain acceptable margin, effectively substituting the original network for edge computation.
  • For certain kinds of algorithms such as network sparsification [20], steps two and three can be executed simultaneously
Highlights
  • Riding on the recent success of deep learning in numerous fields, there is an emergent trend to utilize deep neural networks (DNNs) even for safety-critical applications such as self-driving cars and wearable health monitors
  • We assert the problem of attribution preservation in compressed deep neural networks based on the observation that compression techniques significantly alters the generated attributions
  • We propose our attribution map matching framework which effectively and efficiently enforces the attribution maps of the compressed networks to be the same as those of the full networks
  • The results show that our framework preserves the interpretation of the original networks and yields significant performance gains over the model without attribution preservation
  • As discussed in Section 1, we believe that people trying to deploy deep learning models to safety-critical fields must be aware of this finding to ensure the reliability and trustworthiness of the system
  • Suppose that a Convolutional Neural Networks (CNN) classifier vision module trained with our matching regularizer is utilized in a self-driving system
Methods
  • Full (Teacher)

    For samples with correct pred. #Param AUC Point Acc 15.22M 88.79 80.21 attribution maps.

    Knowledge Distillation 0.29M 78.74 67.26

    To resolve this newfound unintended issue, the authors propose a novel attributionaware compression framework to ensure

    Structured Pruning Unstructured Pruning KD (w/ Ours)

    75.29 75.43 79.12 both the reliability and trustworthiness of the compressed model.
  • The authors observe that the network trained with the framework effectively preserves the attribution maps, and consistently outperforms the network distilled without the method in terms of prediction performance, which is measured in mean-average-precision and F1 score.
  • This result is partly expected from the work [4].
  • The authors use random resized crop and random horizontal flip provided by Torchvision and Pytorch. [33]
Conclusion
  • The authors assert the problem of attribution preservation in compressed deep neural networks based on the observation that compression techniques significantly alters the generated attributions
  • To this end, the authors propose the attribution map matching framework which effectively and efficiently enforces the attribution maps of the compressed networks to be the same as those of the full networks.
  • In case of an accident, the authors may inspect the records of the deep learning module to learn the decision that caused the accident
  • In this situation, the model trained with the regularizer will provide more accurate attribution, leading to a cleaner and more just assessment
Tables
  • Table1: Evaluation of how many samples were broken compared to the ground truth (segmentation labels) by various compression methods. Here, AUC denotes the degree of overlap between the segmentation and attribution map (see Section 4). Point accuracy [<a class="ref-link" id="c8" href="#r8">8</a>] is a measure of whether the max value of the standard decision procedures and resort heatmap is inside the segmentation map. Only the samples that to using shortcuts and hints that are the predictions of the network were correct are counted
  • Table2: Results of knowledge distillation models evaluated against
  • Table3: Knowledge distillation results the ground truth (segmentation)
  • Table4: Unstructured pruning models evaluated against ground truth (segmentation). Among the results of iterative pruning, the last remaining small-est network was evaluated
  • Table5: Unstructured pruning results for attribution map deformation from teacher to student network
  • Table6: Table 6
  • Table7: Table 7
  • Table8: Attribution deformation and preservation results on other attribution methods. For this experiment, we use the knowledge distillation with VGG/8. We report the AUC and Point accuracy to evaluate the localization ability of the attribution maps
  • Table9: ROC-AUC of four attribution methods on different network compression methods for the PASCALVOC dataset
  • Table10: Point accuracy of four attribution methods on different network compression methods for the PASCALVOC dataset
  • Table11: Results of unstructured pruning on ImageNet
  • Table12: Results of 1-structured pruning on ImageNet
Download tables as Excel
Related work
  • Attribution Methods Recent advances in producing human-understandable explanations for predictions of DNNs have gained much attention throughout the machine learning community. Among a variety of approaches towards this goal, one widely adopted method of interpretation is input attribution. Attribution approaches try to explain deep neural networks by producing visual explanations about the decisions of the network. By examining how the network’s output reacts to change in the input, the contributions of each input variable are calculated. In computer vision, these contributions are displayed in a 2-D manner, forming an attribution map. Attribution maps identify the spatial locations of the parts of the image the network deems significant in producing such a decision. Early works toward this direction use the gradient of the network output with respect to the input pixels to represent the sensitivity and significance of specific input pixels [9, 10, 11]. More recent studies such as Guided Backprop [12], Grad-Cam [6] or integrated gradients [13] proposed to process and combine these gradient signals in more careful ways. Another line of works proposed to propagate relevance values in a way that their total amount is preserved for a single layer. These relevance scores are backpropagated through the network from the output layer to the input layer. Several studies such as EBP [14], LRP [15] proposed to define novel relevance scores differing from vanilla gradients and backpropagate these values according to a set of novel backpropagation rules.
Funding
  • Acknowledgments and Disclosure of Funding This work was supported by the National Research Foundation of Korea (NRF) grants (No.2018R1A5A1059921, No.2019R1C1C1009192), Institute of Information & Communications Technology Planning & Evaluation (IITP) grants (No.2017-0-01779, A machine learning and statistical inference framework for explainable artificial intelligence, No.2019-0-01371, Development of brain-inspired AI with human-like intelligence, No.2019-0-00075, Artificial Intelligence Graduate School Program (KAIST)) funded by the Korea government (MSIT)
  • This work is also supported by Samsung Advanced Institute of Technology (SAIT)
Study subjects and analysis
students: 3
For our experiments on knowledge distillation, we use the standard network distillation technique introduced in [3]: we train a smaller student model using a linear combination of the typical crossentropy loss with ground truth label and the KL divergence between the teacher and student output logits. we use the VGG16 network [10] and create smaller student versions of the VGG16 network by maintaining the overall architecture but reducing the number of channels for all layers. We prepare 3 students: one-half (VGG16/2), one-quarter (VGG16/4), and one-eighth (VGG16/8). The teacher network is first initialized with off-the-shelf ImageNet pretrained weights, then trained with the measuring the deformation of attribution maps from teacher to student

Reference
  • Song Han, Huizi Mao, and William J. Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. In International Conference on Learning Representations, 2015.
    Google ScholarLocate open access versionFindings
  • Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710, 2016.
    Findings
  • Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
    Findings
  • S. Zagoruyko and N. Komodakis. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. In International Conference on Learning Representations, 2017.
    Google ScholarLocate open access versionFindings
  • Jie Xu, Kanmin Xue, and Kang Zhang. Current status and future trends of clinical diagnoses via image-based deep learning. Theranostics, 9(25):7556, 2019.
    Google ScholarLocate open access versionFindings
  • Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618–626, 2017.
    Google ScholarLocate open access versionFindings
  • Bin Liang, Yuan Tian, Xinyuan Chen, Hui Yan, Lingling Yan, Tao Zhang, Zongmei Zhou, Lvhua Wang, and Jianrong Dai. Prediction of radiation pneumonitis with dose distribution: A convolutional neural network (cnn) based model. Frontiers in oncology, 9:1500, 2020.
    Google ScholarLocate open access versionFindings
  • Vitali Petsiuk, Abir Das, and Kate Saenko. Rise: Randomized input sampling for explanation of black-box models. In Proceedings of the British Machine Vision Conference (BMVC), 2018.
    Google ScholarLocate open access versionFindings
  • Bolei Zhou, Aditya Khosla, Àgata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. CoRR, abs/1512.04150, 2015.
    Findings
  • Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013.
    Findings
  • Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. Visualizing higher-layer features of a deep network. University of Montreal, 2009.
    Google ScholarFindings
  • Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806, 2014.
    Findings
  • Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 3319–3328. JMLR. org, 2017.
    Google ScholarLocate open access versionFindings
  • Jianming Zhang, Sarah Adel Bargal, Zhe Lin, Jonathan Brandt, Xiaohui Shen, and Stan Sclaroff. Top-down neural attention by excitation backprop. In European Conference on Computer Vision, 2016.
    Google ScholarLocate open access versionFindings
  • Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller. Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 73:1–15, 2018.
    Google ScholarLocate open access versionFindings
  • Yann LeCun, John S Denker, and Sara A Solla. Optimal brain damage. In Advances in neural information processing systems, pages 598–605, 1990.
    Google ScholarLocate open access versionFindings
  • Babak Hassibi, David G Stork, and Gregory Wolff. Optimal brain surgeon: Extensions and performance comparisons. In Advances in neural information processing systems, pages 263– 270, 1994.
    Google ScholarLocate open access versionFindings
  • Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network. In Advances in Neural Information Processing Systems, pages 1135–1143, 2015.
    Google ScholarLocate open access versionFindings
  • Hengyuan Hu, Rui Peng, Yu-Wing Tai, and Chi-Keung Tang. Network trimming: A data-driven neuron pruning approach towards efficient deep architectures. arXiv preprint arXiv:1607.03250, 2016.
    Findings
  • Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. Learning structured sparsity in deep neural networks. In Advances in neural information processing systems, pages 2074–2082, 2016.
    Google ScholarLocate open access versionFindings
  • Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550, 2014.
    Findings
  • Yunhun Jang, Hankook Lee, Sung Ju Hwang, and Jinwoo Shin. Learning what and where to transfer. In Proceedings of the 36th International Conference on Machine Learning. PMLR, 2019.
    Google ScholarLocate open access versionFindings
  • Sungsoo Ahn, Shell Xu Hu, Andreas Damianou, Neil D Lawrence, and Zhenwen Dai. Variational information distillation for knowledge transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9163–9171, 2019.
    Google ScholarLocate open access versionFindings
  • Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929–1958, 2014.
    Google ScholarLocate open access versionFindings
  • Durk P Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparameterization trick. In Advances in neural information processing systems, pages 2575– 2583, 2015.
    Google ScholarLocate open access versionFindings
  • Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, and Kate Saenko. Adversarial dropout regularization. arXiv preprint arXiv:1711.01575, 2017.
    Findings
  • Sang-Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung-Woo Ha, and Byoung-Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. In Advances in neural information processing systems, pages 4652–4662, 2017.
    Google ScholarLocate open access versionFindings
  • M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http://www.pascalnetwork.org/challenges/VOC/voc2012/workshop/index.html, 2012.
    Locate open access versionFindings
  • Woo-Jeoung Nam, Shir Gur, Jaesik Choi, Lior Wolf, and Seong-Whan Lee. Relative attributing propagation: Interpreting the comparative contributions of individual units in deep neural networks, 2019.
    Google ScholarFindings
  • Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211–252, 2015.
    Google ScholarLocate open access versionFindings
  • Jang Hyun Cho and Bharath Hariharan. On the efficacy of knowledge distillation. In Proceedings of the IEEE International Conference on Computer Vision, pages 4794–4802, 2019.
    Google ScholarLocate open access versionFindings
  • Daniel Kuettel, Matthieu Guillaumin, and Vittorio Ferrari. Segmentation propagation in imagenet. In European Conference on Computer Vision, pages 459–473.
    Google ScholarLocate open access versionFindings
  • Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
    Google ScholarFindings
  • Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. A benchmark for interpretability methods in deep neural networks. In Advances in Neural Information Processing Systems, pages 9737–9748, 2019.
    Google ScholarLocate open access versionFindings
Author
Geondo Park
Geondo Park
June Yong Yang
June Yong Yang
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科