AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
For regulators, it is of utter importance to understand the certified properties of different certification methods precisely, so to avoid legal model deployment in safety critical applications based on misconceptions

Certified Defense to Image Transformations via Randomized Smoothing

NIPS 2020, (2020)

Cited by: 0|Views17
EI
Full Text
Bibtex
Weibo

Abstract

We extend randomized smoothing to cover parameterized transformations (e.g., rotations, translations) and certify robustness in the parameter space (e.g., rotation angle). This is particularly challenging as interpolation and rounding effects mean that image transformations do not compose, in turn preventing direct certification of the pe...More

Code:

Data:

0
Introduction
  • Deep neural networks are vulnerable to adversarial examples [1] – small changes that preserve semantics (e.g., p-noise or geometric transformations such as rotations) [2], but can affect the output of a network in undesirable ways.
  • Guarantees with p norms When considering p norms, existing certification methods can be directly used to obtain either of the above two guarantees: for an image x and adversarial noise δ, δ p < r, proving that a classifier f is r-robust around x := x + δ is enough to guarantee f (x) = f (x )
  • That is, it suffices to prove robustness of a perturbed input in order to certify that the perturbation did not change the classification, as the r-ball around x includes x
Highlights
  • Deep neural networks are vulnerable to adversarial examples [1] – small changes that preserve semantics (e.g., p-noise or geometric transformations such as rotations) [2], but can affect the output of a network in undesirable ways
  • Certification guarantees There are two principal robustness guarantees a certified defense can provide at inference time: (i) the distributional guarantee, where a robustness score is computed offline on the test set to be interpreted in expectation for images drawn from the data distribution, and (ii) an individual guarantee, where a certificate is computed online for the input
  • We presented two certified defenses with distributional and individual guarantees
  • We showed both defenses are applicable in an online setting and realistic datasets
  • Especially for regulators, it is of utter importance to understand the certified properties of different certification methods precisely, so to avoid legal model deployment in safety critical applications based on misconceptions
Results
  • A thorough evaluation of all methods on common image datasets, achieving provable distributional robust accuracy of 73% for rotations with up to ±30◦ on Restricted ImageNet. On MNIST they report 87.01% of certified accuracy for rotations with ±30◦ (35s per image), which with further refinement can be increased to 97%, and for translations with ±2 pixels 76.30% (263 s per image)
Conclusion
  • The authors presented a generalization of randomized smoothing to image transformations, a challenging task as image transformations do not compose
  • Based on this generalization, the authors presented two certified defenses with distributional and individual guarantees.
  • Methods from artificial intelligence can be applied in beneficial and malicious ways
  • While this poses a threat in itself, verification techniques provide formal guarantees for the robustness of the model, independently of the intended use case.
  • Especially for regulators, it is of utter importance to understand the certified properties of different certification methods precisely, so to avoid legal model deployment in safety critical applications based on misconceptions
Tables
  • Table1: Evaluation of BASESPT. We obtain Acc angular vignette for rotation and translation re- for b on the test set and evaluate adv. Acc. on spectively, to reduce error estimates in areas of 3000 images obtained by the worst-of-100 attack
  • Table2: Evaluation of DISTSPT for T I := RI . max is computed on the training set. We show the test set accuracy of b, certified accuracy of g and distribution of the obtained certification radius rγ, along with the average run time t and the number of used samples nγ, nδ
Download tables as Excel
Related work
  • We now survey the most closely related work in neural network certification and defenses.

    p norm based certification and defenses The discovery of adversarial examples [1, 14] triggered interest in training and certifying robust neural networks. An attempt to improve model robustness are empirical defenses [15, 16], strategies which harden a model against an adversary. While this may improve robustness to current adversaries, typically robustness cannot be formally verified with current certification methods. This is because complete methods [17,18,19] do not scale and incomplete methods relying on over approximation lose too much precision [3, 20, 21, 6, 10, 22], even for networks trained to be amenable to certification. Recently, randomized smoothing was introduced, which could for the first time, certify a (smoothed) classifier against norm bound 2 noise on ImageNet [23, 24, 7, 8, 25], by relaxing exact certificates to high confidence probabilistic ones. Smoothing scales to large models, however, it is currently limited to norm-based perturbations.
Funding
  • We do not have any additional funding or compensation to disclose
Study subjects and analysis
samples: 3000
Acc. we use the worst-of-k proposed by Engstrom et al [2], which returns the γ yielding the highest cross-entropy loss out of k randomly sampled γ ∼ U (Γ). We apply worst-of-k to 1000 images and produce 3 attacked images each, resulting 3000 samples on which we then evaluate b and g. For g, the average inference time per image t is generally fast, where most time is spent on sampling transformations

samples: 10
However for large images (ImageNet), the optimization over γ for many images is computationally expensive. Thus, for ImageNet we replace the max in Eq (6) with the maximum over 10 samples γ ∈ U(Γ). This formally restricts the certificate to only hold against random attacks

samples: 1000
Here we use αE = 0.001 and expect qE to be close to 1 for all datasets. We show qE ≥ 0.99 with confidence 0.999 by using 1000 samples for x and 8000 for β (and correction for possible test errors over β). Subsequently, we evaluate the accuracy of b and g

samples: 1000
Subsequently, we evaluate the accuracy of b and g. For b we use the whole test set, while for g we use 1000 samples. We clip obtained robustness radius rγ to Γ (indicated by †) in order to provide a sound guarantee

samples: 100
To these images we then apply INDIVSPT. For rotations (Γ± = 10, σγ = 30, nγ = 2000, 3 attacks per image, 1000 images) we fix E = 0.7 and use 100 samples of β to obtain the correct αE (Eq (9)). g was correct on 98% of attacked images. For 76% of these we could certify that the attacked image that classifies the same as the original

Reference
  • Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In Yoshua Bengio and Yann LeCun, editors, 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL http://arxiv.org/abs/1312.6199.
    Findings
  • Logan Engstrom, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. A rotation and a translation suffice: Fooling cnns with simple transformations. CoRR, abs/1712.02779, 2017. URL http://arxiv.org/abs/1712.02779.
    Findings
  • Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, and Martin T. Vechev. AI2: safety and robustness certification of neural networks with abstract interpretation. In 2018 IEEE Symposium on Security and Privacy, SP 2018, Proceedings, 21-23 May 2018, San Francisco, California, USA, pages 3–18. IEEE Computer Society, 2018. doi: 10.1109/SP.2018.00058. URL https://doi.org/10.1109/SP.2018.00058.
    Locate open access versionFindings
  • Matthew Mirman, Timon Gehr, and Martin T. Vechev. Differentiable abstract interpretation for provably robust neural networks. In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 3575–3583. PMLR, 2018. URL http://proceedings.mlr.press/v80/mirman18b.html.
    Locate open access versionFindings
  • Eric Wong and J. Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 5283–5292. PMLR, 2018. URL http://proceedings.mlr.press/v80/wong18a.html.
    Locate open access versionFindings
  • Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Semidefinite relaxations for certifying robustness to adversarial examples. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montréal, Canada., pages 10900–10910, 2018. URL http://papers.nips.cc/paper/8285-semidefinite-relaxations-for-certifying-robustness-to-adversarial-examples.
    Locate open access versionFindings
  • Jeremy M. Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via randomized smoothing. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 1310–1320. PMLR, 2019. URL http://proceedings.mlr.press/v97/cohen19c.html.
    Locate open access versionFindings
  • Hadi Salman, Greg Yang, Jerry Li, Pengchuan Zhang, Huan Zhang, Ilya P. Razenshteyn, and Sébastien Bubeck. Provably robust deep learning via adversarially trained smoothed classifiers. CoRR, abs/1906.04584, 2019. URL http://arxiv.org/abs/1906.04584.
    Findings
  • Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. Towards practical verification of machine learning: The case of computer vision systems. CoRR, abs/1712.01785, 2017. URL http://arxiv.org/abs/1712.01785.
    Findings
  • Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin T. Vechev. An abstract domain for certifying neural networks. PACMPL, 3(POPL):41:1–41:30, 2019. doi: 10.1145/3290354. URL https://doi.org/10.1145/3290354.
    Locate open access versionFindings
  • Mislav Balunovic, Maximilian Baader, Gagandeep Singh, Timon Gehr, and Martin T. Vechev. Certifying geometric robustness of neural networks. In NeurIPS, pages 15287–15297, 2019.
    Google ScholarLocate open access versionFindings
  • Jeet Mohapatra, Tsui-Wei Weng, Pin-Yu Chen, Sijia Liu, and Luca Daniel. Towards verifying robustness of neural networks against semantic perturbations. CoRR, abs/1912.09533, 2019.
    Findings
  • Linyi Li, Maurice Weber, Xiaojun Xu, Luka Rimanic, Tao Xie, Ce Zhang, and Bo Li. Provable robust learning based on transformation-specific smoothing. CoRR, abs/2002.12398, 2020.
    Findings
  • Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Srndic, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In ECML/PKDD (3), volume 8190 of Lecture Notes in Computer Science, pages 387–402.
    Google ScholarLocate open access versionFindings
  • Xiaoyu Cao and Neil Zhenqiang Gong. Mitigating evasion attacks to deep neural networks via region-based classification. In Proceedings of the 33rd Annual Computer Security Applications Conference, Orlando, FL, USA, December 4-8, 2017, pages 278–287. ACM, 2017. doi: 10.1145/3134600.3134606. URL https://doi.org/10.1145/3134600.3134606.
    Locate open access versionFindings
  • Xuanqing Liu, Minhao Cheng, Huan Zhang, and Cho-Jui Hsieh. Towards robust neural networks via random self-ensemble. In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part VII, volume 11211 of Lecture Notes in Computer Science, pages 381–397. Springer, 2018. doi: 10.1007/978-3-030-01234-2\_23. URL https://doi.org/10.1007/978-3-030-01234-2_23.
    Locate open access versionFindings
  • Rüdiger Ehlers. Formal verification of piece-wise linear feed-forward neural networks. In Deepak D’Souza and K. Narayan Kumar, editors, Automated Technology for Verification and Analysis - 15th International Symposium, ATVA 2017, Pune, India, October 3-6, 2017, Proceedings, volume 10482 of Lecture Notes in Computer Science, pages 269–286. Springer, 20doi: 10.1007/978-3-319-68167-2\_19. URL https://doi.org/10.1007/978-3-319-68167-2_19.
    Locate open access versionFindings
  • Guy Katz, Clark W. Barrett, David L. Dill, Kyle Julian, and Mykel J. Kochenderfer. Reluplex: An efficient SMT solver for verifying deep neural networks. In Rupak Majumdar and Viktor Kuncak, editors, Computer Aided Verification - 29th International Conference, CAV 2017, Heidelberg, Germany, July 24-28, 2017, Proceedings, Part I, volume 10426 of Lecture Notes in Computer Science, pages 97–117. Springer, 2017. doi: 10.1007/978-3-319-63387-9\_5. URL https://doi.org/10.1007/978-3-319-63387-9_5.
    Locate open access versionFindings
  • Rudy Bunel, Ilker Turkaslan, Philip H. S. Torr, Pushmeet Kohli, and Pawan Kumar Mudigonda. A unified view of piecewise linear neural network verification. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montréal, Canada., pages 4795–4804, 2018. URL http://papers.nips.cc/paper/7728-a-unified-view-of-piecewise-linear-neural-network-verification.
    Locate open access versionFindings
  • Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, and Suman Jana. Efficient formal safety analysis of neural networks. In NeurIPS, pages 6369–6379, 2018.
    Google ScholarLocate open access versionFindings
  • Tsui-Wei Weng, Huan Zhang, Hongge Chen, Zhao Song, Cho-Jui Hsieh, Luca Daniel, Duane S. Boning, and Inderjit S. Dhillon. Towards fast computation of certified robustness for relu networks. In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 5273–5282. PMLR, 2018. URL http://proceedings.mlr.press/v80/weng18a.html.
    Locate open access versionFindings
  • Hadi Salman, Greg Yang, Huan Zhang, Cho-Jui Hsieh, and Pengchuan Zhang. A convex relaxation barrier to tight robustness verification of neural networks. In NeurIPS, pages 9832– 9842, 2019.
    Google ScholarLocate open access versionFindings
  • Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness to adversarial examples with differential privacy. 2019 IEEE Symposium on Security and Privacy (SP), pages 656–672, 2018.
    Google ScholarLocate open access versionFindings
  • Bai Li, Changyou Chen, Wenlin Wang, and Lawrence Carin. Second-order adversarial attack and certifiable robustness. CoRR, abs/1809.03113, 2018. URL http://arxiv.org/abs/1809.03113.
    Findings
  • Runtian Zhai, Chen Dan, Di He, Huan Zhang, Boqing Gong, Pradeep Ravikumar, Cho-Jui Hsieh, and Liwei Wang. Macer: Attack-free and scalable robust training via maximizing certified radius. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=rJx1Na4Fwr.
    Locate open access versionFindings
  • Can Kanbak, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Geometric robustness of deep networks: Analysis and improvement. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 4441–4449. IEEE Computer Society, 2018. doi: 10.1109/CVPR. 2018.00467. URL http://openaccess.thecvf.com/content_cvpr_2018/html/Kanbak_ Geometric_Robustness_of_CVPR_2018_paper.html.
    Locate open access versionFindings
  • Hend Dawood. Theories of interval arithmetic: mathematical foundations and applications. LAP Lambert Academic Publishing, 2011.
    Google ScholarFindings
  • Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
    Google ScholarFindings
  • Logan Engstrom, Andrew Ilyas, Shibani Santurkar, and Dimitris Tsipras. Robustness (python library), 2019. URL https://github.com/MadryLab/robustness.
    Findings
  • Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211–252, 2015. doi: 10.1007/s11263-015-0816-y.
    Locate open access versionFindings
  • Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL https://openreview.net/forum?id=SyxAb30cY7.
    Locate open access versionFindings
  • Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
    Google ScholarFindings
  • Yann LeCun, Bernhard E. Boser, John S. Denker, Donnie Henderson, Richard E. Howard, Wayne E. Hubbard, and Lawrence D. Jackel. Handwritten digit recognition with a back-propagation network. In David S. Touretzky, editor, Advances in Neural Information Processing Systems 2, [NIPS Conference, Denver, Colorado, USA, November 27-30, 1989], pages 396–404. Morgan Kaufmann, 1989. URL http://papers.nips.cc/paper/293-handwritten-digit-recognition-with-a-back-propagation-network.
    Locate open access versionFindings
  • Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Francis R. Bach and David M. Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pages 448–456. JMLR.org, 2015. URL http://proceedings.mlr.press/v37/ioffe15.html.
    Locate open access versionFindings
  • Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., 15(1):1929–1958, 2014. URL http://dl.acm.org/citation.cfm?id=2670313.
    Locate open access versionFindings
  • Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017.
    Google ScholarFindings
  • J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural Networks, (0):–, 2012. ISSN 08936080. doi: 10.1016/j.neunet.2012.02.016. URL http://www.sciencedirect.com/science/article/pii/S0893608012000457.
    Findings
  • Pete Warden. Speech commands: A dataset for limited-vocabulary speech recognition. CoRR, abs/1804.03209, 2018. URL http://arxiv.org/abs/1804.03209.
    Findings
  • S. Davis and P. Mermelstein. Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. In IEEE Transactions on Acoustics, Speech, and Signal Processing. IEEE, 1980.
    Google ScholarLocate open access versionFindings
Author
Marc Fischer
Marc Fischer
Maximilian Baader
Maximilian Baader
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科