Heuristic Black-box Adversarial Attacks on Video Recognition Models

Zhi-Peng Wei
Zhi-Peng Wei
Xingxing Wei
Xingxing Wei
Lingxi Jiang
Lingxi Jiang
Fengfeng Zhou
Fengfeng Zhou

national conference on artificial intelligence, 2020.

Cited by: 2|Bibtex|Views128
Other Links: academic.microsoft.com
Weibo:
We proposed a heuristic black-box adversarial attack algorithm for video recognition models

Abstract:

We study the problem of attacking video recognition models in the black-box setting, where the model information is unknown and the adversary can only make queries to detect the predicted top-1 class and its probability. Compared with the black-box attack on images, attacking videos is more challenging as the computation cost for searchin...More

Code:

Data:

0
Introduction
Highlights
  • Deep neural networks are vulnerable to adversarial samples (Goodfellow, Shlens, and Szegedy 2014; Szegedy et al 2013)
  • This paper studies the problem of generating adversarial samples to attack video recognition models in the blackbox settings, where the model is not revealed and we can only make queries to detect the predicted top-1 class and its probability
  • We propose an efficient black-box adversarial attack model that heuristically select a subset consisting of the key frames to generate adversarial perturbations
  • We proposed a heuristic black-box adversarial attack algorithm for video recognition models
  • To reduce query numbers and improve attack efficiency, our method explores the sparsity of adversarial perturbations in both temporal and spatial domains
  • For C3D, introducing temporal sparsity alone helps to reduce the query numbers around 9% while introducing both spatial and temporal sparsity reduce the query numbers for more than 28% on UCF-101
  • The experimental results demonstrate that video recognition models are vulnerable to adversarial attack, and our algorithm achieves small human-imperceptible perturbation using fewer queries
Methods
  • The authors introduce the proposed heuristic blackbox attack algorithm for video recognition models.
  • The DNN F (x) takes a clean video x ∈ RT ×W ×H×C as an input and output the top-1 class y and its probability P (y|x), where T , W , H, C denote the number of frames, width, height, and the number of channels respectively.
Results
  • As the perturbations are generated only for the salient regions of the selected frames, the proposed method significantly reduces the number of queries.
  • Extensive experiments on two benchmark data sets demonstrate that the proposed method is efficient and effective
  • It reduces more than 28% reduction in query numbers for the untargeted attack.
  • Compared to Optattack, the method that considers both temporal and spatial sparsity significantly reduces the number of queries.
  • Similar trends are observed as the untargeted attack
  • By introducing both spatial and temporal sparsity, the method significantly reduces the query numbers.
  • For C3D model, the query numbers have been reduced by more than 19.59% on both datasets
Conclusion
  • The authors proposed a heuristic black-box adversarial attack algorithm for video recognition models.
  • To reduce query numbers and improve attack efficiency, the method explores the sparsity of adversarial perturbations in both temporal and spatial domains.
  • The authors' algorithm is adaptive to multiple target models and video datasets and enjoys global sparsity and query efficiency improvement.
  • The experimental results demonstrate that video recognition models are vulnerable to adversarial attack, and the algorithm achieves small human-imperceptible perturbation using fewer queries.
  • The most pertinent area of future work is to further investigate the black-box attack for the targeted attack using fewer queries
Summary
  • Introduction:

    Deep neural networks are vulnerable to adversarial samples (Goodfellow, Shlens, and Szegedy 2014; Szegedy et al 2013).
  • Recent works have shown that adding a small humanimperceptible perturbation to a clean sample can fool the deep models, leading them to make wrong predictions with high confidence (Moosavi-Dezfooli et al 2017)
  • As results, it has raised serious security concerns for the deployment of deep models in security-critical applications, such as face recognition (Kurakin, Goodfellow, and Bengio 2016), video surveillance (Sultani, Chen, and Shah 2018), etc.
  • It is crucial to investigating the adversarial samples for video models
  • Methods:

    The authors introduce the proposed heuristic blackbox attack algorithm for video recognition models.
  • The DNN F (x) takes a clean video x ∈ RT ×W ×H×C as an input and output the top-1 class y and its probability P (y|x), where T , W , H, C denote the number of frames, width, height, and the number of channels respectively.
  • Results:

    As the perturbations are generated only for the salient regions of the selected frames, the proposed method significantly reduces the number of queries.
  • Extensive experiments on two benchmark data sets demonstrate that the proposed method is efficient and effective
  • It reduces more than 28% reduction in query numbers for the untargeted attack.
  • Compared to Optattack, the method that considers both temporal and spatial sparsity significantly reduces the number of queries.
  • Similar trends are observed as the untargeted attack
  • By introducing both spatial and temporal sparsity, the method significantly reduces the query numbers.
  • For C3D model, the query numbers have been reduced by more than 19.59% on both datasets
  • Conclusion:

    The authors proposed a heuristic black-box adversarial attack algorithm for video recognition models.
  • To reduce query numbers and improve attack efficiency, the method explores the sparsity of adversarial perturbations in both temporal and spatial domains.
  • The authors' algorithm is adaptive to multiple target models and video datasets and enjoys global sparsity and query efficiency improvement.
  • The experimental results demonstrate that video recognition models are vulnerable to adversarial attack, and the algorithm achieves small human-imperceptible perturbation using fewer queries.
  • The most pertinent area of future work is to further investigate the black-box attack for the targeted attack using fewer queries
Tables
  • Table1: Test Accuracy(%) of the target models. Model UCF-101 HMDB-51
  • Table2: Results of our algorithm with various ω in the untargeted attack. ω F R(%) M Q
  • Table3: Results of our algorithm with various φ in the untargeted attack. φ F R(%) M Q
  • Table4: Results of our algorithm with various ω in the targeted attack. ω F R(%) M Q
  • Table5: Results of our algorithm with various φ in the targeted attack. φ F R(%) M Q
  • Table6: Untargeted and targeted attacks against C3D/LRCN Models. For all attack models, the Fooling Rate (FR) is 100%
Download tables as Excel
Related work
Funding
  • The work was funded by the National Research Foundation, Prime Ministers Office, Singapore under its IRC@Singapore Funding Initiative, and the NSFC Projects (No.61806109)
  • The work was also funded by the Jilin Provincial Key Laboratory of Big Data Intelligent Computing (20180622002JC), the Education Department of Jilin Province (JJKH20180145KJ), and the startup grant of the Jilin University, the Bioknow MedAI Institute (BMCPP2018-001), the High Performance Computing Center of Jilin University, and the Fundamental Research Funds for the Central Universities, JLU
Reference
  • [Bradski 2000] Bradski, G. 2000. The OpenCV Library. Dr. Dobb’s Journal of Software Tools.
    Google ScholarLocate open access versionFindings
  • [Brendel, Rauber, and Bethge 2017] Brendel, W.; Rauber, J.; and Bethge, M. 2017. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. arXiv preprint arXiv:1712.04248.
    Findings
  • [Carlini and Wagner 2017] Carlini, N., and Wagner, D. 2017. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), 39– 57. IEEE.
    Google ScholarLocate open access versionFindings
  • [Chen et al. 2017] Chen, P.-Y.; Zhang, H.; Sharma, Y.; Yi, J.; and Hsieh, C.-J. 2017. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, 15–26. ACM.
    Google ScholarLocate open access versionFindings
  • [Cheng et al. 2018] Cheng, M.; Le, T.; Chen, P.-Y.; Yi, J.; Zhang, H.; and Hsieh, C.-J. 2018. Query-efficient hard-label black-box attack: An optimization-based approach. arXiv preprint arXiv:1807.04457.
    Findings
  • [Donahue et al. ] Donahue, J.; Anne Hendricks, L.; Guadarrama, S.; Rohrbach, M.; Venugopalan, S.; Saenko, K.; and Darrell, T. Long-term recurrent convolutional networks for visual recognition and description. In Proceedings of CVPR.
    Google ScholarLocate open access versionFindings
  • [Dong et al. 2018] Dong, Y.; Liao, F.; Pang, T.; Su, H.; Zhu, J.; Hu, X.; and Li, J. 2018. Boosting adversarial attacks with momentum. In Proceedings of CVPR.
    Google ScholarLocate open access versionFindings
  • [Goodfellow, Shlens, and Szegedy 2014] Goodfellow, I. J.; Shlens, J.; and Szegedy, C. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572.
    Findings
  • [Goodfellow, Shlens, and Szegedy 2015] Goodfellow, I. J.; Shlens, J.; and Szegedy, C. 2015. Explaining and harnessing adversarial examples. CoRR abs/1412.6572.
    Findings
  • [Hara, Kataoka, and Satoh 2018] Hara, K.; Kataoka, H.; and Satoh, Y. 2018. Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet? In Proceedings of CVPR.
    Google ScholarLocate open access versionFindings
  • [Hou and Zhang 2007] Hou, X., and Zhang, L. 2007. Saliency detection: A spectral residual approach. In Proceedings of CVPR.
    Google ScholarLocate open access versionFindings
  • [Ilyas et al. 2018] Ilyas, A.; Engstrom, L.; Athalye, A.; and Lin, J. 2018. Black-box adversarial attacks with limited queries and information. arXiv preprint arXiv:1804.08598.
    Findings
  • [Inkawhich et al. 2019] Inkawhich, N.; Inkawhich, M.; Li, H.; and Chen, Y. 2019. Adversarial attacks for optical flowbased action recognition classifiers.
    Google ScholarFindings
  • [Kuehne et al. 2011] Kuehne, H.; Jhuang, H.; Garrote, E.; Poggio, T.; and Serre, T. 2011. HMDB: a large video database for human motion recognition. In Proceedings of ICCV.
    Google ScholarLocate open access versionFindings
  • [Kurakin, Goodfellow, and Bengio 2016] Kurakin, A.; Goodfellow, I.; and Bengio, S. 2016. Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533.
    Findings
  • [Kurakin, Goodfellow, and Bengio 2017] Kurakin, A.; Goodfellow, I. J.; and Bengio, S. 2017. Adversarial machine learning at scale. CoRR abs/1611.01236.
    Findings
  • [Li et al. 2018] Li, S.; Neupane, A.; Paul, S.; Song, C.; Krishnamurthy, S. V.; Roy-Chowdhury, A. K.; and Swami, A. 2018. Stealthy adversarial perturbations against real-time video classification systems. CoRR abs/1807.00458.
    Findings
  • [Liu et al. 2016] Liu, Y.; Chen, X.; Liu, C.; and Song, D. 2016. Delving into transferable adversarial examples and black-box attacks. arXiv preprint arXiv:1611.02770.
    Findings
  • [Moosavi-Dezfooli et al. 2017] Moosavi-Dezfooli, S.-M.; Fawzi, A.; Fawzi, O.; and Frossard, P. 2017. Universal adversarial perturbations. In Proceedings of CVPR.
    Google ScholarLocate open access versionFindings
  • [Moosavi-Dezfooli, Fawzi, and Frossard 2016] MoosaviDezfooli, S.-M.; Fawzi, A.; and Frossard, P. 2016. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of CVPR.
    Google ScholarLocate open access versionFindings
  • [Papernot et al. 2016] Papernot, N.; McDaniel, P.; Jha, S.; Fredrikson, M.; Celik, Z. B.; and Swami, A. 2016. The limitations of deep learning in adversarial settings. In 2016 IEEE European Symposium on Security and Privacy (EuroS&P), 372–387. IEEE.
    Google ScholarLocate open access versionFindings
  • [Papernot et al. 2017] Papernot, N.; McDaniel, P.; Goodfellow, I.; Jha, S.; Celik, Z. B.; and Swami, A. 2017. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security, 506–519. ACM.
    Google ScholarLocate open access versionFindings
  • [Peng, Zhao, and Zhang 2018] Peng, Y.; Zhao, Y.; and Zhang, J. 2018. Two-stream collaborative learning with spatial-temporal attention for video classification. IEEE Transactions on Circuits and Systems for Video Technology 29(3):773–786.
    Google ScholarLocate open access versionFindings
  • [Sarkar et al. 2017] Sarkar, S.; Bansal, A.; Mahbub, U.; and Chellappa, R. 2017. Upset and angri: Breaking high performance image classifiers. arXiv preprint arXiv:1707.01159.
    Findings
  • [Su et al. 2009] Su, D.; Su, Z.; Wang, J.; Yang, S.; and Ma, J. 2009. Ucf-101, a novel omi/htra2 inhibitor, protects against cerebral ischemia/reperfusion injury in rats. The Anatomical Record: Advances in Integrative Anatomy and Evolutionary Biology: Advances in Integrative Anatomy and Evolutionary Biology 292(6):854–861.
    Google ScholarLocate open access versionFindings
  • [Su, Vargas, and Sakurai 2019] Su, J.; Vargas, D. V.; and Sakurai, K. 2019. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation.
    Google ScholarLocate open access versionFindings
  • [Sultani, Chen, and Shah 2018] Sultani, W.; Chen, C.; and Shah, M. 2018. Real-world anomaly detection in surveillance videos. In Proceedings of CVPR.
    Google ScholarLocate open access versionFindings
  • [Szegedy et al. 2013] Szegedy, C.; Zaremba, W.; Sutskever, I.; Bruna, J.; Erhan, D.; Goodfellow, I.; and Fergus, R. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199.
    Findings
  • [Szegedy et al. 2016] Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; and Wojna, Z. 2016. Rethinking the inception architecture for computer vision. In Proceedings of CVPR, 2818–2826.
    Google ScholarLocate open access versionFindings
  • [Tu et al. 2018] Tu, C.-C.; Ting, P.; Chen, P.-Y.; Liu, S.; Zhang, H.; Yi, J.; Hsieh, C.-J.; and Cheng, S.-M. 2018. Autozoom: Autoencoder-based zeroth order optimization method for attacking black-box neural networks. arXiv preprint arXiv:1805.11770.
    Findings
  • [Uesato et al. 2018] Uesato, J.; O’Donoghue, B.; Oord, A. v. d.; and Kohli, P. 2018. Adversarial risk and the dangers of evaluating against weak attacks. arXiv preprint arXiv:1802.05666.
    Findings
  • [Wei, Zhu, and Su 2019] Wei, X.; Zhu, J.; and Su, H. 2019. Sparse adversarial perturbations for videos. CoRR abs/1803.02536.
    Findings
Full Text
Your rating :
0

 

Tags
Comments