Rotation Consistent Margin Loss for Efficient Low-Bit Face Recognition

CVPR, pp. 6865-6875, 2020.

Cited by: 9|Bibtex|Views197|DOI:https://doi.org/10.1109/CVPR42600.2020.00690
EI
Other Links: dblp.uni-trier.de|academic.microsoft.com
Weibo:
We investigate the effect of quantization errors on face recognition and propose rotation consistent margin loss for efficient low-bit face recognition training

Abstract:

In this paper, we consider the low-bit quantization problem of face recognition (FR) under the open-set protocol. Different from well explored low-bit quantization on closed-set image classification task, the open-set task is more sensitive to quantization errors (QEs). We redefine the QEs in angular space and disentangle it into class er...More

Code:

Data:

0
Introduction
  • The problem of face recognition (FR) has been well investigated [41, 42, 12, 4, 25, 8, 46, 54] in recent years.
  • Quantization technique [6, 5, 31, 55, 56, 10] emerges as an elegant compression solution to address this problem.
  • The core idea of this method is to reduce bit-width of weights and activations by mapping continuous values to discrete integers, which can reduce the memory footprint and accelerate the inference directly.
  • Low bit-width may degrade accuracy due to quantization errors (QEs).
  • To minimize QEs, many methods have been proposed [53, 36, 2, 1], and have been demonstrated successful in the fields of closed-set computer
Highlights
  • The problem of face recognition (FR) has been well investigated [41, 42, 12, 4, 25, 8, 46, 54] in recent years
  • We argue that improving performance of low-bit face recognition models is not entirely equivalent to reducing the quantization errors, and propose a novel rotation consistent margin (RCM) loss function for efficient low-bit face recognition quantization
  • Labeled Face in the Wild [17] is a standard face verification testing dataset in unconstrained conditions, and all images are collected from the website
  • Compared with other popular loss functions, our proposed rotation consistent margin achieves state-of-the-art performance at different FPR
  • The proposed rotation consistent margin loss improves the performance of the low-bit model from the perspective of the discriminative essence of open-set tasks
  • We investigate the effect of quantization errors on face recognition and propose rotation consistent margin loss for efficient low-bit face recognition training
Methods
  • FP SoftMax l2-Softmax [35] CosFace [46] CosFace+Mimic ArcFace [8] ArcFace+Mimic LFW 4-bit 3-bit YTF Model ResNet18 MobileNetV2.
  • FP Softmax CosFace [46] CosFace+Mimic ArcFace [8] ArcFace+Mimic.
  • FP Softmax l2-Softmax [35] CosFace [46] CosFace+Mimic ArcFace [8] ArcFace+Mimic.
  • Accuracy(%) 4-bit 3-bit TPR (%) SoftMax
Results
  • LFW [17] is a standard face verification testing dataset in unconstrained conditions, and all images are collected from the website
  • It contains 13,233 face images from 5,749 identities with a total of 6,000 ground-truth matches.
  • The proposed RCM loss improves the performance of the low-bit model from the perspective of the discriminative essence of open-set tasks
  • It can be combined with previous methods to boost their performance further.
  • The authors re-implement the recent two state-of-the-art quantization methods Dorefa-Net [55] and DSQ [10] as quantization baselines and combine them with different loss functions
Conclusion
  • The authors investigate the effect of quantization errors on FR and propose rotation consistent margin loss for efficient low-bit FR training.
  • It is hoped that the substantial explorations will inspire more researches on quantization problems for the open-set scenario
Summary
  • Introduction:

    The problem of face recognition (FR) has been well investigated [41, 42, 12, 4, 25, 8, 46, 54] in recent years.
  • Quantization technique [6, 5, 31, 55, 56, 10] emerges as an elegant compression solution to address this problem.
  • The core idea of this method is to reduce bit-width of weights and activations by mapping continuous values to discrete integers, which can reduce the memory footprint and accelerate the inference directly.
  • Low bit-width may degrade accuracy due to quantization errors (QEs).
  • To minimize QEs, many methods have been proposed [53, 36, 2, 1], and have been demonstrated successful in the fields of closed-set computer
  • Methods:

    FP SoftMax l2-Softmax [35] CosFace [46] CosFace+Mimic ArcFace [8] ArcFace+Mimic LFW 4-bit 3-bit YTF Model ResNet18 MobileNetV2.
  • FP Softmax CosFace [46] CosFace+Mimic ArcFace [8] ArcFace+Mimic.
  • FP Softmax l2-Softmax [35] CosFace [46] CosFace+Mimic ArcFace [8] ArcFace+Mimic.
  • Accuracy(%) 4-bit 3-bit TPR (%) SoftMax
  • Results:

    LFW [17] is a standard face verification testing dataset in unconstrained conditions, and all images are collected from the website
  • It contains 13,233 face images from 5,749 identities with a total of 6,000 ground-truth matches.
  • The proposed RCM loss improves the performance of the low-bit model from the perspective of the discriminative essence of open-set tasks
  • It can be combined with previous methods to boost their performance further.
  • The authors re-implement the recent two state-of-the-art quantization methods Dorefa-Net [55] and DSQ [10] as quantization baselines and combine them with different loss functions
  • Conclusion:

    The authors investigate the effect of quantization errors on FR and propose rotation consistent margin loss for efficient low-bit FR training.
  • It is hoped that the substantial explorations will inspire more researches on quantization problems for the open-set scenario
Tables
  • Table1: Identification accuracy (%) of rank-1 of 4-bit ResNet18 on the MegaFace dataset. “+Mimic” refers to ArcFace+Mimic
  • Table2: Verification accuracy (%) on the LFW and the YTF datasets. The model ResNet18 is trained on WebFace
  • Table3: Identification accuracy (%) of rank-1 on MegaFace dataset. The size of distractor is 1M
  • Table4: Identification accuracy (%) of rank-1 4-bit ResNet18 on the MegaFace dataset. The size of distractor is 1M and the FP models are trained on MS1MV1
Download tables as Excel
Related work
  • 2.1. Large-scale face recognition

    Practical applications of FR are usually under the openset protocol, where test categories are different from training categories. Therefore, FR is regarded as a typical metric learning task [33, 40, 47, 25, 46], whose objective is to increase the intra-class compactness and interclass discrepancy. To this end, many loss functions have been proposed. Sun et al [41, 42] and Wang et al [50] combine softmax loss with contrast loss or center loss, respectively, to explicitly increase the margin between different classes or reduce the distance between positive sample pairs. FaceNet [38] adopts triplet loss to optimize the embedding features directly, and greatly boost the performance. Recent works [26, 25, 46, 8, 45] propose the cosinebased softmax loss function and incorporate with margin to enhance the feature discriminative power. In these methods, features and class weights are normalized to a fixed scale, and the hypersphere manifold distribution hypothesis arises correspondingly and is widely recognized due to the concise geometric interpretation and impressive performance achieved by those loss functions.
Funding
  • This work was supported in part by the National Natural Science Foundation of China under Grant Nos
Reference
  • Ron Banner, Yury Nahshan, Elad Hoffer, and Daniel Soudry. Aciq: Analytical clipping for integer quantization of neural networks. arXiv preprint arXiv:1810.05723, 2018.
    Findings
  • R Banner, Y Nahshan, E Hoffer, and D Soudry. Post training 4-bit quantization of convolution networks for rapiddeployment. CoRR, abs/1810.05723, 1:2, 2018.
    Findings
  • Zhaowei Cai, Xiaodong He, Jian Sun, and Nuno Vasconcelos. Deep learning with low precision by half-wave gaussian quantization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5918– 5926, 2017.
    Google ScholarLocate open access versionFindings
  • Ken Chen, Yichao Wu, Haoyu Qin, Ding Liang, Xuebo Liu, and Junjie Yan. R3 adversarial network for cross model face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9868– 9876, 2019.
    Google ScholarLocate open access versionFindings
  • Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. Binaryconnect: Training deep neural networks with binary weights during propagations. In Advances in neural information processing systems, pages 3123–3131, 2015.
    Google ScholarLocate open access versionFindings
  • Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.02830, 2016.
    Findings
  • deepglint. Face feature test/trillion pairs. http://trillionpairs.deepglint.com/overview.
    Findings
  • Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4690– 4699, 2019.
    Google ScholarLocate open access versionFindings
  • Jiong Gong, Haihao Shen, Guoming Zhang, Xiaoli Liu, Shane Li, Ge Jin, Niharika Maheshwari, Evarist Fomenko, and Eden Segal. Highly efficient 8-bit low precision inference of convolutional neural networks with intelcaffe. In Proceedings of the 1st on Reproducible Quality-Efficient Systems Tournament on Co-designing Pareto-efficient Deep Learning, page 2. ACM, 2018.
    Google ScholarLocate open access versionFindings
  • Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan. Differentiable soft quantization: Bridging full-precision and low-bit neural networks. In Proceedings of the IEEE International Conference on Computer Vision, pages 4852–4861, 2019.
    Google ScholarLocate open access versionFindings
  • Manuel Gunther, Steve Cruz, Ethan M Rudd, and Terrance E Boult. Toward open-set face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 71–80, 2017.
    Google ScholarLocate open access versionFindings
  • Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In European Conference on Computer Vision, pages 87–102.
    Google ScholarLocate open access versionFindings
  • Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149, 2015.
    Findings
  • Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
    Google ScholarLocate open access versionFindings
  • Lanqing He, Zhongdao Wang, Yali Li, and Shengjin Wang. Softmax dissection: Towards understanding intra-and interclas objective for embedding learning. arXiv preprint arXiv:1908.01281, 2019.
    Findings
  • Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7132–7141, 2018.
    Google ScholarLocate open access versionFindings
  • Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. 2008.
    Google ScholarFindings
  • Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran ElYaniv, and Yoshua Bengio. Binarized neural networks. In Advances in neural information processing systems, pages 4107–4115, 2016.
    Google ScholarLocate open access versionFindings
  • Benoit Jacob et al. gemmlowp: a small self-contained lowprecision gemm library.(2017), 2017.
    Google ScholarFindings
  • Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2704–2713, 2018.
    Google ScholarLocate open access versionFindings
  • Ira Kemelmacher-Shlizerman, Steven M Seitz, Daniel Miller, and Evan Brossard. The megaface benchmark: 1 million faces for recognition at scale. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4873–4882, 2016.
    Google ScholarLocate open access versionFindings
  • Fengfu Li, Bo Zhang, and Bin Liu. Ternary weight networks. arXiv preprint arXiv:1605.04711, 2016.
    Findings
  • Rundong Li, Feng Liang, Hongwei Qin, Yan Wang, Rui Fan, and Junjie Yan. Fully quantized network for object detection. In Proceedings of the IEEE International Conference on Computer Vision, 2019.
    Google ScholarLocate open access versionFindings
  • Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pages 2980–2988, 2017.
    Google ScholarLocate open access versionFindings
  • Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 212–220, 2017.
    Google ScholarLocate open access versionFindings
  • Weiyang Liu, Yandong Wen, Zhiding Yu, and Meng Yang. Large-margin softmax loss for convolutional neural networks. In ICML, volume 2, page 7, 2016.
    Google ScholarLocate open access versionFindings
  • Ping Luo, Zhenyao Zhu, Ziwei Liu, Xiaogang Wang, and Xiaoou Tang. Face model compression by distilling knowledge from neurons. In Thirtieth AAAI Conference on Artificial Intelligence, 2016.
    Google ScholarLocate open access versionFindings
  • Brianna Maze, Jocelyn Adams, James A Duncan, Nathan Kalka, Tim Miller, Charles Otto, Anil K Jain, W Tyler Niggel, Janet Anderson, Jordan Cheney, et al. Iarpa janus benchmark-c: Face dataset and protocol. In 2018 International Conference on Biometrics (ICB), pages 158–165. IEEE, 2018.
    Google ScholarLocate open access versionFindings
  • Jeffrey L McKinstry, Steven K Esser, Rathinakumar Appuswamy, Deepika Bablani, John V Arthur, Izzet B Yildiz, and Dharmendra S Modha. Discovering low-precision networks close to full-precision networks for efficient embedded inference. arXiv preprint arXiv:1809.04191, 2018.
    Findings
  • Szymon Migacz. 8-bit inference with tensorrt. In GPU technology conference, volume 2, page 7, 2017.
    Google ScholarLocate open access versionFindings
  • Asit Mishra and Debbie Marr. Apprentice: Using knowledge distillation techniques to improve low-precision network accuracy. arXiv preprint arXiv:1711.05852, 2017.
    Findings
  • Hong-Wei Ng and Stefan Winkler. A data-driven approach to cleaning large face datasets. In 2014 IEEE International Conference on Image Processing (ICIP), pages 343–347. IEEE, 2014.
    Google ScholarLocate open access versionFindings
  • Hyun Oh Song, Yu Xiang, Stefanie Jegelka, and Silvio Savarese. Deep metric learning via lifted structured feature embedding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4004– 4012, 2016.
    Google ScholarLocate open access versionFindings
  • Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, et al. Deep face recognition. In bmvc, volume 1, page 6, 2015.
    Google ScholarLocate open access versionFindings
  • Rajeev Ranjan, Carlos D Castillo, and Rama Chellappa. L2constrained softmax loss for discriminative face verification. arXiv preprint arXiv:1703.09507, 2017.
    Findings
  • Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. Xnor-net: Imagenet classification using binary convolutional neural networks. In European Conference on Computer Vision, pages 525–542.
    Google ScholarLocate open access versionFindings
  • Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4510–4520, 2018.
    Google ScholarLocate open access versionFindings
  • Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815–823, 2015.
    Google ScholarLocate open access versionFindings
  • Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
    Findings
  • Kihyuk Sohn. Improved deep metric learning with multiclass n-pair loss objective. In Advances in Neural Information Processing Systems, pages 1857–1865, 2016.
    Google ScholarLocate open access versionFindings
  • Yi Sun, Yuheng Chen, Xiaogang Wang, and Xiaoou Tang. Deep learning face representation by joint identificationverification. In Advances in neural information processing systems, pages 1988–1996, 2014.
    Google ScholarLocate open access versionFindings
  • Yi Sun, Xiaogang Wang, and Xiaoou Tang. Deeply learned face representations are sparse, selective, and robust. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2892–2900, 2015.
    Google ScholarLocate open access versionFindings
  • Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, and Lior Wolf. Deepface: Closing the gap to human-level performance in face verification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1701–1708, 2014.
    Google ScholarLocate open access versionFindings
  • Frederick Tung and Greg Mori. Clip-q: Deep network compression learning by in-parallel pruning-quantization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7873–7882, 2018.
    Google ScholarLocate open access versionFindings
  • Feng Wang, Jian Cheng, Weiyang Liu, and Haijun Liu. Additive margin softmax for face verification. IEEE Signal Processing Letters, 25(7):926–930, 2018.
    Google ScholarLocate open access versionFindings
  • Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5265–5274, 2018.
    Google ScholarLocate open access versionFindings
  • Jian Wang, Feng Zhou, Shilei Wen, Xiao Liu, and Yuanqing Lin. Deep metric learning with angular loss. In Proceedings of the IEEE International Conference on Computer Vision, pages 2593–2601, 2017.
    Google ScholarLocate open access versionFindings
  • Kuan Wang, Zhijian Liu, Yujun Lin, Ji Lin, and Song Han. Haq: Hardware-aware automated quantization with mixed precision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8612–8620, 2019.
    Google ScholarLocate open access versionFindings
  • Peisong Wang, Qinghao Hu, Yifan Zhang, Chunjie Zhang, Yang Liu, and Jian Cheng. Two-step quantization for lowbit neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4376– 4384, 2018.
    Google ScholarLocate open access versionFindings
  • Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A discriminative feature learning approach for deep face recognition. In European conference on computer vision, pages 499–515.
    Google ScholarLocate open access versionFindings
  • Lior Wolf, Tal Hassner, and Itay Maoz. Face recognition in unconstrained videos with matched background similarity. IEEE, 2011.
    Google ScholarFindings
  • Dong Yi, Zhen Lei, Shengcai Liao, and Stan Z Li. Learning face representation from scratch. arXiv preprint arXiv:1411.7923, 2014.
    Findings
  • Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, and Gang Hua. Lq-nets: Learned quantization for highly accurate and compact deep neural networks. In Proceedings of the European Conference on Computer Vision (ECCV), pages 365– 382, 2018.
    Google ScholarLocate open access versionFindings
  • Xiao Zhang, Rui Zhao, Yu Qiao, Xiaogang Wang, and Hongsheng Li. Adacos: Adaptively scaling cosine logits for effectively learning deep face representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 10823–10832, 2019.
    Google ScholarLocate open access versionFindings
  • Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160, 2016.
    Findings
  • Shu-Chang Zhou, Yu-Zhi Wang, He Wen, Qin-Yao He, and Yu-Heng Zou. Balanced quantization: An effective and efficient approach to quantized neural networks. Journal of Computer Science and Technology, 32(4):667–682, 2017.
    Google ScholarLocate open access versionFindings
  • Chenzhuo Zhu, Song Han, Huizi Mao, and William J Dally. Trained ternary quantization. arXiv preprint arXiv:1612.01064, 2016.
    Findings
  • Xiaotian Zhu, Wengang Zhou, and Houqiang Li. Adaptive layerwise quantization for deep neural network compression. In 2018 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE, 2018.
    Google ScholarLocate open access versionFindings
Full Text
Your rating :
0

 

Tags
Comments