PF-cpGAN: Profile to Frontal Coupled GAN for Face Recognition in the Wild

arxiv, 2020.

Cited by: 0|Views14
Weibo:
We thoroughly evaluated our model on several standard datasets and the results demonstrate that our model notably outperforms other state-of-theart algorithms for profile to frontal face verification

Abstract:

In recent years, due to the emergence of deep learning, face recognition has achieved exceptional success. However, many of these deep face recognition models perform relatively poorly in handling profile faces compared to frontal faces. The major reason for this poor performance is that it is inherently difficult to learn large pose in...More

Code:

Data:

0
Full Text
Bibtex
Weibo
Introduction
  • Due to the emergence of deep learning, face recognition has achieved exceptional success in recent years [3].
  • Many of these deep face recognition models perform relatively poorly in handling profile faces compared to frontal faces [36].
  • Expression, and lighting variations are considered to be major obstacles in attaining high unconstrained face recognition performance.
  • There are three major difficulties related to face frontalization or normalization in unconstrained environment:
Highlights
  • Due to the emergence of deep learning, face recognition has achieved exceptional success in recent years [3]
  • We propose an embedding model for profile to frontal face verification based on a deep coupled learning framework which uses a generative adversarial network (GAN) to find the hidden relationship between the profile face features and frontal face features in a latent common embedding subspace
  • We first perform evaluation on the Celebrities in FrontalProfile (CFP) dataset [36], a challenging dataset created to examine the problem of frontal to profile face verification in the wild
  • We focus on unconstrained face recognition on IARPA Janus Benchmark A dataset to quantify the superiority of our PF-coupled generative adversarial network
  • We proposed a new framework which uses a coupled generative adversarial network for profile to frontal face recognition
  • We thoroughly evaluated our model on several standard datasets and the results demonstrate that our model notably outperforms other state-of-theart algorithms for profile to frontal face verification
Methods
  • The authors describe the method for profile to frontal face recognition.
  • Inspired by the success of GANs [12], the authors explore adversarial networks to project profile and frontal images to a common subspace for recognition.
  • As shown in Table 2, the authors perform better than the state-of-the-art methods for both verification and identification.
  • The rank1 recognition rate shows an improvement of around 1.6% in comparison to the best state-of-the-art method, FNM [31]
Results
  • Evaluation on CFP with Frontal

    Profile Setting

    The authors first perform evaluation on the Celebrities in FrontalProfile (CFP) dataset [36], a challenging dataset created to examine the problem of frontal to profile face verification in the wild.
  • Evaluation on CFP with Frontal.
  • The authors first perform evaluation on the Celebrities in FrontalProfile (CFP) dataset [36], a challenging dataset created to examine the problem of frontal to profile face verification in the wild.
  • For fair comparison and as given in [36], the authors consider different types of feature extraction techniques like HoG [9], LBP [1], and Fisher Vector [38] along with metric learning techniques like Sub-SML [4], and Diagonal metric learning (DML) as reported in [38].
  • The authors focus on unconstrained face recognition on IJB-A dataset to quantify the superiority of the PF-cpGAN
Conclusion
  • The authors proposed a new framework which uses a coupled GAN for profile to frontal face recognition.
  • The coupled GAN contains two sub-networks which project the profile and frontal images into a common embedding subspace, where the goal of each sub-network is to maximize the pairwise correlation between profile and frontal images during the process of projection.
  • The improvement achieved by different losses in the proposed algorithm has been studied in an ablation study
Summary
  • Introduction:

    Due to the emergence of deep learning, face recognition has achieved exceptional success in recent years [3].
  • Many of these deep face recognition models perform relatively poorly in handling profile faces compared to frontal faces [36].
  • Expression, and lighting variations are considered to be major obstacles in attaining high unconstrained face recognition performance.
  • There are three major difficulties related to face frontalization or normalization in unconstrained environment:
  • Methods:

    The authors describe the method for profile to frontal face recognition.
  • Inspired by the success of GANs [12], the authors explore adversarial networks to project profile and frontal images to a common subspace for recognition.
  • As shown in Table 2, the authors perform better than the state-of-the-art methods for both verification and identification.
  • The rank1 recognition rate shows an improvement of around 1.6% in comparison to the best state-of-the-art method, FNM [31]
  • Results:

    Evaluation on CFP with Frontal

    Profile Setting

    The authors first perform evaluation on the Celebrities in FrontalProfile (CFP) dataset [36], a challenging dataset created to examine the problem of frontal to profile face verification in the wild.
  • Evaluation on CFP with Frontal.
  • The authors first perform evaluation on the Celebrities in FrontalProfile (CFP) dataset [36], a challenging dataset created to examine the problem of frontal to profile face verification in the wild.
  • For fair comparison and as given in [36], the authors consider different types of feature extraction techniques like HoG [9], LBP [1], and Fisher Vector [38] along with metric learning techniques like Sub-SML [4], and Diagonal metric learning (DML) as reported in [38].
  • The authors focus on unconstrained face recognition on IJB-A dataset to quantify the superiority of the PF-cpGAN
  • Conclusion:

    The authors proposed a new framework which uses a coupled GAN for profile to frontal face recognition.
  • The coupled GAN contains two sub-networks which project the profile and frontal images into a common embedding subspace, where the goal of each sub-network is to maximize the pairwise correlation between profile and frontal images during the process of projection.
  • The improvement achieved by different losses in the proposed algorithm has been studied in an ablation study
Tables
  • Table1: Performance comparison on CFP dataset. Mean Accuracy and equal error rate (EER) with standard deviation over 10 folds
  • Table2: Performance comparison on IJB-A benchmark. Results reported are the ’average±standard deviation’ over the 10 folds specified in the IJB-A protocol. Symbol ’-’ indicates that the metric is not available for that protocol
  • Table3: Performance comparison on IJB-C benchmark. Results reported are the ’average±standard deviation’ over the 10 folds specified in the IJB-C protocol. Symbol ’-’ indicates that the metric is not available for that protocol
  • Table4: Rank-1 recognition rates (%) across poses and illuminations under Multi-PIE Setting-1
Download tables as Excel
Related work
  • Face recognition using Deep Learning: Before the advent of deep learning, traditional methods for face recognition (FR) used one or more layer representations, such as the histogram of the feature codes, filtering responses, or distribution of the dictionary atoms [50]. FR research was concentrated more toward separately improving preprocessing, local descriptors, and feature transformation; however, overall improvement in FR accuracy was very slow. This all changed with the advent of deep learning, and now deep learning is the prominent technique used for FR.

    Recently various deep learning models such as [8, 43] are used as baseline model for FR. Simultaneously, various loss functions have been explored and used in FR. These loss functions can be categorized as the Euclidean-distancebased loss, angular/cosine-margin-based loss, and softmax loss and its variations. The contrastive loss and the triplet loss are the commonly used Euclidean-distance-based loss functions [34,40,41,42]. For avoiding misclassification of difficult samples [45, 46], the learned face features need to be well separated. Angular/cosine-margin based loss [2,10,25] are commonly used to make the learned features more separable with a larger angular/cosine distance. Finally, in the category of softmax loss and its variants for FR [14, 26, 49], the softmax loss is modified to improve the FR performance as in [26], where the cosine distance among data features is optimized along with normalization of features and weights.
Reference
  • T. Ahonen, A. Hadid, and M. Pietikainen. Face description with local binary patterns: Application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12):2037–2041, Dec 2006.
    Google ScholarLocate open access versionFindings
  • S. Boumi, A. Vela, and J. Chini. Quantifying the relationship between student enrollment patterns and student performance. arXiv preprint arXiv:2003.10874, 2020.
    Findings
  • K. Cao, Y. Rong, C. Li, X. Tang, and C. C. Loy. Poserobust face recognition via deep residual equivariant mapping. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5187–5196, 2018.
    Google ScholarLocate open access versionFindings
  • Q. Cao, Y. Ying, and P. Li. Similarity metric learning for face recognition. In 2013 IEEE International Conference on Computer Vision, pages 2408–2415, Dec 2013.
    Google ScholarLocate open access versionFindings
  • J. Chen, V. M. Patel, and R. Chellappa. Unconstrained face verification using deep cnn features. In IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1–9, March 2016.
    Google ScholarLocate open access versionFindings
  • J.-C. Chen, V. M. Patel, and R. Chellappa. Unconstrained face verification using deep cnn features. In 2016 IEEE winter conference on applications of computer vision (WACV), pages 1–9. IEEE, 2016.
    Google ScholarLocate open access versionFindings
  • F. Cole, D. Belanger, D. Krishnan, A. Sarna, I. Mosseri, and W. T. Freeman. Synthesizing normalized faces from facial identity features. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3386–3395, 2017.
    Google ScholarLocate open access versionFindings
  • A. Dabouei, F. Taherkhani, S. Soleymani, J. Dawson, and N. Nasrabadi. Boosting deep face recognition via disentangling appearance and geometry. In The IEEE Winter Conference on Applications of Computer Vision (WACV), 2020.
    Google ScholarLocate open access versionFindings
  • N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), volume 1, pages 886–893, June 2005.
    Google ScholarLocate open access versionFindings
  • J. Deng, J. Guo, N. Xue, and S. Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
    Google ScholarLocate open access versionFindings
  • C. Ding and D. Tao. Pose-invariant face recognition with homography-based normalization. Pattern Recognition, 66:144–152, 2017.
    Google ScholarLocate open access versionFindings
  • I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Proc. Neural Information Processing Systems, pages 2672–2680. 2014.
    Google ScholarLocate open access versionFindings
  • R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker. Multi-pie. In 2008 8th IEEE International Conference on Automatic Face Gesture Recognition, pages 1–8, Sep. 2008.
    Google ScholarLocate open access versionFindings
  • M. A. Hasnat, J. Bohne, J. Milgram, S. Gentric, and L. Chen. von mises-fisher mixture model-based deep learning: Application to face verification. ArXiv, abs/1706.04264, 2017.
    Findings
  • K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770– 778, 2015.
    Google ScholarLocate open access versionFindings
  • Y. Hu, X. Wu, B. Yu, R. He, and Z. Sun. Pose-guided photorealistic face rotation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8398–8406, 2018.
    Google ScholarLocate open access versionFindings
  • R. Huang, S. Zhang, T. Li, and R. He. Beyond face rotation: Global and local perception gan for photorealistic and identity preserving frontal view synthesis. In Proceedings of the IEEE International Conference on Computer Vision, pages 2439–2448, 2017.
    Google ScholarLocate open access versionFindings
  • P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros. Image-to-image translation with conditional adversarial networks. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5967–5976, 2017.
    Google ScholarLocate open access versionFindings
  • J. Jiang, R. Hu, Z. Wang, and Z. Cai. CDMMA: Coupled discriminant multi-manifold analysis for matching lowresolution face images. Signal Processing, 124:162–172, 2016.
    Google ScholarLocate open access versionFindings
  • J. Johnson, A. Alahi, and L. Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In Proc. European Conference on Computer Vision (ECCV), 2016.
    Google ScholarLocate open access versionFindings
  • D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2015.
    Findings
  • B. Klare, B. Klein, E. Taborsky, A. Blanton, J. Cheney, K. E. Allen, P. Grother, A. Mah, M. Burge, and A. K. Jain. Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus benchmark a. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1931–1939, 2015.
    Google ScholarLocate open access versionFindings
  • B. F. Klare, B. Klein, E. Taborsky, A. Blanton, J. Cheney, K. Allen, P. Grother, A. Mah, and A. K. Jain. Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus benchmark a. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1931–1939, 2015.
    Google ScholarLocate open access versionFindings
  • P. Li, L. Prieto, D. Mery, and P. J. Flynn. On low-resolution face recognition in the wild: Comparisons and new techniques. IEEE Transactions on Information Forensics and Security, page 11, 2019.
    Google ScholarLocate open access versionFindings
  • W. Liu, Y. Wen, Z. Yu, M. Li, B. Raj, and L. Song. Sphereface: Deep hypersphere embedding for face recognition. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6738–6746, June 2017.
    Google ScholarLocate open access versionFindings
  • Y. Liu, H. Li, and X. Wang. Rethinking feature discrimination and polymerization for large-scale recognition. ArXiv, abs/1710.00870, 2017.
    Findings
  • I. Masi, S. Rawls, G. Medioni, and P. Natarajan. Poseaware face recognition in the wild. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4838–4846, June 2016.
    Google ScholarLocate open access versionFindings
  • B. Maze, J. Adams, J. A. Duncan, N. Kalka, T. Miller, C. Otto, A. K. Jain, W. T. Niggel, J. Anderson, J. Cheney, and P. Grother. IARPA Janus benchmark - c: Face dataset and protocol. In 2018 International Conference on Biometrics (ICB), pages 158–165, Feb 2018.
    Google ScholarLocate open access versionFindings
  • M. Mirza and S. Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
    Findings
  • O. M. Parkhi, A. Vedaldi, and A. Zisserman. Deep face recognition. In BMVC, 2015.
    Google ScholarLocate open access versionFindings
  • Y. Qian, W. Deng, and J. Hu. Unsupervised face normalization with extreme pose and expression in the wild. In IEEE Conference on Computer Vision and Pattern Recognition, 2019.
    Google ScholarLocate open access versionFindings
  • S. Reed, Z. Akata, X. Yan, L. Logeswaran, B. Schiele, and H. Lee. Generative adversarial text to image synthesis. In International Conference on Machine Learning (ICML), 2016.
    Google ScholarLocate open access versionFindings
  • O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 234–241.
    Google ScholarLocate open access versionFindings
  • F. Schroff, D. Kalenichenko, and J. Philbin. Facenet: A unified embedding for face recognition and clustering. In Proc. IEEE conference on computer vision and pattern recognition, pages 815–823, 2015.
    Google ScholarLocate open access versionFindings
  • F. Schroff, D. Kalenichenko, and J. Philbin. Facenet: A unified embedding for face recognition and clustering. CoRR, abs/1503.03832, 2015.
    Findings
  • S. Sengupta, J. Chen, C. Castillo, V. M. Patel, R. Chellappa, and D. W. Jacobs. Frontal to profile face verification in the wild. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1–9, March 2016.
    Google ScholarLocate open access versionFindings
  • S. Shekhar, V. M. Patel, and R. Chellappa. Synthesisbased robust low resolution face recognition. arXiv preprint arXiv:1707.02733, 2017.
    Findings
  • K. Simonyan, O. M. Parkhi, A. Vedaldi, and A. Zisserman. Fisher vector faces in the wild. In BMVC, 2013.
    Google ScholarFindings
  • K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
    Findings
  • Y. Sun, Y. Chen, X. Wang, and X. Tang. Deep learning face representation by joint identification-verification. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS’14, pages 1988–1996, Cambridge, MA, USA, 2014. MIT Press.
    Google ScholarLocate open access versionFindings
  • F. Taherkhani, H. Kazemi, A. Dabouei, J. Dawson, and N. M. Nasrabadi. A weakly supervised fine label classifier enhanced by coarse supervision. In Proceedings of the IEEE International Conference on Computer Vision, pages 6459– 6468, 2019.
    Google ScholarLocate open access versionFindings
  • F. Taherkhani, H. Kazemi, and N. M. Nasrabadi. Matrix completion for graph-based deep semi-supervised learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 5058–5065, 2019.
    Google ScholarLocate open access versionFindings
  • F. Taherkhani, N. M. Nasrabadi, and J. Dawson. A deep face identification network enhanced by facial attributes prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 553–560, 2018.
    Google ScholarLocate open access versionFindings
  • Y. Taigman, M. Yang, M. Ranzato, and L. Wolf. Deepface: Closing the gap to human-level performance in face verification. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 1701–1708, June 2014.
    Google ScholarLocate open access versionFindings
  • V. Talreja, T. Ferrett, M. C. Valenti, and A. Ross. Biometricsas-a-service: A framework to promote innovative biometric recognition in the cloud. In 2018 IEEE ICCE, pages 1–6. IEEE, 2018.
    Google ScholarLocate open access versionFindings
  • V. Talreja, M. C. Valenti, and N. M. Nasrabadi. Multibiometric secure system based on deep learning. In 2017 IEEE Global conference on signal and information processing (globalSIP), pages 298–302. IEEE, 2017.
    Google ScholarLocate open access versionFindings
  • L. Tran, X. Yin, and X. Liu. Disentangled representation learning gan for pose-invariant face recognition. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1283–1292, July 2017.
    Google ScholarLocate open access versionFindings
  • L. Tran, X. Yin, and X. Liu. Disentangled representation learning gan for pose-invariant face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1415–1424, 2017.
    Google ScholarLocate open access versionFindings
  • F. Wang, X. Xiang, J. Cheng, and A. L. Yuille. Normface: L2 hypersphere embedding for face verification. In Proceedings of the 25th ACM International Conference on Multimedia, pages 1041–1049, 2017.
    Google ScholarLocate open access versionFindings
  • M. Wang and W. Deng. Deep face recognition: A survey. ArXiv, abs/1804.06655, 2018.
    Findings
  • C. Whitelam, E. Taborsky, A. Blanton, B. Maze, J. Adams, T. Miller, N. Kalka, A. K. Jain, J. A. Duncan, K. Allen, J. Cheney, and P. Grother. IARPA Janus benchmark-b face dataset. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 592–600, July 2017.
    Google ScholarLocate open access versionFindings
  • C. Xiong, X. Zhao, D. Tang, K. Jayashree, S. Yan, and T.-K. Kim. Conditional convolutional neural network for modality-aware face recognition. In Proceedings of the IEEE International Conference on Computer Vision, pages 3667– 3675, 2015.
    Google ScholarLocate open access versionFindings
  • J. Yim, H. Jung, B. Yoo, C. Choi, D.-S. Park, and J. Kim. Rotating your face using multi-task deep neural network. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 676–684, 2015.
    Google ScholarLocate open access versionFindings
  • X. Yin, X. Yu, K. Sohn, X. Liu, and M. Chandraker. Towards large-pose face frontalization in the wild. In Proceedings of the IEEE International Conference on Computer Vision, pages 3990–3999, 2017.
    Google ScholarLocate open access versionFindings
  • X. Yin, X. Yu, K. Sohn, X. Liu, and M. K. Chandraker. Towards large-pose face frontalization in the wild. 2017 IEEE International Conference on Computer Vision (ICCV), pages 4010–4019, 2017.
    Google ScholarLocate open access versionFindings
  • P. Zhang, X. Ben, W. Jiang, R. Yan, and Y. Zhang. Coupled marginal discriminant mappings for low-resolution face recognition. Optik, 126(23):4352–4357, 2015.
    Google ScholarLocate open access versionFindings
  • J. Zhao, Y. Cheng, Y. Xu, L. Xiong, J. Li, F. Zhao, K. Jayashree, S. Pranata, S. Shen, J. Xing, et al. Towards pose invariant face recognition in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2207–2216, 2018.
    Google ScholarLocate open access versionFindings
  • J. Zhao, Y. Cheng, Y. Xu, L. Xiong, J. Li, F. Zhao, K. Jayashree, S. Pranata, S. Shen, J. Xing, S. Yan, and J. Feng. Towards pose invariant face recognition in the wild. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2207–2216, June 2018.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments