DeepID3: Face Recognition with Very Deep Neural Networks

CoRR, 2015.

Cited by: 641|Bibtex|Views584
EI
Other Links: dblp.uni-trier.de|academic.microsoft.com|arxiv.org
Weibo:
This paper proposes two significantly deeper neural network architectures, coined DeepID3, for face recognition

Abstract:

The state-of-the-art of face recognition has been significantly advanced by the emergence of deep learning. Very deep neural networks recently achieved great success on general object recognition because of their superb learning capacity. This motivates us to investigate their effectiveness on face recognition. This paper proposes two v...More

Code:

Data:

0
Introduction
  • Using deep neural networks to learn effective feature representations has become popular in face recognition [12, 20, 17, 22, 14, 13, 18, 21, 19, 15].
  • By classifying training images into a large amount of identities, the last hidden layer of deep neural networks would form rich identity-related features
  • With this technique, deep learning got close to human performance for the first time on tightly cropped face images of the extensively evaluated LFW face verification dataset [6].
  • This motivates them to investigate whether the superb learning capacity brought by very deep net structures can benefit face recognition
Highlights
  • Using deep neural networks to learn effective feature representations has become popular in face recognition [12, 20, 17, 22, 14, 13, 18, 21, 19, 15]
  • The learned face representation could contain significant intrapersonal variations. Motivated by both [12] and [14], an approach of learning deep face representation by joint face identification-verification was proposed in DeepID2 [13] and was further improved in DeepID2+ [15]
  • We propose two deep neural network architectures, referred to as DeepID3, which are significantly deeper than the previous state-of-the-art DeepID2+ architecture for face recognition
  • This paper proposes two significantly deeper neural network architectures, coined DeepID3, for face recognition
  • The proposed DeepID3 networks achieve the state-of-the-art performance on both LFW face verification and identification tasks
  • We achieve a mean accuracy of 99.53% under this protocol
  • The effectiveness of very deep neural networks would be further investigated on larger scale training data in the future
Methods
  • DeepID3 net1 and net2 are used to extract features on either the original or the horizontally method

    High-dim LBP [4] TL Joint Bayesian [2] DeepFace [17] DeepID [14] GaussianFace [7, 8] DeepID2 [13, 11] DeepID2+ [15] DeepID3 accuracy (%)

    95.17 ± 1.13 96.33 ± 1.08 97.35 ± 0.25 97.45 ± 0.26 98.52 ± 0.66 99.15 ± 0.13 99.47 ± 0.12 99.53 ± 0.10 flipped face region but not both.
  • Feature extraction takes 50 times of forward propagation with half from DeepID3 net1 and the other half from net2.
  • These features are concatenated into a long feature vector of approximately 30, 000 dimensions.
  • With PCA, it is reduced to 300 dimensions on which a Joint Bayesian model is learned for face recognition.
  • Comparisons with previous works on mean accuracy and ROC curves are shown in Tab. 1 and Fig. 5, respectively
Results
  • An ensemble of the proposed two architectures achieves 99.53% LFW face verification accuracy and 96.0% LFW rank-1 face identification accuracy, respectively.
  • Being trained on the same dataset as DeepID2+, DeepID3 improves the face verification accuracy from 99.47% to 99.53% and rank-1 face identification accuracy from 95.0% to 96.0% on LFW, compared with DeepID2+.
  • DeepID3 net1 and DeepID3 net2 reduce the error rate by 0.81% and 0.26% compared to DeepID2+ net, respectively.
  • The authors achieve 96.0% closedset and 81.4% open-set face identification accuracies, respectively
Conclusion
  • There are three test face pairs which are labeled as the same person but are different people as announced on the LFW website.
  • DeepID2+ classified all the three wrongly labeled positive face pairs as different people
  • When these three wrong labels are corrected, the true face verification accuracy of DeepID2+ is 99.52% [15].
  • DeepID3, taking similar very deep architectures as VGG and GoogLeNet, does not improve over DeepID2+, with significantly shallower architecture, on the LFW face verification task.
  • The effectiveness of very deep neural networks would be further investigated on larger scale training data in the future
Summary
  • Introduction:

    Using deep neural networks to learn effective feature representations has become popular in face recognition [12, 20, 17, 22, 14, 13, 18, 21, 19, 15].
  • By classifying training images into a large amount of identities, the last hidden layer of deep neural networks would form rich identity-related features
  • With this technique, deep learning got close to human performance for the first time on tightly cropped face images of the extensively evaluated LFW face verification dataset [6].
  • This motivates them to investigate whether the superb learning capacity brought by very deep net structures can benefit face recognition
  • Methods:

    DeepID3 net1 and net2 are used to extract features on either the original or the horizontally method

    High-dim LBP [4] TL Joint Bayesian [2] DeepFace [17] DeepID [14] GaussianFace [7, 8] DeepID2 [13, 11] DeepID2+ [15] DeepID3 accuracy (%)

    95.17 ± 1.13 96.33 ± 1.08 97.35 ± 0.25 97.45 ± 0.26 98.52 ± 0.66 99.15 ± 0.13 99.47 ± 0.12 99.53 ± 0.10 flipped face region but not both.
  • Feature extraction takes 50 times of forward propagation with half from DeepID3 net1 and the other half from net2.
  • These features are concatenated into a long feature vector of approximately 30, 000 dimensions.
  • With PCA, it is reduced to 300 dimensions on which a Joint Bayesian model is learned for face recognition.
  • Comparisons with previous works on mean accuracy and ROC curves are shown in Tab. 1 and Fig. 5, respectively
  • Results:

    An ensemble of the proposed two architectures achieves 99.53% LFW face verification accuracy and 96.0% LFW rank-1 face identification accuracy, respectively.
  • Being trained on the same dataset as DeepID2+, DeepID3 improves the face verification accuracy from 99.47% to 99.53% and rank-1 face identification accuracy from 95.0% to 96.0% on LFW, compared with DeepID2+.
  • DeepID3 net1 and DeepID3 net2 reduce the error rate by 0.81% and 0.26% compared to DeepID2+ net, respectively.
  • The authors achieve 96.0% closedset and 81.4% open-set face identification accuracies, respectively
  • Conclusion:

    There are three test face pairs which are labeled as the same person but are different people as announced on the LFW website.
  • DeepID2+ classified all the three wrongly labeled positive face pairs as different people
  • When these three wrong labels are corrected, the true face verification accuracy of DeepID2+ is 99.52% [15].
  • DeepID3, taking similar very deep architectures as VGG and GoogLeNet, does not improve over DeepID2+, with significantly shallower architecture, on the LFW face verification task.
  • The effectiveness of very deep neural networks would be further investigated on larger scale training data in the future
Tables
  • Table1: Face verification on LFW
  • Table2: Closed- and open-set identification tasks on LFW
Download tables as Excel
Funding
  • An ensemble of the proposed two architectures achieves 99.53% LFW face verification accuracy and 96.0% LFW rank-1 face identification accuracy, respectively
  • Being trained on the same dataset as DeepID2+, DeepID3 improves the face verification accuracy from 99.47% to 99.53% and rank-1 face identification accuracy from 95.0% to 96.0% on LFW, compared with DeepID2+
  • On average, DeepID3 net1 and DeepID3 net2 reduce the error rate by 0.81% and 0.26% compared to DeepID2+ net, respectively
  • We achieve a mean accuracy of 99.53% under this protocol
  • We achieve 96.0% closedset and 81.4% open-set face identification accuracies, respectively
Study subjects and analysis
given face pairs: 6000
With PCA, it is reduced to 300 dimensions on which a Joint Bayesian model is learned for face recognition.

We evaluate DeepID3 networks under the LFW face verification [6] and LFW face identification [1, 18] protocols, respectively. For face verification, 6000 given face pairs are verified to tell if they are from the same person. We achieve a mean accuracy of 99.53% under this protocol

training samples: 300000
Then an additional Joint Bayesian model [3] is learned on these features for face verification or identification. All the DeepID3 networks and Joint Bayesian models are learned on the same approximately 300 thousand training samples as used in DeepID2+ [15], which is a combination of CelebFaces+ [14] and WDRef [3] datasets, and tested on LFW [6]. People in these two training data sets and the LFW test set are mutually exclusive

given face pairs: 6000
We evaluate DeepID3 networks under the LFW face verification [6] and LFW face identification [1, 18] protocols, respectively. For face verification, 6000 given face pairs are verified to tell if they are from the same person. We achieve a mean accuracy of 99.53% under this protocol

subjects with a single face image per subject: 4249
For face identification, we take one closed-set and one open-set identification protocols. For closed-set identification, the gallery set contains 4249 subjects with a single face image per subject, and the probe set contains 3143 face images from the same set of subjects in the gallery. For open-set identification, the gallery set contains 596 subjects with a single face image per subject, and the probe set contains 596 genuine probes and 9494 imposter ones

subjects with a single face image per subject: 596
For closed-set identification, the gallery set contains 4249 subjects with a single face image per subject, and the probe set contains 3143 face images from the same set of subjects in the gallery. For open-set identification, the gallery set contains 596 subjects with a single face image per subject, and the probe set contains 596 genuine probes and 9494 imposter ones. Table 2 compares Rank-1 identification accuracy of closedset identification and Rank-1 Detection and Identification rate (DIR) at a 1% False Alarm Rate (FAR) of open-set identification, respectively

test face pairs: 3
We achieve 96.0% closedset and 81.4% open-set face identification accuracies, respectively. There are three test face pairs which are labeled as the same person but are actually different people as announced on the LFW website. Among these three pairs, two are classified as the same person while the other one is classified as different people by our DeepID3 algorithm

pairs: 3
There are three test face pairs which are labeled as the same person but are actually different people as announced on the LFW website. Among these three pairs, two are classified as the same person while the other one is classified as different people by our DeepID3 algorithm. Therefore, when the label of these three face pairs are corrected, the actual face verification accuracy of DeepID3 is 99.52%

face pairs: 3
Among these three pairs, two are classified as the same person while the other one is classified as different people by our DeepID3 algorithm. Therefore, when the label of these three face pairs are corrected, the actual face verification accuracy of DeepID3 is 99.52%. For DeepID2+ [15], its face

face pairs: 3
There are nine common false positives and three common false negatives in total, around half of all wrongly classified face pairs by DeepID3. The three face pairs labeled as the same person but being classified as different people are shown in Fig. 6. The first pair of faces show great contrast of ages

face pairs: 9
The third one is an actress with significantly different makeups. Fig. 7 shows the nine face pairs labeled as different people while being classified as the same person by algorithms. Most of them look similar or have interference such as occlusions

Reference
  • L. Best-Rowden, H. Han, C. Otto, B. Klare, and A. K. Jain. Unconstrained face recognition: Identifying a person of interest from a media collection. TR MSU-CSE-14-1, 2014.
    Google ScholarFindings
  • X. Cao, D. Wipf, F. Wen, and G. Duan. A practical transfer learning algorithm for face verification. In Proc. ICCV, 2013.
    Google ScholarLocate open access versionFindings
  • D. Chen, X. Cao, L. Wang, F. Wen, and J. Sun. Bayesian face revisited: A joint formulation. In Proc. ECCV, 2012.
    Google ScholarLocate open access versionFindings
  • D. Chen, X. Cao, F. Wen, and J. Sun. Blessing of dimensionality: High-dimensional feature and its efficient compression for face verification. In Proc. CVPR, 2013.
    Google ScholarLocate open access versionFindings
  • G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. CoRR, abs/1207.0580, 2012.
    Findings
  • G. B. Huang, M. Ramesh, T. Berg, and E. LearnedMiller. Labeled Faces in the Wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst, 2007.
    Google ScholarFindings
  • C. Lu and X. Tang. Surpassing human-level face verification performance on LFW with GaussianFace. Technical report, arXiv:1404.3840, 2014.
    Findings
  • C. Lu and X. Tang. Surpassing human-level face verification performance on LFW with GaussianFace. In Proc. AAAI, 2015.
    Google ScholarLocate open access versionFindings
  • V. Nair and G. E. Hinton. Rectified linear units improve restricted Boltzmann machines. In Proc. ICML, 2010.
    Google ScholarLocate open access versionFindings
  • K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. Technical report, arXiv:1409.1556, 2014.
    Findings
  • Y. Sun, Y. Chen, X. Wang, and X. Tang. Deep learning face representation by joint identification-verification. In Proc. NIPS, 2014.
    Google ScholarLocate open access versionFindings
  • Y. Sun, X. Wang, and X. Tang. Hybrid deep learning for face verification. In Proc. ICCV, 2013.
    Google ScholarLocate open access versionFindings
  • Y. Sun, X. Wang, and X. Tang. Deep learning face representation by joint identification-verification. Technical report, arXiv:1406.4773, 2014.
    Findings
  • Y. Sun, X. Wang, and X. Tang. Deep learning face representation from predicting 10,000 classes. In Proc. CVPR, 2014.
    Google ScholarLocate open access versionFindings
  • Y. Sun, X. Wang, and X. Tang. Deeply learned face representations are sparse, selective, and robust. Technical report, arXiv:1412.1265, 2014.
    Findings
  • C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. Technical report, arXiv:1409.4842, 2014.
    Findings
  • Y. Taigman, M. Yang, M. Ranzato, and L. Wolf. DeepFace: Closing the gap to human-level performance in face verification. In Proc. CVPR, 2014.
    Google ScholarLocate open access versionFindings
  • Y. Taigman, M. Yang, M. Ranzato, and L. Wolf. Webscale training for face identification. Technical report, arXiv:1406.5266, 2014.
    Findings
  • D. Yi, Z. Lei, S. Liao, and S. Z. Li. Learning face representation from scratch. Technical report, arXiv:1411.7923, 2014.
    Findings
  • Z. Zhu, P. Luo, X. Wang, and X. Tang. Deep learning identity-preserving face space. In Proc. ICCV, 2013.
    Google ScholarLocate open access versionFindings
  • Z. Zhu, P. Luo, X. Wang, and X. Tang. Deep learning and disentangling face representation by multi-view perceptron. In Proc. NIPS, 2014.
    Google ScholarLocate open access versionFindings
  • Z. Zhu, P. Luo, X. Wang, and X. Tang. Recover canonicalview faces in the wild with deep neural networks. Technical report, arXiv:1404.3543, 2014.
    Findings
Full Text
Your rating :
0

 

Tags
Comments