•Describes a conceptual framework for 3D-2D (or 2D-3D) face recognition.•Proposes a novel 3D-2D system for 2D image face recognition from 3D datasets.•Proposes a method to build subject-specific 3D gallery models, using 3D+2D data, and a method for model-based, texture representation and relighting.•3D-2D recognition surpasses 2D-2D on challenging 2D+3D data with pose and illumination variations, and can approximate 3D-3D, shape-based similarity methods.•Representation and normalization using 3D models can compensate for non-frontal poses or different lighting conditions.

Asymmetric 3D to 2D face recognition has gained attention from the research community since the real-world application of 3D to 3D recognition is limited by the unavailability of inexpensive 3D data acquisition equipment. A 3D to 2D face recognition system explicitly relies on 3D facial data to account for uncontrolled image conditions related to head pose or illumination. We build upon such a system, which matches relit gallery textures with pose-normalized probe images, using the gallery facial meshes. The relighting process, however, is based on an assumption of indoor lighting conditions and limits recognition performance on outdoor images. In this paper, we propose a novel method for minimizing illumination difference by unlighting a 3D face texture via albedo estimation using lighting maps. The algorithm is evaluated on challenging databases (UHDB30, UHDB11, FRGC v2.0) with drastic lighting and pose variations. The experimental results demonstrate the robustness of our method for estimating the albedo from both indoor and outdoor captured images, and the effectiveness and efficiency for illumination normalization in face recognition.

Performing face recognition under extreme poses and lighting conditions remains a challenging task for current state-of-the-art biometric algorithms. The recognition task is even more challenging when there is insufficient training data available in the gallery, or when the gallery dataset originates from one side of the face while the probe dataset originates from the other. The authors present a new method for computing the distance between two biometric signatures acquired under such challenging conditions. This method improves upon an existing Semi-Coupled Dictionary Learning method by computing a jointly-optimized solution that incorporates the reconstruction cost, the discrimination cost, and the semi-coupling cost. The use of a semi-coupling term allows the method to handle partial 3D face meshes where, for example, only the left side of the face is available for gallery and the right side of the face is available for probe. The method also extends to 2D signatures under varying poses and lighting changes by using 3D signatures as a coupling term. The experiments show that this method can improve recognition performance of existing state-of-the-art wavelet signatures used in 3D face recognition and provide excellent recognition results in the 3D-2D face recognition application.

3D face recognition for partial data is a very challenging task. The task is even more challenging when the gallery sample originates from one side of the face while the probe sample originates from the other. We present a new method for computing the similarity of partial 3D data for the purpose of face recognition. This method improves upon an existing Semi-Coupled Dictionary Learning method by computing a jointly-optimized solution that incorporates the reconstruction cost, the discrimination cost and the semi-coupling cost. Our experiments demonstrate that this method can improve the recognition performance of existing state-of-the-art wavelet signatures used for 3D face recognition.

Our face is our password—face recognition promises to revolutionize the way we identify individuals in a nonintrusive and convenient manner. Even though research in face recognition has spanned over nearly three decades, only 2D systems, with limited adoption to practical applications, have been developed so far. The primary reason behind this is the low accuracy of 2D face recognition systems in the presence of: (i) pose variations between the gallery and probe datasets, (ii) variations in lighting, and (iii) variations in the presence of expressions and/or accessories. The above conditions generally arise when noncooperative subjects are involved, which is the very case that demands accurate recognition. Face recognition using 3D images was introduced in order to overcome these challenges. It was partly made possible by significant advances in 3D scanner technology. However, even 3D face recognition has faced significant challenges which have hindered its adoption for practical applications. The main problem of 3D face recognition is the high cost and fragility of 3D scanners. Over the last seven years, our research team has focused on exploring the usefulness of 3D data and the development of models for face recognition (under the general name URxD). In this chapter, we present advances that aid in overcoming the challenges encountered in 3D face recognition. First, we present a fully automatic 3D face recognition system, UR3D, which has been proven to be robust under variations in expressions. The fundamental idea of this system is the description of facial data using an Annotated Face Model (AFM). The AFM is fitted to the facial scan using

Patrick J. Flynn, University of Notre Dame, U.S.A. Program Committee Mohamed Abdel-Mottaleb, University of Miami, U.S.A. George Bebis, University of Nevada – Reno, U.S.A. Olga Bellon, Universidade Federal do Parana, Brazil Ross Beveridge, Colorado State University, U.S.A. Vijyaykumar Bhagavatula, Carnegie-Mellon University, U.S.A. Bir Bhanu, University of California – Riverside, U.S.A. Wageeh Boles, Queensland University of Technology, Australia Julien Bringer, Morpho, France Mark Burge, MITRE, U.S.A. Patrizio Campisi, Universita degli Studi Roma TRE, Italy Christophe Champod, Universite de Lausanne, Switzerland Rama Chellappa, University of Maryland, U.S.A. Michal Choras, University of Technology and Life Sciences, Poland John Daugman, Cambridge University, U.K. Bernadette Dorizzi, Télécom and Management SudParis, France Sonia Garcia-Salicetti Télécom and Management SudParis, France Venu Govindaraju, University of Buffalo, U.S.A. Patrick Grother, NIST, U.S.A. Anil Jain, Michigan State University, U.S.A. Xudong Jiang, Nanyang Technological University, Singapore Ioannis Kakadiaris, University of Houston, U.S.A. Josef Kittler, University of Surrey, U.K. Mark Koch, Sandia National Labs, U.S.A. Krzysztof Kryszczuk, IBM Zurich Research Laboratory, Switzerland Ajay Kumar, The Hong Kong Polytechnic University, Hong Kong Rick Lazarick, CSC Identity Labs, U.S.A. Stan Li, Chinese Academy of Sciences, China Chengjun Liu, New Jersey Institute of Technology, U.S.A. Brian Martin, L1 ID, U.S.A. Aleix Martinez, The Ohio State University, U.S.A. Christopher Miles, Department of Homeland Security, U.S.A. Karthik Nandakumar, Institute for Infocomm Research, Singapore Elaine Newton, NIST, U.S.A. Larry O’Gorman, Alcatel-Lucent Bell Labs, U.S.A. Jonathon Phillips, National Institute of Standards and Technology, U.S.A. Norman Poh, University of Surrey, U.K. Salil Prabhakar, Digital Persona, U.S.A. Karl Ricanek, University of North Carolina, Wilmington, U.S.A. Fabio Roli, University of Cagliari, Italy Trina Russ, Digital Signal Corporation, U.S.A. Marios Savvides, Carnegie Mellon University, U.S.A. Natalia Schmid, West Virginia University, U.S.A. Michael Schuckers, St. Lawrence University, U.S.A. Stephanie Schuckers, Clarkson University, U.S.A. Kuntal Sengupta, MERL, U.S.A. Nicole Spaun, Federal Bureau of Investigation, U.S.A. Alex Stoianov, Office of the Information and Privacy Commissioner, Canada Andrew Teoh, Yonsei University, Korea Tieniu Tan, NLPR, China Massimo Tistarelli, University of Sassari, Italy Raymond Veldhuis, University Twente, Netherlands Fred Wheeler, GE Global Research, U.S.A. Damon Woodard, Clemson University, U.S.A. Jian Yang, Nanjing University of Science and Technology, China Pong Yuen, Hong Kong Baptist University, Hong Kong Yong Zhang, Youngstown State University, U.S.A. Jie Zhou, Tsinghua University, China Program Co-chairs

Performing face recognition under extreme poses and lighting conditions remains a challenging task for current state-of-the-art biometric algorithms. The recognition task is even more challenging when there is insufficient training data available in the gallery, or when the gallery dataset originates from one side of the face while the probe dataset originates from the other. The authors present a new method for computing the distance between two biometric signatures acquired under such challenging conditions. This method improves upon an existing Semi-Coupled Dictionary Learning method by computing a jointly-optimized solution that incorporates the reconstruction cost, the discrimination cost, and the semi-coupling cost. The use of a semi-coupling term allows the method to handle partial 3D face meshes where, for example, only the left side of the face is available for gallery and the right side of the face is available for probe. The method also extends to 2D signatures under varying poses and lighting changes by using 3D signatures as a coupling term. The experiments show that this method can improve recognition performance of existing state-of-the-art wavelet signatures used in 3D face recognition and provide excellent recognition results in the 3D-2D face recognition application.