Deep neural networks have become prevalent in human analysis, boosting the performance of applications, such as biometric recognition, action recognition, as well as person re-identification. However, the performance of such networks scales with the available training data. In human analysis, the demand for large-scale datasets poses a severe challenge, as data collection is tedious, time-expensive, costly and must comply with data protection laws. Current research investigates the generation of synthetic data as an efficient and privacy-ensuring alternative to collecting real data in the field. This survey introduces the basic definitions and methodologies, essential when generating and employing synthetic data for human analysis. We summarise current state-of-the-art methods and the main benefits of using synthetic data. We also provide an overview of publicly available synthetic datasets and generation models. Finally, we discuss limitations, as well as open research problems in this field. This survey is intended for researchers and practitioners in the field of human analysis.

Face morphing attacks pose a severe security threat to face recognition systems, enabling the morphed face image to be verified against multiple identities. To detect such manipulated images, the development of new face morphing methods becomes essential to increase the diversity of training datasets used for face morph detection. In this study, we present a representation-level face morphing approach, namely LADIMO, that performs morphing on two face recognition embeddings. Specifically, we train a Latent Diffusion Model to invert a biometric template - thus reconstructing the face image from an FRS latent representation. Our subsequent vulnerability analysis demonstrates the high morph attack potential in comparison to MIPGAN-II, an established GAN-based face morphing approach. Finally, we exploit the stochastic LADMIO model design in combination with our identity conditioning mechanism to create unlimited morphing attacks from a single face morph image pair. We show that each face morph variant has an individual attack success rate, enabling us to maximize the morph attack potential by applying a simple re-sampling strategy. We will publish our code and pre-trained models upon the acceptance of this paper.

Biometric Sample Quality Assessment (BSQA) estimates the usefulness of the captured image based on its utility for the recognition task. In this regard, the majority of studies have been proposed in the last decade for computing a sample quality score from facial images. In particular, methods that learn a regressor from pseudo-labels have obtained reliable results on various benchmarks. However, they fail to correctly estimate the quality of samples having both, low quality and low intra-class variability. This paper proposes a new BSQA approach, Perceptual Classifiability Ratio for Hand Image Quality Assessment (PCR-HIQA), which computes hand image quality by combining the relative classifiability of the sample with its fidelity-related properties. On the one hand, the classifiability ratio is calculated by mapping the feature representation of the training samples in the angular space with respect to its class centroid to the nearest negative class centroid. On the other hand, the fidelity properties encode the human perception of the input sample quality. Experimental results on the challenging HaGRID database, containing different hand gestures, underline the superiority of the proposed BSQA method which outperforms state-of-the-art techniques by up to 30%. 1

Evaluating the quality of facial images is essential for operating face recognition systems with sufficient accuracy. The recent advances in face quality standardisation (ISO/IEC CD3 29794-5) recommend the usage of component quality measures for breaking down face quality into its individual factors, hence providing valuable feedback for operators to re-capture low-quality images. In light of recent advances in 3D-aware generative adversarial networks, we propose a novel dataset, Syn-YawPitch, comprising 1000 identities with varying yaw-pitch angle combinations. Utilizing this dataset, we demonstrate that pitch angles beyond 30 degrees have a significant impact on the biometric performance of current face recognition systems. Furthermore, we propose a lightweight and explainable pose quality predictor that adheres to the draft international standard of ISO/IEC CD3 29794-5 and benchmark it against state-of-the-art face image quality assessment algorithms

Face recognition systems have numerous practical applications, including device authentication, border control, and attendance monitoring. These systems must achieve high recognition performance while processing large volumes of data. To ensure that low-quality images do not degrade biometric performance, it is crucial to quantify the quality of biometric samples. The draft international standard ISO/IEC 29794-5 introduces the concept of component quality, which measures the quality of individual quality elements. In this work, we focus on NeutrEx, a recently proposed component quality measure that quantifies the expression neutrality of facial images. We optimize NeutrEx to improve its efficiency in terms of parameters, storage space, and inference time. We investigate the applicability of optimization techniques, including pruning and knowledge distillation, to enhance the throughput rate of NeutrEx to tailor it for high frequency real-world applications. All code and pretrained experiments will be publicly available upon acceptance.

The increasing employment of large scale biometric systems such as the European “Entry-Exit System” and planned national initiatives such as the “Live Enrolment” procedure require quality assessment algorithms to ensure reliable recognition accuracy. Among other factors, facial image quality and hence face recognition accuracy can be negatively impacted by underexposure. Therefore, quality assessment algorithms analyse the exposure of live-captured facial images. To this end, mainly handcrafted measures have been proposed which are also referenced in current standards. However, this work shows that handcrafted measures, which use basic statistical approaches to analyse facial brightness patterns, exhibit racial bias. It is found that these algorithms disproportionately classify images of black people as underexposed as they do not take into account natural differences in skin color, particularly when relying on average pixel brightness values. To ensure fair biometric quality assessment, we have fine-tuned a data-efficient image transformer (DeiT) on synthetic data. The resulting underexposure estimation outperforms state-of-the-art algorithms in detection accuracy and biometric fairness. Precisely, an Equal Error Rate (EER) of approximately 7% is achieved. Our findings highlight the importance of developing robust and fair biometric classification methods to mitigate discrimination and ensure fair performance for all users, regardless of their skin color.

The recognition performance of biometric systems strongly depends on the quality of the compared biometric samples. Motivated by the goal of establishing a common understanding of face image quality and enabling system interoperability, the committee draft of ISO/IEC 29794-5 introduces expression neutrality as one of many component quality elements affecting recognition performance. In this study, we train classifiers to assess facial expression neutrality using seven datasets. We conduct extensive performance benchmarking to evaluate their classification and face recognition utility prediction abilities. Our experiments reveal significant differences in how each classifier distinguishes "neutral" from "non-neutral" expressions. While Random Forests and AdaBoost classifiers are most suitable for distinguishing neutral from non-neutral facial expressions with high accuracy, they underperform compared to Support Vector Machines in predicting face recognition utility.

The impact of long-term facial ageing on face recognition remains understudied due to a lack of openly available data. The age-related impermanence in facial identity affects many applications, including forensics and border control, limiting the accurate authentication of individuals in large-scale face image datasets. Recent advances in generative models have enabled facial ageing simulation with improved accuracy and identity preservation. In this work, we propose AgeDiff, a latent diffusion-based face age editing model using a Dual Cross-Attention conditioning mechanism to disentangle and control identity and age facial features. We conduct extensive performance evaluations and comparisons to existing methods, exploring how age simulations can assist with face image identification in forensic applications. We will publish our code and pre-trained models upon paper acceptance.

Accurate face recognition systems are increasingly important in sensitive applications like border control or migration management. Therefore, it becomes crucial to quantify the quality of facial images to ensure that low-quality images are not affecting recognition accuracy. In this context, the current draft of ISO/IEC 29794-5 introduces the concept of component quality to estimate how single factors of variation affect recognition outcomes. In this study, we propose a quality measure (NeutrEx) based on the accumulated distances of a 3D face reconstruction to a neutral expression anchor. Our evaluations demonstrate the superiority of our proposed method compared to baseline approaches obtained by training Support Vector Machines on face embeddings extracted from a pre-trained Convolutional Neural Network for facial expression classification. Furthermore, we highlight the explainable nature of our NeutrEx measures by computing per-vertex distances to unveil the most impactful face regions and allow operators to give actionable feedback to subjects.

Biometric data stored in automated recognition systems are at risk of attacks. This is particularly true for large-scale biometric identification systems, where the reference database is often accessed remotely. A popular approach for the protection of the stored templates is homomorphic encryption, which grants privacy protection while maintaining the biometric performance of the unprotected system. However, it introduces a significant computational overhead that can render identification transactions infeasible. To reduce this workload, biometric indexing in the encrypted domain has become a recent research interest. In this work, we show that in such schemes, auxiliary indexing data can leak additional privacy-sensitive information that violate standardized requirements for biometric template protection. In response to this leakage, we propose a novel framework HEBI that protects biometric indexing approaches at a post-quantum security level while requiring a computational effort of only 0.12 milliseconds per cluster.

Today, face recognition systems (FRS) are widely used in applications such as forensic and border control systems. Despite the increasing ability of deep neural networks to identify individuals based on their facial images, it remains challenging to recognize faces with long age gaps between reference and probe samples. To improve the robustness of FRS towards aging, training datasets can be enriched with synthetic data by simulating recurring aging effects. Typical aging signs include craniofacial changes during the child-to-adult age transition or textural changes that occur during adult-to-adult aging (e.g., wrinkles or furrows). Building upon the recent achievements of 3D-aware generative adversarial networks, we propose a geometry-aware face age modification algorithm (Age-EG3D) that enables lifespan face age simulation. We demonstrate the effectiveness of our approach by providing a comprehensive performance evaluation and comparison of Age-EG3D to prior works. All code and pre-trained models are available at https://github.com/johndoe133/eg3d-age.

Generative Adversarial Networks (GANs) have witnessed significant advances in recent years, generating increasingly higher quality images, which are non-distinguishable from real ones. Recent GANs have proven to encode features in a disentangled latent space, enabling precise control over various semantic attributes of the generated facial images such as pose, illumination, or gender. GAN inversion, which is projecting images into the latent space of a GAN, opens the door for the manipulation of facial semantics of real face images. This is useful for numerous applications such as evaluating the performance of face recognition systems. In this work, EGAIN, an architecture for constructing GAN inversion models, is presented. This architecture explicitly addresses some of the shortcomings in previous GAN inversion models. A specific model with the same name, egain, based on this architecture is also proposed, demonstrating superior reconstruction quality over state-of-the-art models, and illustrating the validity of the EGAIN architecture.

The vast progress in synthetic image synthesis enables the generation of facial images in high resolution and photorealism. In biometric applications, the main motivation for using synthetic data is to solve the shortage of publicly-available biometric data while reducing privacy risks when processing such sensitive information. These advantages are exploited in this work by simulating human face ageing with recent face age modification algorithms to generate mated samples, thereby studying the impact of ageing on the performance of an open-source biometric recognition system. Further, a real dataset is used to evaluate the effects of short-term ageing, comparing the biometric performance to the synthetic domain. The main findings indicate that short-term ageing in the range of 1-5 years has only minor effects on the general recognition performance. However, the correct verification of mated faces with long-term age differences beyond 20 years poses still a significant challenge and requires further investigation.

Enabling highly secure applications (such as border crossing) with face recognition requires extensive biometric performance tests through large scale data. However, using real face images raises concerns about privacy as the laws do not allow the images to be used for other purposes than originally intended. Using representative and subsets of face data can also lead to unwanted demographic biases and cause an imbalance in datasets. One possible solution to overcome these issues is to replace real face images with synthetically generated samples. While generating synthetic images has benefited from recent advancements in computer vision, generating multiple samples of the same synthetic identity resembling real-world variations is still unaddressed, i.e., mated samples. This work proposes a non-deterministic method for generating mated face images by exploiting the well-structured latent space of StyleGAN. Mated samples are generated by manipulating latent vectors, and more precisely, we exploit Principal Component Analysis (PCA) to define semantically meaningful directions in the latent space and control the similarity between the original and the mated samples using a pre-trained face recognition system. We create a new dataset of synthetic face images (SymFace) consisting of 77,034 samples including 25,919 synthetic IDs. Through our analysis using well-established face image quality metrics, we demonstrate the differences in the biometric quality of synthetic samples mimicking characteristics of real biometric data. The analysis and results thereof indicate the use of synthetic samples created using the proposed approach as a viable alternative to replacing real biometric data.

Face Age Progression (FAP) refers to synthesizing face images while simulating ageing effects, thus enabling predicting the future appearance of an individual.The generation of age-progressed face images brings benefits for various applications, ranging from face recognition systems to forensic investigations and digital entertainment.In particular, the recent success achieved with deep generative networks significantly leveraged the quality of age-synthesized face images in terms of visual fidelity, ageing accuracy and identity preservation.However, the high number of contributions in recent years requires systematically structuring new findings and ideas to identify a common taxonomy, accelerate future research and reduce redundancy.Therefore, we present a comparative analysis of recent deep learning based face age progression methods for both adult and child-based face ageing, broken down into three high-level concepts: translation-based, condition-based, and sequence-based FAP.Further, we offer a comprehensive summary of the most common performance evaluation techniques, cross-age datasets, and open challenges to steer future research in the right direction.

Face verification has come into increasing focus in various applications including the European Entry/Exit System, which integrates face recognition mechanisms. At the same time, the rapid advancement of biometric authentication requires extensive performance tests in order to inhibit the discriminatory treatment of travellers due to their demographic background. However, the use of face images collected as part of border controls is restricted by the European General Data Protection Law to be processed for no other reason than its original purpose. Therefore, this paper investigates the suitability of synthetic face images generated with StyleGAN and StyleGAN2 to compensate for the urgent lack of publicly available large-scale test data. Specifically, two deep learning-based (SER-FIQ, FaceQnet v1) and one standard-based (ISO/IEC TR 29794-5) face image quality assessment algorithm is utilized to compare the applicability of synthetic face images compared to real face images extracted from the FRGC dataset. Finally, based on the analysis of impostor score distributions and utility score distributions, our experiments reveal negligible differences between StyleGAN vs. StyleGAN2, and further also minor discrepancies compared to real face images.

In recent years, the popularity of fingerprint-based biometric authentication systems significantly increased. However, together with many advantages, biometric systems are still vulnerable to presentation attacks (PAs). In particular, this applies for unsupervised applications, where new attacks unknown to the system operator may occur. Therefore, presentation attack detection (PAD) methods are used to determine whether samples stem from a bona fide subject or from a presentation attack instrument (PAI). In this context, most works are dedicated to solve PAD as a two-class classification problem, which includes training a model on both bona fide and PA samples. In spite of the good detection rates reported, these methods still face difficulties detecting PAIs from unknown materials. To address this issue, we propose a new PAD technique based on autoencoders (AEs) trained only on bona fide samples (i.e., one-class), which are captured in the short wave infrared domain. On the experimental evaluation over a database of 19,711 bona fide and 4,339 PA images including 45 different PAI species, a detection equal error rate (D-EER) of 2.00% was achieved. Additionally, our best performing AE model is compared to further one-class classifiers (support vector machine, Gaussian mixture model). The results show the effectiveness of the AE model as it significantly outperforms the previously proposed methods.