FF-PPQA: Face frontalization without glasses based on perceptual quality and pixel-level quality assessment

Face frontalization is the process of synthesizing realistic and identity-preserving frontal views from face images in different poses and is an essential preprocessing step for face recognition. However, for side faces wearing glasses, the previous frontalization algorithms will distort the glasses after face reconstruction, affecting the image’s perceived quality and subsequent face recognition. Therefore, this paper first removes glasses, a factor that will cause distortion in face frontalization, and designs the perceptual and pixel-level face image quality assessment modules to improve the face frontalization performance. On the one hand, by constructing a saliency gradient, the pixel-level quality of face images is calculated and guides the network to generate frontal face images that are more conducive to face recognition. On the other hand, in order to obtain the perceptual quality for face image, the natural face images are used to construct a high-quality feature space, and the Bhattacharyya distance between it and the generated image is calculated to ensure the perceptual quality of the generated frontal image. Finally, the GAN network is used to generate a frontal face image that can consider both recognizability and perceptual quality. Quantitative and qualitative evaluations on controlled and in-the-wild databases show that our method outperforms the state-of-the-art.