“Edutainment 2017” a visual and semantic representation of 3D face model for reshaping face in images

Reshaping faces is an interesting thing and a common demand in today’s media industry and cosmetic industry. Especially, the reshaped face images contribute a lot to special effects industry and plastic-beauty industry. In this paper, we provide a novel parametric representation of 3D face model that describes the face shape as a linear combination of semantic and non-semantic bases. The semantic (local) bases correspond to each individual face part and they are used to semantically edit each face part, while the non-semantic bases explain other shape variations to span the subspace of the face shape. First, we build a sparse and spatially localized parametric face model from a dataset of 3D face models by sparse principal component analysis. Second, to define the semantic bases, we train a regression model to correlate semantically significant values like nose height and mouth width. Finally, the bases of the resulting parametric face model is orthogonalized to all defined semantic bases by the Gram–Schmidt algorithm for generating the novel parametric representation of 3D face model. The novel representation can be applied in reshaping face in the images. The experiment results demonstrate that our representation not only retains the accuracy for 3D face reconstruction but also provides users a user-friendly tool to edit facial parts for desired facial shapes. Graphical abstract