A 3DMM-Based Framework for Deformation Measurement in Face Rehabilitation

Reconstructing a 3D model of the face from a single 2D image is a long studied problem, but it remains challenging especially when capturing local and asymmetric deformations of the face is important. Computing a measure of such local deformations can find application in monitoring rehabilitation exercises in patients that are recovering from a stroke or in patient with Parkinson's and Alzheimer's disease. In this study, we present a complete framework for accurately deforming a 3D Morphable Shape Model (3DMM) of the face to a target RGB image. The used 3DMM is based on localized components of deformation, while the 3D to 2D fitting transformation is guided by the correspondence between landmarks detected in the target image and landmarks manually annotated on the average 3DMM. The fitting has also the peculiarity of being performed in two steps, disentangling face deformations that are due to the identity of the target subject from those induced by facial actions. In the experimental validation of the method, we used the MICC-3D dataset that includes 11 subjects each acquired in one neutral pose plus 18 facial actions that deform the face in localized and asymmetric ways. For each acquisition, we fit the 3DMM to an RGB frame with an apex facial action and to the neutral frame, and computed the extent of the deformation. Results indicated that the proposed approach can accurately capture the face deformation even for localized and asymmetric ones. Interestingly, these results were obtained just using RGB targets without the need for 3D scans captured with costly devices. This opens the way to the use of the proposed tool for remote medical monitoring of rehabilitation.