To See Facial Expressions Through Occlusions via Adversarial Disentangled Features Learning with 3D Supervision

Facial expression recognition (FER) is still a challenging problem if face images are contaminated by occlusions, which lead to not only noisy features but also loss of discriminative features. To address the issue, this paper proposes a novel adversarial disentangled features learning (ADFL) method for recognizing expressions on occluded face images. Unlike previous methods, our method defines an explicit noise component in addition to the identity and expression components to isolate the occlusion-caused noise features. Besides, we learn shape features with joint supervision of 3D shape reconstruction and facial expression recognition to compensate for the occlusion-caused loss of features. Evaluation on both in-the-lab and in-the-wild face images demonstrates that our proposed method effectively improves FER accuracy for occluded images, and can even deal with noise beyond occlusions.