A weakly supervised method for makeup-invariant face verification.

Face verification, which aims to determine whether two face images belong to the same identity, is an important task in multimedia area. Face verification becomes more challenging when the person is wearing makeup. However, collecting sufficient makeup and non-makeup image pairs are tedious, which brings great challenges for deep learning methods of face verification. In this paper, we propose a new weakly supervised method for face verification. Our method takes advantages of the plentiful video resources available from the Internet. Our face verification model is pre-trained on the free videos and fine-tuned on small makeup and non-makeup datasets. To fully exploit the video contexts and the limited makeup and non-makeup datasets, many techniques are used to improve the performance. A novel loss function with a triplet term and two pairwise terms is defined, and multiple facial parts are combined by the proposed voting strategy to generate better verification results. Experiments on a benchmark dataset (Guo et al., 2014) [1] and a newly collected face dataset show the priority of the proposed method. HighlightsPropose a weakly supervised method for face verification robust to cosmetic changes.Free video contexts are used to pre-train the proposed deep learning framework.Many techniques are used in the network to prevent overfitting.A large scale video face dataset and a beforeafter makeup dataset are collected.Our method achieves state-of-the-art performance on a benchmark dataset.