CNN based key frame extraction for face in video recognition

Nowadays we see an increasing demand for face in video recognition. However, in order to overcome the large variations in face quality in video streams, as well as for the purpose of improving the processing speed of face recognition system, frame selection becomes a necessary and essential step prior to performing face recognition. In this paper, we propose a convolutional neural network (CNN) based key-frame extraction (KFE) engine with Graphic Processing Unit (GPU) acceleration, which targets at extracting key-frames with high quality faces correctly and swiftly. We evaluated our method with ChokePoint dataset following NIST standards and compared against several representative key-frame selection approaches. The experimental results show that our CNN-based KFE engine can largely reduce the total processing time for face in video recognition, as well as improves the recognition accuracy of the face recognition back-end. With GPU acceleration, our KFE engine reaches and exceeds real-time processing speed requirement under HD resolution, making it capable of processing multiple video steams on the fly. On top ofthat, our proposed KFE engine is adaptive to different face recognition back-end.