Facial Micro-expression Recognition Using Enhanced Temporal Feature-Wise Model

Automatic facial micro-expression recognition is challenging for the subtlety and transience in facial motion, and limited databases. Most researches focus on handcrafted techniques for facial micro-expression analysis on two-dimensional images. However, spatiotemporal facial feature representation is a critical issue for facial micro-expression recognition due to its short duration and subtle facial movement. To deeply extract the appearance characteristics and facial changes effectively from facial image sequences, a feature-wise deep learning model was proposed by applying temporal Convolutional Neural Network (3D-CNN) and Long Short-Term Memory (LSTM) to enhance temporal feature learning. There are two stages involved: (1) The CNN was extended to convolute along spatio and temporal simultaneously, to better represent the facial texture and motion. (2) The feature vector obtained by 3D-CNN was fed into LSTM for temporal enrichment. It was demonstrated that the proposed model achieved promising good performance on CASME II and SMIC databases on person-independent and cross-database experiments.