Facial Emotion Recognition Based on Brain and Machine Collaborative Intelligence

Facial emotion is an important way for humans to convey the feeling and feed back to others. It is also a key component of human-computer interaction systems(HCISs). Naturally, facial emotion recognition(FER) has become a hot topic of current research. At present, the methods of FER typically rely on vision, using computer technology to extract visual features from face images. However, these features are derived from data-driven models, lacking the cognitive minds from the brain, so the recognition performance is not ideal in some cases. Factually, EEG features evoked by facial emotion images have high-level representations of emotion and good discrimination. For this, we propose a novel brain-machine collaborative method for FER. Firstly, EEG emotional features are extracted from the EEG signals collected when people observe emotion images. Secondly, the image visual features are extracted from the original facial emotion images. Thirdly, a regression model is used to find a mapping relationship between these two features in training stage. Finally, the EEG-like features predicted by pre-trained regression model are used in the test set to identify emotions. This method has been verified on CFAPS and found that the average recognition accuracy of the seven emotions is 88.28%, which is better than the simple image-based method.