Deception Detection Algorithm Based on Global and Local Feature Fusion with Multi-head Attention

Deception is a prevalent human behavior that significantly impacts our perception of essential facts. Therefore, developing accurate deception detection technology holds great significance. However, current research on pure visual deception detection algorithms does not leverage deep learning methods to extract detailed features such as facial Action Units (AUs) and Gaze Angles. Additionally, the global information within facial video sequences is often overlooked. To address these limitations, this paper introduces a novel deception detection model that combines global and local facial features through attention mechanisms. Firstly, the model focuses on the local features of the face, computing AU Strength and Gaze Angle for each frame to create a multivariate time series for every video. Subsequently, the Siamese Transformer model, employing Patching, extracts deep temporal and channel features from the multivariate time series. Additionally, the occurrence frequency of five specific AUs is selected as a manual feature. Secondly, the model conducts video understanding based on the global features of the face. Local features are extracted from each frame using Shallow CNNs with multiple sensitivity fields. Then, a Video Transformer model with spatiotemporal separation attention is applied to globally model the sequence of face frames. Finally, the extracted local and global facial features are concatenated and fed into a classifier to determine deception. Extensive experiments on existing datasets validate the outstanding performance of the proposed method.