MM-Net: Facial Expression Recognition Based on Multi-level and Multi-scale Attention Mechanisms

Facial expression recognition (FER) has gained significant attention due to its diverse applications. Achieving accurate facial expression recognition requires the consideration of both global central features and subtle local features. To this end, we propose the Multi-level and Multi-scale Network (MM-Net), an FER network that leverages both multi-level and multi-scale attention mechanisms. Specifically, we design a multi-hierarchical feature learning mechanism to facilitate the FER task with Multi-Level Attention Block (MLAB) and Multi-scale Attention Block (MSAB). The MLAB focuses on learning fine-grained features with adaptive attention across different blocks in the shallow network. Meanwhile, the MSAB facilitates the multi-scale fusion of deep features, enabling the network to capture richer semantic information and expression of features. In addition, we propose Limited Center Loss, which optimizes the network by minimizing the distance between the same classes while increasing the gap between different classes. Experimental results on public datasets show that our proposed MM-Net outperforms current state-of-the-art methods, achieving results of 90.42% on the RAF-DB dataset, 90.05% on FERPlus, 65.91% on AffectNet, and 57.52% on SFEW.