Exploring Basic Expression Representation for Compound Facial Expression Recognition

The field of automatic Facial Expression Recognition (FER) has predominantly focused on classifying basic expressions. However, real-world facial expressions are often more complex, involving compound expressions that combine basic expressions within a single facial expression. This work aims to advance Compound Facial Expression Recognition (CFER), a more challenging task in FER. We identify two critical limitations in existing facial expression datasets: the scarcity of compound expression data and the presence of data ambiguity in basic expression datasets, where samples with compound expressions are labeled as basic expressions. To address these limitations, we propose a novel framework called Dual Basic data Enhancement Learning (DuBEL). DuBEL analyzes compound facial expressions and divides basic expression data into potential compound and purer basic expressions. By leveraging this distinction, the framework enables models to recognize compound facial expressions through their underlying basic expressions. DuBEL employs a dual enhancement training strategy for basic expression data, comprising two key schemes. The first is a pseudo compound expression training scheme, which relabels and utilizes potential compound expression samples to enrich the learning of compound expressions. The second is a basic expression enhancement training scheme, which strengthens basic expression representation learning by incorporating purer basic expression samples. Experimental results demonstrate that DuBEL effectively identifies compound expression samples within basic expression datasets and enhances the representation of basic expressions. Our method significantly improves CFER performance, achieving state-of-the-art results.