Multiview nonlinear discriminant structure learning for emotion recognition

Multiview subspace learning (MSL) has been widely used in various practical applications including emotion recognition. Despite the recent progress in MSL, two challenges remain to address. First, most existing MSL methods indiscriminately utilize both helpful and defective information contained in different views. Second, the most recent methods are linear approaches that do not perform well on emotion datasets with weak linear separability. Therefore, in this study, we introduce a framework for emotion recognition: multiview nonlinear discriminant structure learning (MNDSL). MNDSL fully exploits useful information in each input through local information preservation and discriminant reconstruction (LPDR) and obtains latent subspaces using multiview discriminant latent subspace learning (MDLSL). In addition, an out-of-sample extension was introduced to satisfy the requirements of large-scale applications and obtain the projections of new samples. The proposed framework constructs interviews and intra-view-weighted connections to explore discriminant structures and preserve locality under complementarity and correlation principles. The results demonstrate the superiority of the proposed framework compared with state-of-the-art methods.