Decoding Emotion Dimensions Arousal and Valence Elicited on EEG Responses to Videos and Images: A Comparative Evaluation.

This study aims to compare the automatic classification of emotions based on the self-reported level of arousal and valence with the Self-Assessment Manikin (SAM) when subjects were exposed to videos or images. The classification is performed on electroencephalographic (EEG) signals from the DEAP public dataset, and a dataset collected at the University of Tsukuba, Japan. The experiments were defined to classify low versus high arousal/valence using a Convolutional Neural Network (CNN). The obtained results show a higher performance when the subjects were exposed to videos, i.e., using DEAP dataset we obtained an area under the receiver operating characteristic (AUROC) of 0.844 ± 0.008 and 0.836 ± 0.009 to classify low versus high arousal/valence, respectively. In contrast, when subjects were stimulated with images, the obtained performance was 0.621 ± 0.007 for both, arousal and valence classification. The obtained difference was confirmed by testing the experiments using a method based on the Discrete Wavelet Transform (DWT) for feature extraction and classification using random forest. Using image-based stimulation may help to better understand low and high arousal/valence when analyzing event-related potentials (ERP), however, according to the obtained results, for classification purposes, the performance is higher using video-based stimulation.