A Multi-Task Cascaded Network for Prediction of Affect, Personality, Mood and Social Context Using EEG Signals

This paper presents a multi-task cascaded deep neural network which jointly predicts people's affective levels (valence and arousal) and personal factors using EEG signals recorded in response to presentation of affective multimedia content. Studied personal factors are the Big-five personality traits, mood (Positive Affect and Negative Affect Schedules) and social context (individual vs group). The cascaded network consists of two levels of prediction. The first level consists of a hybrid network designed to predict affective levels for individual video segments by combining the capabilities of CNN and RNN units. The second level consists of an RNN unit designed to predict personal factors based on the sequence of affective levels predictions of consecutive video segments. The first level reduces the dimensionality of the input while keeping affective information. The fusion of decisions of RNN and CNN networks produces an increase in performance of 3% and 4% f1-score respectively for valence and arousal recognition. And, our results for personal factors recognition, on average, outperform baseline studies by at least 2:7% mean f1-score.