emteqPRO: Face-mounted Mask for Emotion Recognition and Affective Computing

ABSTRACTThe paper presents the emteqPROtm system, which uses a face-mounted multi-sensor mask to measure the facial physiological responses, facial muscle activations, and motions from the user. These responses are then analyzed by machine learning algorithms to recognize and better understand the user's affective state and context, i.e., emotions, arousal, valence, stress response, activities, etc. The system can work by itself, as an open mask, or can be combined with a commercial Virtual Reality head mounted display. It comprises 3 sensor modalities: a 7-contact f-EMG sensor, a PPG sensor, and a 3-axis IMU, enabling it to measure the affective state of the user in real time. We will demonstrate how the system is used in practice in a Virtual Reality environment. This newly developed technology has the potential to significantly improve the way we collect data, design experiences, and interact within Virtual, Mixed and Augmented Realities.