Sensor and Feature Selection for Lightweight Emotion Recognition on Resource-Constrained Smartwatches

Emotion recognition (ER) systems are generally based on cumbersome sensors specifically suited for laboratory settings, which limits their portability. To mitigate this shortcoming, efforts are being made to achieve lightweight ER systems to make them available on person’s daily-life devices. Such devices are required to have the capabilities to collect measurements and to handle machine learning (ML) methodologies. In this regard, smartwatches represent a viable solution for gathering such measurements in a simpler and unobtrusive manner. However, smartwatches, and specifically low-cost ones, are generally equipped with few sensors and, due to their limited computational and memory resources, they can only execute lightweight ML models (e.g., described by a low number of features). In this work, we aim at designing a lightweight ML-based ER systems for resource-constrained devices such as smartwatches. We consider an ER problem of detecting person’s happiness or sadness state. We model the problem as a binary classification problem and train lightweight ML models such as, e.g., Random Forest and Extreme Gradient Boosting, to tackle it. Considering public data sets, we investigate the impact of sensor selection and feature selection on classification accuracy and computational and memory requirements. Experimental results show that sub-sets of sensors can be enough to guarantee same performance as when considering all sensors. Similarly, results show that feature selection can significantly reduce number of features to be used by ER systems while maintaining performance levels. In terms of memory requirements, selecting subset of sensors and features permits up to 46% reduction.