Detecting Sequential Human Mental Workload Using U-Net With Continuity-Aware Loss Applied To Streamed Physiological Signals

Human mental workload perception is an important problem with multiple applications in the areas of industry, transportation, military and medical contexts. In this paper, we present a deep learning model developed for detecting sequential driver mental workload using physiological signals. We modify a fully convolutional U-Net in order to learn from multiple temporal physiological signals as inputs. We also demonstrate that the element-wise cross-entropy loss function, which has been adopted by many other deep learning architectures for classification or segmentation purposes, is not sufficient for the sequential workload segmentation because it treats each data point as an independent classification object while ignoring the neighboring relation between the points. We introduce a new loss function that is sensitive to local continuity property in workload sequence. The proposed method is evaluated on two sets of data collected by wearable sensors to record multiple physiological signals. The effectiveness of an introduced scalar used to control the influence of the continuity-aware loss is investigated. Our experiments show that the proposed U-Net is effective in detecting sequential human mental workload. Furthermore, when the proposed continuity-aware loss is combined with the element-wise binary cross-entropy loss with the properly selected controlling scalar value, the quality of the predicted mental workload segments is significantly improved.