Personalized Driver Workload Estimation Using Deep Neural Network Learning From Physiological and Vehicle Signals

Drivers often engage in secondary in-vehicle activities that can be functional and/or to relieve monotony. Drivers believe they can safely do so when their perceived workload is low. However, driving requires concurrent execution of cognitive, physical, perceptual and motor tasks. Over allocation of a driver's attention to secondary tasks may impair the driver's control of the vehicle and attention to the surrounding traffic. Accurate assessment of a driver's workload is an important, but challenging, research topic with many applications in intelligent vehicle systems related to driving safety and enhance driving experience. In this article, we present our research on driver workload detection based on driver's physiological, vehicle signals as well as traffic contexts such as congestion level and traffic events. We obtained a collection of data from real driving scenarios. Twenty participants were recruited, whose workload was scaled to “low” as a base level, and “elevated” as a higher level. We developed two convolutional neural networks for multivariate temporal series (MTS-CNN). Extensive experiments were conducted in two scopes: within driver and cross-driver. The within-driver scope concentrates on experiments using data from a single driver, while in the cross-driver scope, transfer learning is leveraged and discussed. The experimental results demonstrate that one of the proposed models, i.e., MTS-CNN2, which combines features captured by the convolutional layers at all levels, is capable of learning well from the combined temporal physiological and vehicle signals and obtains the best performance.