Learning-based techniques hold considerable promise in achieving human-like autonomous driving. However, one deployed policy encounters difficulties in satisfying the drivers’ diverse decision-making preferences simultaneously. Meanwhile, training personalized policies for each driver from scratch is time-consuming and resource-intensive. To address these challenges, this paper proposes a human-guided continual learning framework, wherein the human drivers could real-time take over a deployed policy when it performs unsatisfactorily, and the autonomous vehicle (AV) agent would automatically acquire human demonstrations and dynamically alter itself in accordance with personalized decision-making preference. Furthermore, a priority experience memory-enabled elastic weight consolidation (PEM-EWC) mechanism is developed to prevent the AV agent from overfitting to a limited number of human demonstrations and catastrophically forgetting its acquired fundamental driving abilities. Driver-in-the-loop simulations and real-world experiments are conducted in representative autonomous driving decision-making scenarios, and experimental results demonstrate the superior equilibrium of our proposed approach in terms of driving safety, human likeness, and training efficiency, compared to other baselines, which suggests that it provides a promising solution for personalized decision-making in autonomous driving. The supplementary video is available at https://youtu.be/HKF0ayxMycc.

Autonomous driving systems require a comprehensive understanding and accurate prediction of the surrounding environment to facilitate informed decision-making in complex scenarios. Recent advances in learning-based systems have highlighted the importance of integrating prediction and planning. However, this integration poses significant alignment challenges through consistency between prediction patterns, to interaction between future prediction and planning. To address these challenges, we introduce a Hybrid-Prediction integrated Planning (HPP) framework, which operates through three novel modules collaboratively. First, we introduce marginal-conditioned occupancy prediction to align joint occupancy with agent-specific motion forecasting. Our proposed MS-OccFormer module achieves spatial-temporal alignment with motion predictions across multiple granularities. Second, we propose a game-theoretic motion predictor, GTFormer, to model the interactive dynamics among agents based on their joint predictive awareness. Third, hybrid prediction patterns are concurrently integrated into the Ego Planner and optimized by prediction guidance. The HPP framework establishes state-of-the-art performance on the nuScenes dataset, demonstrating superior accuracy and safety in end-to-end configurations. Moreover, HPP's interactive open-loop and closed-loop planning performance are demonstrated on the Waymo Open Motion Dataset (WOMD) and CARLA benchmark, outperforming existing integrated pipelines by achieving enhanced consistency between prediction and planning.

The human-automation collaboration has garnered widespread attention due to the significant challenges remaining in achieving full automation in air traffic control (ATC) systems. Accurately identifying the cognitive workload of air traffic controllers (ATCOs) is critical to ensure a seamless transition between human operators and automated control. However, previous studies have predominantly focused on discrete workload classification, which proves challenging to implement in real-time scenarios where workload levels fluctuate continuously. To tackle this problem, this study proposes a semi-supervised learning approach to quantitatively estimate the workload of ATCOs in a continuous value. Specifically, a co-training approach that integrates ensemble learning has been developed to collaboratively extract representations from EEG signals and eye movement data, leveraging both labeled and unlabeled samples. A self-collected dataset based on ATC scenarios has been constructed to evaluate the effectiveness and accuracy of the proposed method. The results demonstrate that our model can accurately provide continuous workload estimations for ATCOs and outperforms the baseline models. The validity of these estimations is further confirmed by comparing them with the task difficulties outlined in the experimental design. The findings of this study can facilitate the development of adaptive automation within future human-AI collaborative ATC systems, thereby enhancing overall operational efficiency and air safety.

Accurate recognition of driver distraction is significant for the design of human-machine cooperation driving systems. Existing studies mainly focus on classifying varied distracted driving behaviors, which depend heavily on the scale and quality of datasets and only detect the discrete distraction categories. Therefore, most data-driven approaches have limited capability of recognizing unseen driving activities and cannot provide a reasonable solution for downstream applications. To address these challenges, this paper develops a vision Transformer-enabled weakly supervised contrastive (W-SupCon) learning framework, in which distracted behaviors are quantified by calculating their distances from the normal driving representation set. The Gaussian mixed model (GMM) is employed for the representation clustering, which centralizes the distribution of the normal driving representation set to better identify distracted behaviors. A novel driver behavior dataset and the other three ones are employed for the evaluation, experimental results demonstrate that our proposed approach has more accurate and robust performance than existing methods in the recognition of unknown driver activities. Furthermore, the rationality of distraction levels for different driving behaviors is evaluated through driver skeleton poses. The constructed dataset and demo videos are available at https://yanghh.io/Driver-Distraction-Quantification.

Reinforcement learning is capable of providing state-of-art performance in end-to-end robotic control tasks. Nevertheless, many real-world control tasks necessitate the balancing of multiple conflicting objectives while simultaneously ensuring that the learned policies adhere to constraints. Additionally, individual users may typically prefer to explore the personalized and diversified robotic control modes via specific preferences. Therefore, this paper presents a novel constrained multi-objective reinforcement learning algorithm for personalized end-to-end robotic control with continuous actions, allowing a trained single model to approximate the Pareto optimal policies for any user-specified preferences. The proposed approach is formulated as a constrained multi-objective Markov decision process, incorporating a nonlinear constraint design to facilitate the agent in learning optimal policies that align with specified user preferences across the entire preference space. Meanwhile, a comprehensive index based on hypervolume and entropy is presented to measure the convergence, diversity and evenness of the learned control policies. The proposed scheme is evaluated on nine multi-objective end-to-end robotic control tasks with continuous action space, and its effectiveness is demonstrated in comparison with the competitive baselines, including classical and state-of-the-art algorithms.

Misoperation when a driver is distracted, e.g., turning the steering wheel unconsciously, preventing the machine from turning the steering wheel, etc., can lead to a false take-over of the vehicle, which poses a danger to shared control. Therefore, it is essential to evaluate the feasibility of driver take-over in distraction status. This paper proposes a steering authority allocation strategy based on driver take-over feasibility to reduce the interference of distracted driver behavior on shared control. First, a driver take-over feasibility framework is established based on quantified driver distraction levels and take-over intention. Then the shared control system is established based on a linear parameter-varying vehicle model and non-cooperative game theory to suppress the interference of time-varying speed and parameter uncertainty on shared control. Subsequently, the steering authority allocation strategy is constructed in conjunction with driver take-over feasibility and environmental risk evaluation. The proposed sigmoid-based mapping function ensures fast and smooth authority allocation. Finally, a series of driver-in-the-loop experiments are conducted to validate the effectiveness of our proposed strategy across various driving scenarios, and the results demonstrate that our strategy has superior performance than other advanced shared control schemes in reducing steering burden, improving driving safety, and enhancing vehicle stability, etc.

Although deep reinforcement learning (DRL) methods are promising for making behavioral decisions in autonomous vehicles (AVs), their low training efficiency and difficulty to adapt to untrained cases hinder their applications. Introducing a human role in the DRL paradigm could improve training efficiency by using human prior knowledge and overcome untrained cases in deployment by online human takeover. In this study, a novel value-based DRL algorithm that leverages human guidance to improve its performance is proposed for addressing high-level decision-making problems in autonomous driving. We develop a new learning objective for DRL to increase the value of the human policy over the undertrained DRL policy so that the DRL agent can be encouraged to mimic human behaviors and thereby utilizing human guidance more efficiently. Our method can autonomously evaluate the importance of different human guidance, which makes it more robust for variation of human performance. The proposed DRL algorithm was used to address a challenging multiobjective lane-change decision-making problem. We collected human guidance from a human-in-the-loop driving experiment and evaluated our method in a high-fidelity simulator. Results validated the advantages of the proposed algorithm in terms of training efficiency and optimality in the decision-making problem compared to the baselines of state-of-the-art existing methods. Results also revealed the favorable fine-tuning ability of the proposed algorithm, which is promising for addressing the long-tail issue in DRL-based autonomous driving. Our methodology does not introduce additional domain knowledge so that it can be seamlessly applied to other similar issues. The supplementary video is available at https://youtu.be/Ec7WkqeLsB8.

Driver drowsiness detection is of great significance in improving driving safety and has been widely studied in recent years. However, some existing methods have not fully utilized the drowsiness-related information, and some methods are susceptible to interference from the redundant information of input data. To address these issues, a video-based driver drowsiness detection method according to the key facial features including facial landmarks and local facial areas (VBFLLFA) is proposed in this paper. In order to fully utilize the key facial features related to drowsiness and exclude the interference of redundant information, the head movement information is obtained through facial landmark analysis and the movement information of eyes and mouth is acquired from the local facial areas. And the spatial filtering based on the common spatial pattern (CSP) algorithm is introduced to improve the discrimination of different classes of samples. To adequately extract the temporal and spatial features, a two-branch multi-head attention (TB-MHA) module is designed in this paper. Furthermore, the center loss with center vector distance penalty is introduced to further improve the discrimination of different classes of samples in the feature space. In addition to two public datasets, we specifically create a novel video-based driver drowsiness detection (VBDDD) dataset to evaluate the effectiveness of our method. The experimental results verify that our method can achieve very excellent performance in driver drowsiness detection tasks.

Conditional Automated Driving (CAD) has attracted widespread attention due to the substantial gap in achieving fully autonomous driving, wherein an essential endeavor entails determining the transition timing between automated and manual driving modes. Driver cognitive workload serves as a crucial indicator for identifying transition timing, while its precise determination is challenging with discrete workload levels in previous studies. To address this issue, this work develops a dual-stage learning framework to quantify driver cognitive workload continuously. Specifically, a semi-supervised co-training strategy is first designed to approximate workload values, and then supervised contrastive learning is employed to align them with their feature representations in the latent space. A novel driver workload dataset is constructed for the evaluation, and experimental results demonstrate that our proposed approach outperforms other state-of-the-art baselines in estimation accuracy. Furthermore, the rationality of quantified cognitive workload is analyzed through the driver’ subjective assessment, indicating it is a more reliable solution for achieving the driving authority transition.

This paper investigates an improved adaptive sliding mode fault-tolerant control strategy for a magnetorheological semi-active suspension system with parametric uncertainties and actuator faults. Using the experimental data collected by a quarter-vehicle test rig, an adaptive-network-based fuzzy inference system is employed to establish a learning-based magnetorheological damper model firstly. The Takagi-Sugeno fuzzy approach is introduced to deal with the uncertainties of sprung mass and pitch rotary inertia and then the corresponding Takagi-Sugeno faulty semi-active suspension system is constructed. An adaptive sliding mode fault-tolerant controller is proposed, in which the magnetorheological damper fault gain is observed by the designed estimation law, and the asymptotical stability of the system is further analyzed. Finally, numerical simulation tests are conducted to demonstrate the effectiveness of the designed control scheme.

With the growing density of air passenger traffic, accurately recognizing the level of mental fatigue (MF) experienced by air traffic controllers (ATCOs) is crucial for developing intelligent ATCOs' mental state monitoring systems, which can achieve a more effective and safer human–machine cooperative pattern. However, the existing methods for recognizing ATCOs' MF face significant challenges due to pattern variations between ATCOs and sensor artifacts. This study introduces a framework for ATCOs' MF recognition, utilizing a deep neural network called RecMF, which incorporates multi-sensor information fusion to enhance the performance of MF detection. Specifically, the RecMF employs an attention-enabled CNN-LSTM architecture that simultaneously captures time-series feature representations of electroencephalogram (EEG) signals and eye movements. To validate the effectiveness of RecMF, a fatigue-inducing experiment is conducted involving 28 subjects who are tasked with performing a series of air traffic control (ATC) tasks. The model's performance is evaluated across various time horizons and typical cognitive tasks to gain insights into its capabilities. The evaluation results indicate that the proposed model outperforms other existing methods, thereby confirming its feasibility and effectiveness. Additionally, the effects of MF on ATCOs' cognitive performance are analyzed using analysis of variance (ANOVA). The results reveal that higher levels of MF significantly reduce ATCOs' reaction speed and operational accuracy.

Reinforcement learning (RL) is a promising approach in unmanned ground vehicles (UGVs) applications, but limited computing resource makes it challenging to deploy a well-behaved RL strategy with sophisticated neural networks. Meanwhile, the training of RL on navigation tasks is difficult, which requires a carefully-designed reward function and a large number of interactions, yet RL navigation can still fail due to many corner cases. This shows the limited intelligence of current RL methods, thereby prompting us to rethink combining RL with human intelligence. In this paper, a human-guided RL framework is proposed to improve RL performance both during learning in the simulator and deployment in the real world. The framework allows humans to intervene in RL's control progress and provide demonstrations as needed, thereby improving RL's capabilities. An innovative human-guided RL algorithm is proposed that utilizes a series of mechanisms to improve the effectiveness of human guidance, including human-guided learning objective, prioritized human experience replay, and human intervention-based reward shaping. Our RL method is trained in simulation and then transferred to the real world, and we develop a denoised representation for domain adaptation to mitigate the simulation-to-real gap. Our method is validated through simulations and real-world experiments to navigate UGVs in diverse and dynamic environments based only on tiny neural networks and image inputs. Our method performs better in goal-reaching and safety than existing learning- and model-based navigation approaches and is robust to changes in input features and ego kinetics. Furthermore, our method allows small-scale human demonstrations to be used to improve the trained RL agent and learn expected behaviors online.

Since emotions generally affect driver’s behavior, judgment, and reaction time, accurately identifying driver’s emotions is of great significance to improve the safety and comfort of intelligent driving system. However, the gender, skin color, age, and appearance of different drivers often have big differences, which will greatly interfere with the emotional recognition process. Besides, light intensity inside the vehicle varies with different time, weather, and location, which will also pose a challenge to driver emotion recognition. In this paper, a robust driver emotion recognition method based on feature separation is proposed to overcome the interference of individual differences and illumination changes. In order to realize the separation of expression-related features and irrelevant features, we design a high-purity feature separation (HPFS) framework based on partial feature exchange and the constraints of multiple loss functions. To verify that the proposed method can overcome the interference of illumination changes, we specifically create a multiple light intensities driver emotion recognition (MLI-DER) dataset and conduct a great deal of experiments on the dataset. In addition, to further demonstrate that our method can largely alleviate the interference of individual difference, some cross-subject emotion recognition experiments are conducted on two famous facial expression recognition datasets FACES and Oulu-CASIA and the experimental results are compared with that of some state-of-the-art methods.

Reinforcement learning holds the promise of allowing autonomous vehicles to learn complex decision making behaviors through interacting with other traffic participants. However, many real-world driving tasks involve unpredictable perception errors or measurement noises which may mislead an autonomous vehicle into making unsafe decisions, even cause catastrophic failures. In light of these risks, to ensure safety under perception uncertainty, autonomous vehicles are required to be able to cope with the worst case observation perturbations. Therefore, this paper proposes a novel observation adversarial reinforcement learning approach for robust lane change decision making of autonomous vehicles. A constrained observation-robust Markov decision process is presented to model lane change decision making behaviors of autonomous vehicles under policy constraints and observation uncertainties. Meanwhile, a black-box attack technique based on Bayesian optimization is implemented to approximate the optimal adversarial observation perturbations efficiently. Furthermore, a constrained observation-robust actor-critic algorithm is advanced to optimize autonomous driving lane change policies while keeping the variations of the policies attacked by the optimal adversarial observation perturbations within bounds. Finally, the robust lane change decision making approach is evaluated in three stochastic mixed traffic flows based on different densities. The results demonstrate that the proposed method can not only enhance the performance of an autonomous vehicle but also improve the robustness of lane change policies against adversarial observation perturbations.

Driver workload inference is significant for the design of intelligent human-machine cooperative driving schemes since it allows the systems to alert drivers before potentially dangerous maneuvers and achieve a safer control transition. However, pattern variations among individual drivers and sensor artifacts pose great challenges to the existing cognitive workload recognition approaches. In this article, we develop an attention-enabled recognition network with a decision-level fusion architecture to further improve the workload estimation performance. Specifically, the cross-attention mechanism can enhance useful feature representations learned by hyper long-short-term-memory-based modules from time-series multimodal information, i.e., electroencephalogram signals, eye movements, and vehicle states. A novel dataset containing multiple driving scenarios is constructed to evaluate the model performance across different historical horizons and decision thresholds, and test results demonstrate the superior performance of the proposed model to other existing methods. Furthermore, robustness tests and driver-in-the-loop experiments are conducted to verify the effectiveness of the developed model in real-time workload levels inference. The code and supplementary materials are available at https://yanghh.io/Driver-Workload-Recognition.

Reinforcement learning has demonstrated its potential in a series of challenging domains. However, many real-world decision making tasks involve unpredictable environmental changes or unavoidable perception errors that are often enough to mislead an agent into making suboptimal decisions and even cause catastrophic failures. In light of these potential risks, reinforcement learning with application in safety-critical autonomous driving domain remains tricky without ensuring robustness against environmental uncertainties (e.g., road adhesion changes or measurement noises). Therefore, this paper proposes a novel constrained adversarial reinforcement learning approach for robust decision making of autonomous vehicles at highway on-ramps. Environmental disturbance is modelled as an adversarial agent that can learn an optimal adversarial policy to thwart the autonomous driving agent. Meanwhile, observation perturbation is approximated to maximize the variation of the perturbed policy through a white-box adversarial attack technique. Furthermore, a constrained adversarial actor-critic algorithm is presented to optimize an on-ramp merging policy while keeping the variations of the attacked driving policy and action-value function within bounds. Finally, the proposed robust highway on-ramp merging decision making method of autonomous vehicles is evaluated in three stochastic mixed traffic flows with different densities, and its effectiveness is demonstrated in comparison with the competitive baselines.

This paper proposes a novel adaptive hierarchical control approach for Steer-by-Wire (SbW) vehicles to improve the handling stability. The high-level stability control scheme contains a variable steering ratio (VSR) strategy based on the adaptive-network-based fuzzy inference system (ANFIS) and an active front steering (AFS) controller designed with the integral sliding mode method by tracking the expected yaw rate, in which the desired front wheel angle is generated to enhance the cornering stability performance. Besides, an adaptive tracking controller (ATC) for the SbW system is designed by using the adaptive sliding mode control method to achieve desired steering performance in the lower level. The proposed adaptive control strategy is validated with different driving circles from ISO standards in simulation tests and hardware-in-the-loop (HiL) experiments. The results demonstrate that the designed control approach improve the vehicle handling stability significantly, even in some extreme driving conditions.

Accurate recognition of driver braking intensity is of great importance for intelligent braking system. In this paper, the braking intensity is classified into four clusters based on an unsupervised Gaussian mixture model (GMM). Then, the architecture of an adaptive-network-based fuzzy inference system (ANFIS) is proposed for braking intensity prediction. A batch learning rule that combines the recursive least squares and gradient descent method used for training ANFIS is adopted to improve the generalization capability. The training data are collected from a hybrid vehicle under real driving conditions. In addition, co-simulation with the software of MATLAB/Simulink and Hardware-in-the-Loop (HiL) tests for an Electronic-Hydraulic Brake (EHB) system are carried out. In comparison to other typical learning methods, the simulation and experimental results demonstrate the effectiveness and accuracy of the proposed hybrid learning approach for braking intensity recognition in different braking scenarios.

The purpose of this study is to propose a control scheme for an automobile Steer-By-Wire (SBW) system. A mathematical model of SBW system is built. A co-simulation platform in software of MATLAB/Simulink and a Hardware-in-the Loop (HIL) system is developed. By using ARX algorithm, the parameters of the SBW system are identified. Then, an Internal Model Controller (IMC) for the SBW system is designed to track desired motion states of controlled vehicle. In typical driving conditions, simulations are carried out to examine the effectiveness of designed controller. Compared with a PID controller, the results demonstrate that designed IMC controller is able to provide better control performance. To reduce the steering efforts of the driver, meanwhile ensure the stability of the vehicle in different driving conditions, a variable gearing ratio control strategy is proposed, and also its characteristics are examined in different cases. Besides, a Slide Mode Controller (SMC) for tracking desired yaw rate is designed to realize the active steering. Co-simulation results show that desired yaw rate can be tracked satisfactorily by using the designed controllers, with enhanced handling and stability performance.