Autonomous systems are becoming pervasive, and as they become applied to highly dynamic and heterogeneous environments there is a need to model and understand more complex and nuanced agent interactions than have been previously studied. This paper proposes an agent-based modelling approach, based on norm emergence, to investigate such interactions. While there is typically an ideal set of compatible actions which lead to an optimal norm, in complex environments there may also be combinations that are compatible and yield positive (but not optimal) rewards. We illustrate our model of such scenarios using the case of an autonomous vehicle performing a manoeuvre at a T-intersection.

This paper proposes a discrete time distributed state feedback controller design strategy for a homogenous vehicle platoon system with undirected network topology which is resilient to both external disturbances and random consecutive network packet drop. The system incorporates a distributed state feedback controller design by satisfying bounded $H_{\infty }$ norm using Lyapunov-Krasovskii based linear matrix inequality (LMI) approach that ensures internal stability and performance. The effect of packet drops on internal stability in terms of stability margin are studied for a homogenous vehicle platoon system with undirected network topology and external disturbance. The variation of stability margin, representing absolute value of least stable close-loop pole, is also studied for two common undirected network topologies for vehicle platooning, i.e., bidirectional predecessor following (BPF) and bidirectional predecessor leader following (BPLF) topologies by varying platoon members, packet drop rates with number of contiguous packets dropped. Results demonstrate that the control strategy best satisfies the requirement of maintaining a desired inter-vehicular distance with constant spacing policy and leader trajectory using two network topologies: BPF and BPLF. We show how these topologies are robust in terms of ensuring internal stability and performance to maintain cooperative motion of vehicle platoon system with different number of followers, random multiple consecutive packet drops and external disturbance.

Behaviour prediction function of an autonomous vehicle predicts the future states of the nearby vehicles based on the current and past observations of the surrounding environment. This helps enhance their awareness of the imminent hazards. However, conventional behavior prediction solutions are applicable in simple driving scenarios that require short prediction horizons. Most recently, deep learning-based approaches have become popular due to their promising performance in more complex environments compared to the conventional approaches. Motivated by this increased popularity, we provide a comprehensive review of the state-of-the-art of deep learning-based approaches for vehicle behavior prediction in this article. We firstly give an overview of the generic problem of vehicle behavior prediction and discuss its challenges, followed by classification and review of the most recent deep learning-based solutions based on three criteria: input representation, output type, and prediction method. The article also discusses the performance of several well-known solutions, identifies the research gaps in the literature and outlines potential new research directions.

Supervised feature selection aims to find the signals that best predict a target variable. Typical approaches use measures of correlation or similarity, as seen in filter methods, or predictive power in learned models, as seen in wrapper methods. In both approaches, the selected features often have high entropies and are not suitable for compression. This is a particular drawback in the automotive domain where fast communication and archival of vehicle telemetry data is increasingly important, especially with technologies such as V2V and V2X (vehicle-to-vehicle and vehicle-to-everything communication). This paper aims to select features with good predictive performances and good compression by introducing a compressibility factor into several existing feature selection approaches. Where appropriate, performance guarantees are provided for greedy searches based on monotonicity and submodularity. Using the language of entropy, the relationship between relevance, redundancy, and compressibility is discussed from the perspective of signal selection. The approaches are then demonstrated in selecting features from a vehicle Controller Area Network for use in SVMs in a regression task, namely predicting fuel consumption, and a classification task, namely identifying Points of Interest. We show that while predictive performance is slightly lower when compression is considered, the compressibility of the selected features is significantly improved.

Destination prediction is an active area of research, especially in the context of intelligent transportation systems. Intelligent applications, such as battery management in electric vehicles and congestion avoidance, rely on the accurate prediction of the future destinations of a vehicle. Destination prediction methods can utilise mobility patterns and can harness the latent information within vehicle trajectories. Existing approaches make use of the spatial information contained within trajectories, but this can be insufficient to achieve an accurate prediction at the start of an unfolding trajectory, since several destinations may share a common start to their trajectories. To reduce the prediction error in the early stages of a journey, we propose the Destination Prediction by Trajectory Subclustering (DPTS) method for iteratively clustering similar trajectories into groups using additional information contained within trajectories, such as temporal data. We show in our evaluation that DPTS is able to reduce the mean distance error in the first 40–60% of journeys. The implication of reducing the distance error early in a journey is that location-aware applications could provide more accurate functionality earlier in a journey. In this article, we (i) propose the Destination Prediction by Trajectory Subclustering (DPTS) method by extending an existing destination prediction method through incorporating an iterative clustering stage to decompose groups of similar trajectories into smaller groups and (ii) evaluate DPTS against the baseline performance of the existing method.

Autonomous agents are becoming increasingly ubiquitous and are playing an increasing role in wide range of safety-critical systems, such as driverless cars, exploration robots and unmanned aerial vehicles. These agents operate in highly dynamic and heterogeneous environments, resulting in complex behaviour and interactions. Therefore, the need arises to model and understand more complex and nuanced agent interactions than have previously been studied. In this paper, we propose a novel agent-based modelling approach to investigating norm emergence, in which such interactions can be investigated. To this end, while there may be an ideal set of optimally compatible actions there are also combinations that have positive rewards and are also compatible. Our approach provides a step towards identifying the conditions under which globally compatible norms are likely to emerge in the context of complex rewards. Our model is illustrated using the motivating example of self-driving cars, and we present the scenario of an autonomous vehicle performing a left-turn at a T-intersection.

Recent advances in smart connected vehicles and Intelligent Transportation Systems (ITS) are based upon the capture and processing of large amounts of sensor data. Modern vehicles contain many internal sensors to monitor a wide range of mechanical and electrical systems and the move to semi-autonomous vehicles adds outward looking sensors such as cameras, lidar, and radar. ITS is starting to connect existing sensors such as road cameras, traffic density sensors, traffic speed sensors, emergency vehicle, and public transport transponders. This disparate range of data is then processed to produce a fused situation awareness of the road network and used to provide real-time management, with much of the decision making automated. Road networks have quiet periods followed by peak traffic periods and cloud computing can provide a good solution for dealing with peaks by providing offloading of processing and scaling-up as required, but in some situations latency to traditional cloud data centres is too high or bandwidth is too constrained. Cloud computing at the edge of the network, close to the vehicle and ITS sensor, can provide a solution for latency and bandwidth constraints but the high mobility of vehicles and heterogeneity of infrastructure still needs to be addressed. This paper surveys the literature for cloud computing use with ITS and connected vehicles and provides taxonomies for that plus their use cases. We finish by identifying where further research is needed in order to enable vehicles and ITS to use edge cloud computing in a fully managed and automated way. We surveyed 496 papers covering a seven-year timespan with the first paper appearing in 2013 and ending at the conclusion of 2019.

The state-of-the-art decision and planning approaches for autonomous vehicles have moved away from manually designed systems, instead focusing on the utilisation of large-scale datasets of expert demonstration via Imitation Learning (IL). In this paper, we present a comprehensive review of IL approaches, primarily for the paradigm of end-to-end based systems in autonomous vehicles. We classify the literature into three distinct categories: 1) Behavioural Cloning (BC), 2) Direct Policy Learning (DPL) and 3) Inverse Reinforcement Learning (IRL). For each of these categories, the current state-of-the-art literature is comprehensively reviewed and summarised, with future directions of research identified to facilitate the development of imitation learning based systems for end-to-end autonomous vehicles. Due to the data-intensive nature of deep learning techniques, currently available datasets and simulators for end-to-end autonomous driving are also reviewed.

Previous research into perceptions of autonomous vehicles has largely focused on a priori attitudes, with little work on the perception of specific traffic situations, context and driving styles. The present study used three simulator experiments (total N = 150) to examine the combined effects of vehicle speed, lane position, information presentation and traffic context on occupants' levels of satisfaction with autonomous highway journeys. Overall, occupants preferred being in a vehicle that was mostly overtaking compared to being overtaken, regardless of whether the overtaking vehicles were exceeding the speed limit. This finding remained even when occupants were given additional reminders that they themselves were travelling at an appropriate speed (Experiments 1 & 2). Experiment 3 found that occupants preferred overtaking to being overtaken when following another car, but this preference disappeared when they were following a lorry, suggesting that occupants' sensitivity to position amongst the traffic was partially context dependent. Overall, the findings suggest that journey satisfaction is sensitive to overtaking contexts and the inappropriate behaviour of other drivers (e.g., speeding) can reduce journey satisfaction for occupants in autonomous vehicles that drive within the speed limit, depending on the specific traffic situation. Potential implications for the integration of autonomous vehicles with other traffic and the need for in-vehicle presentation of information are discussed.

Autonomous vehicle interface design represents a challenge combining the hard constraints and risks of a safety-critical domain, with the softer personal preference and market-driven competitive constraints that consumer product designs face. This challenge of the design of a consumer product interface for the extremely diverse potential users, in the context of a safety-critical scenario, brought together human factors methods for complex sociotechnical systems along with those developed for consumer inclusive design. The goal of this approach was to combine the rigour of methods validated in safety-critical contexts, with inclusive, creative, and empathic design methods used in the consumer product design and development. Inclusive design accepts that capabilities in the population are not evenly distributed, and advocates that those with less capability – whether due to age-related changes, disability, or low technological interface experience – should not be unnecessarily discriminated against in the process of designing products and services, nor in the design outcome. The user-centred ecological interface design (UCEID) process was developed to structure the design process for the human–machine interface (HMI), where the three phases explored the user's needs, created design solutions, and evaluated the extent to which their needs would be met. Activities were carried out iteratively within the Explore, Create, and Evaluate categories to identify the pertinent needs, to develop potential solutions to those needs, and to assess how well each potential solution met the needs. This chapter reports on the process outcomes of the stages of the design phases of the HMI to show the effect and influences of this approach on the design ideas and concepts to include and support users through the takeover process.

In this article, an enhanced model reference adaptive control (EMRAC) algorithm is used to design a generic lateral-tracking controller for a vehicle. This EMRAC is different from the EMRAC in the literature as it adopts a σ-modification approach to bind the adaptive gain of the switching action. Moreover, an extended Lyapunov theory for discontinuous systems is used to analytically prove the ultimate boundedness of the closed-loop control system when the adaptive gain of the switching action is bounded with a σ-modification strategy. The control algorithm is applied to a vehicle path-tracking problem and its tracking performance is investigated under conditions of: 1) external disturbances such as crosswind; 2) road surface changes; 3) modeling errors; and 4) parameter missmatches in a co-simulation environment based on IPG Carmaker/MATLAB. The simulation studies show that the controller is effective at tracking a given reference path for performing different autonomous highway driving maneuvers while ensuring the boundedness of all closed-loop signals even when the system is subjected to the conditions mentioned above.

Predicting what drivers will do as vehicle control is handed over to them from automation is a relatively new challenge for the motor vehicle industry. Operator Event Sequence Diagrams (OESDs) offer a way of modeling the interactions between the driver and vehicle automation in the handover of control. In this paper, two studies are presented in which a range of handover strategies are tested. The anticipated driver strategies were modeled using OESDs to serve as predictions of driver behavior. Drivers were then observed in two separate studies: (1) using a Lower-Fidelity (vehicle seat and controls) simulator and (2) using a Higher-Fidelity (whole vehicle) simulator. Driver behavior during a takeover task was categorized according to the signal detection paradigm into hits, misses, false alarms, and correct rejections. The results showed that for all strategies in both sets of studies, the median criterion for validity was exceeded (phi > 0.8), suggesting that OESDs made good predictions of driver behavior during the handover of the vehicle from automation to manual control.

Knowledge of drivers’ mobility patterns is useful for enabling context-aware intelligent vehicle functionality, such as route suggestions, cabin preconditioning, and power management for electric vehicles. Such patterns are often described in terms of the Points of Interest (PoIs) visited by an individual. However, existing PoI extraction methods are general purpose and typically rely on detecting periods of low mobility, meaning that when they are applied to vehicle data, they often extract a large number of false PoIs (for example, incorrectly extracting PoIs due to stopping in traffic), reducing their usefulness. To reduce the number of false PoIs that are extracted, we propose using features derived from vehicle signals, such as the selected gear and status of doors, to classify candidate PoIs and filter out those that are irrelevant. In this paper, we (i) present Activity-based Vehicle PoI Extraction (AVPE), a wrapper method around existing PoI extraction methods, that utilizes a postclustering classification stage to filter out false PoIs, (ii) evaluate the benefits of AVPE compared to three state-of-the-art general purpose PoI extraction algorithms, and (iii) demonstrate the effectiveness of AVPE when applied to real-world driving data.

Drawing together the strengths of ecological interface design (EID) and inclusive user-centred design (UCD), UCEID is a novel human factors (HF) method for complex sociotechnical systems. This method ensures user needs are appropriately represented in the constraints-based models provided by the cognitive work analysis (CWA) framework. The range of methodological approaches adopted, the advantages, disadvantages, tools, and training times of UCEID are described. Examples of how to apply this method to the domain of automated driving to produce design concepts are provided, focusing on interactions between drivers and semi-autonomous vehicles for a planned SAE Level 3 vehicle-to-driver takeover.

Semi-autonomous cars are already on the road, and highly autonomous cars will soon be with us. Little is understood about how drivers will adapt to the changing relationship with their vehicle, but to ensure safety and consumer acceptance, this insight is vital. To this end, an on-road study in a semi-autonomous vehicle was undertaken with six UK drivers. The 'think-aloud' technique was employed, and video and audio footage of their interaction with the vehicle was captured. Neisser's (1976) Perceptual Cycle Model (PCM) was used to analyse the data, and three case studies are presented to highlight how poor synergy between driver and semi-autonomous vehicles can occur from the lens of schema, action, or world information. Seven key design considerations are proposed to ensure a more positive and safer interaction between driver and autonomous vehicle to guide focus by manufacturers. Further evidence for the existence of a 'counter cycle' (Plant and Stanton, 2015) within the PCM is found, and how this relates to the challenges of using verbal protocols expressed during a fast-moving dynamic task is discussed.

It is of great importance to monitor the driver's status to achieve an intelligent and safe take-over transition in the level 3 automated driving vehicle. We present a camera-based system to recognise the non-driving activities (NDAs) which may lead to different cognitive capabilities for take-over based on a fusion of spatial and temporal information. The region of interest (ROI) is automatically selected based on the extracted masks of the driver and the object/device interacting with. Then, the RGB image of the ROI (the spatial stream) and its associated current and historical optical flow frames (the temporal stream) are fed into a two-stream convolutional neural network (CNN) for the classification of NDAs. Such an approach is able to identify not only the object/device but also the interaction mode between the object and the driver, which enables a refined NDA classification. In this paper, we evaluated the performance of classifying 10 NDAs with two types of devices (tablet and phone) and 5 types of tasks (emailing, reading, watching videos, web-browsing and gaming) for 10 participants. Results show that the proposed system improves the averaged classification accuracy from 61.0% when using a single spatial stream to 90.5%.

Vehicles offering autonomous features need effective methods for transferring the control from the driver to the vehicle and back. While most research focuses on presenting information the driver might need after retaking control, our study investigates ways to improve the process of transferring control itself. We investigated multimodal feedback with and without haptics and visuals in a simulator study. Results showed that visual and haptic feedback improved driving during handover. Subjective ratings described multimodal feedback without visual as more disruptive than with the visual feedback included. Furthermore, ratings showed a preference for including visual and haptic feedback. These results lead us a step closer to a safe, clear and accepted control transfer process between driver and vehicle.

Quantifying and encoding occupants' preferences as an objective function for the tactical decision making of autonomous vehicles is a challenging task. This paper presents a low-complexity approach for lane-change initiation and planning to facilitate highly automated driving on freeways. Conditions under which human drivers find different manoeuvres desirable are learned from naturalistic driving data, eliminating the need for an engineered objective function and incorporation of expert knowledge in form of rules. Motion planning is formulated as a finite-horizon optimisation problem with safety constraints. It is shown that the decision model can replicate human drivers' discretionary lane-change decisions with up to 92% accuracy. Further proof of concept simulation of an overtaking manoeuvre is shown, whereby the actions of the simulated vehicle are logged while the dynamic environment evolves as per ground truth data recordings.

Haptic feedback can improve safety and driving behaviour. While vibration has been widely studied, other haptic modalities have been neglected. To address this, we present two studies investigating the use of uni- and bimodal vibrotactile and thermal cues on the steering wheel. First, notifications with three levels of urgency were subjectively rated and then identified during simulated driving. Bimodal feedback showed an increased identification time over unimodal vibrotactile cues. Thermal feedback was consistently rated less urgent, showing its suitability for less time critical notifications, where vibration would be unnecessarily attention-grabbing. The second study investigated more complex thermal and bimodal haptic notifications comprised of two different types of information (Nature and Importance of incoming message). Results showed that both modalities could be identified with high recognition rates of up to 92% for both and up to 99% for a single type, opening up a novel design space for haptic in-car feedback.