A novel decision-making architecture inspired by the role of mirror neurons is evaluated in two user studies. First, the role and efficacy of the negotiation layer of the architecture is assessed. Then, in a “Wizard of Oz” experiment, the performance of the complete architecture is compared with the one of a human decision-maker. The same task of using wooden blocks to create characters in a kind of mechanical model of a 7-segment display is used in both experiments (task details in Sect. 2.4). The same task is used to capture data and train policy modules, an integration module, and a negotiation layer to be combined into decision-making models of involved agents, which build upon our previous work. The evaluation results show a significant improvement in terms of the chosen objective and subjective measures when the robot uses the complete architecture with the negotiation layer. No significant difference was found for any of the measures when comparing the human decision-maker and the complete model. Although the robot with a human decision-maker scored descriptively slightly better in all measures, a further Bayesian comparison of the data suggests a high probability of similarity between the model and the human decision-maker. This was further illustrated by a qualitative analysis of the post-experiment interview questions; in answering the third question, where 17 participants identified that the robot using the complete model was like working with a human, and an equal number opted for identifying the robot controlled by a human decision-maker. In addition, answering the first question, 6 participants found no difference between the robot being controlled by a human decision-maker and being controlled using the complete model.

This Paper proposes a novel Transformer-based end-to-end autonomous driving model named Detrive. This model solves the problem that the past end-to-end models cannot detect the position and size of traffic participants. Detrive uses an end-to-end transformer based detection model as its perception module; a multi-layer perceptron as its feature fusion network; a recurrent neural network with gate recurrent unit for path planning; and two controllers for the vehicle's forward speed and turning angle. The model is trained with an on-line imitation learning method. In order to obtain a better training set, a reinforcement learning agent that can directly obtain a ground truth bird's-eye view map from the Carla simulator as a perceptual output, is used as teacher for the imitation learning. The trained model is tested on the Carla's autonomous driving benchmark. The results show that the Transformer detector based end-to-end model has obvious advantages in dynamic obstacle avoidance compared with the traditional classifier based end-to-end model.

Inspired by the role of mirror neurons and the importance of predictions in joint action, a novel decision-making structure is proposed, designed and tested for both individual and dyadic action. The structure comprises models representing individual decision policies, policy integration layer(s), and a negotiation layer. The latter is introduced to prevent and resolve conflicts among individuals through internal simulation rather than via explicit agent-agent communication. As the main modelling tool, Dynamic Neural Fields (DNFs) were chosen. Data was captured from human-human experiments with a decision-making task performed by either one or two participants. The task involves choosing and picking blocks one by one from seven wooden blocks to create an alpha/numeric character on a 7-segment. The task is designed to be as generic as possible. Recorded hand and blocks movements were used for developing DNF-based models by optimising parameters using a genetic algorithm. Results show that decision policies can be modelled and integrated with acceptable accuracy for individual performances. In the dyadic experiment, using only individual models without the negotiation layer, the model failed to resolve conflicts. However, with the implementation of a negotiation layer, this problem could be overcome. The proposed decision-making structure based on DNFs is developed and tested for a simple pick-and-place task. However, the main primitive underlying action of this task, pick-and-place, is indeed part of many more complex tasks people perform in their day-to-day life. Paired with the possibility to gradually evolve the architecture by adding new policies on demand, the architecture provides a general framework for modelling decision-making in joint action tasks.

Automated Vehicles (AVs) are rapidly maturing in the transportation domain. However, the complexity of the AV design problem is such that no single technique is sufficient to provide adequate validation of key properties such as safety, reliability or trustworthiness. In this vision paper, a combination of a spatial traffic logic and agent-based verification methods with a validation method that uses assertion checking of simulations is proposed. We sketch how to integrate the respective approaches within a methodological framework called Corroborative Verification and Validation (V&V).The Corroborative V&V framework identifies three different verification and validation levels for AVs (formal verification, simulation-based testing, real-world experiments) and specifies connections and evidence between these levels. We define specifications for the formal relationships that must be established between processes, system models and requirements models for the evidence from formal design verification and simulation-based testing to corroborate each other and enhance assurance confidence from verification and validation.

The usability of a novel triple-arm mixed-reality robot teleoperation system is investigated. The system is developed to provide a sense of remote presence for the operator. Different types of interfaces and camera setups have been proposed previously. Our novel approach is to have a moving stereo vision camera mounted on a robotic arm in the remote scene controlled with a virtual reality (VR) headset. By streaming live stereo video into the VR headset in a video see-through configuration the operator experiences a sense of remote presence. The teleoperation task is done using two more robotic arms. These arms are set up in a mirror teleoperation setting so that the remote (follower) arm copies the movements of the control (leader) arm. To investigate the effect of latency on the operator a within-subject usability study of the system with 20 participants has been conducted. Participants completed a pick-and-place task sorting objects into marked containers in two conditions. In one condition, the camera robot arm was controlled by a joint position controller with low latency but jittery robot motion. In the other condition, the camera robot was controlled by a joint velocity controller with higher latency but smooth motion. Participants completed the System Usability Scale questionnaire after each trial. The task completion time and participants' head movement were also recorded as objective measures. The study result did not show a significant difference in any of the objective or subjective measures, although, the position controller scored higher overall. This could be due to the number of participants or the ability of people to adapt to the latency in the system and further analysis in future work is required.

Game engines are increasingly used as simulation platforms by the autonomous vehicle community to develop vehicle control systems and test environments. A key requirement for simulation-based development and verification is determinism, since a deterministic process will always produce the same output given the same initial conditions and event history. Thus, in a deterministic simulation environment, tests are rendered repeatable and yield simulation results that are trustworthy and straightforward to debug. However, game engines are seldom deterministic. This paper reviews and identifies the potential causes and effects of non-deterministic behaviours in game engines. A case study using CARLA, an open-source autonomous driving simulation environment powered by Unreal Engine, is presented to highlight its inherent shortcomings in providing sufficient precision in experimental results. Different configurations and utilisations of the software and hardware are explored to determine an operational domain where the simulation precision is sufficiently high i.e. variance between repeated executions becomes negligible for development and testing work. Finally, a method of a general nature is proposed, that can be used to find the domains of permissible variance in game engine simulations for any given system configuration.

Compliant elements overcome many of the shortfalls of using 3D printing to create mechanisms, as print artefacts such as ‘stair stepping’ can cause issues with conventional joints. One of the key decisions when designing a compliant mechanism is choosing to either concentrate the compliance into a small region that resembles a conventional hinge, or distribute it over a larger area. This research details the simulated deformation and stress difference between these two types of compliant elements for a 3D printed gripper. Results show that for the same gripper deformation, the distributed compliant element experiences much less stress, at the expense of stiffness in secondary loading directions.

Safety and mission performance validation of autonomous vehicles (AVs) is a major challenge. In this paper we describe a methodology for constructing and applying assertion checks to validate the behaviour of an AV operating either in simulation or in the real world. We have identified a taxonomy of assertion types and the general format of their specification, and we have developed procedures for translating driving codes of practice to yield formal logical expressions that can be monitored automatically by computer, either by direct translation or by physical modelling. We have developed examples of assertions derived from the UK Highway Code (UKHC), as an example of a code of practice. We illustrate the approach with an example of assertion checking for vehicle overtaking, using a geospatial information system in an SQL database for validation and performance assessment. We present initial simulation and runtime monitoring experiments that apply assertions relevant in this overtaking scenario together with an analysis of the safety and mission performance characteristics measured.

Autonomous Vehicles (AVs) research has attracted much attention in recent years. In fact, some functionality of AV, such as cruise control, lane keeping and automatic parking have already been employed in commercial vehicles. In a fully AV system, the vehicle drives like a human being and might even outperform humans in some aspects, e.g., shorter reaction time when facing dangerous situations. To have AVs drive like human beings, discovering how to transfer human driving skills to smart vehicles has become a key issue. In this study, we have developed a human-to-vehicle skills transfer system that is able to control the smart vehicle to drive safely in the longitudinal direction under a virtual environment constructed within the Carla simulator. A deep Q-learning algorithm has been employed to build the learning network, which will obtain its control policy during the interaction between the agent and the environment. To achieve human-to-vehicle skills transfer, human data collected with the simulator is employed as the learning target for the learning network. Various experimental results show that longitudinal human driving skills can successfully transfer to a smart vehicle through the developed learning network.

Automated individual part extraction from powder based 3D printers has the potential to save time and cost compared with fully manual part extraction, or part sorting following automated bulk separation of parts and unfused powder. This work details the development of a novel one-shot 3D printed compliant gripping mechanism, able to extract individual solid parts from unfused PA12 powder. It was found that the unfused powder causes grip slip and instability as well as an increase in the perceived object width. A new toothed digit geometry was created, able to reduce the duration of the instability, and localised vibration was shown to eliminate the initial period of increased strain. A combination of toothed geometry and localised vibration showed a relative strain output on the gripper almost identical to that of the same grasp with no powder present. This allows individual solid parts to be extracted from unfused powder in a known pose, ready for automation of subsequent post-processing steps.

Simulation-based verification is beneficial for assessing otherwise dangerous or costly on-road testing of autonomous vehicles (AV). This paper addresses the challenge of efficiently generating effective tests for simulation-based AV verification using software testing agents. The multi-agent system (MAS) programming paradigm offers rational agency, causality and strategic planning between multiple agents. We exploit these aspects for test generation, focusing in particular on the generation of tests that trigger the precondition of an assertion. On the example of a key assertion we show that, by encoding a variety of different behaviours respondent to the agent's perceptions of the test environment, the agency-directed approach generates twice as many effective tests than pseudo-random test generation, while being both efficient and robust. Moreover, agents can be encoded to behave naturally without compromising the effectiveness of test generation. Our results suggest that generating tests using agency-directed testing significantly improves upon random and simultaneously provides more realistic driving scenarios.

We present an approach for the verification and validation (V&V) of robot assistants in the context of human–robot interactions, to demonstrate their trustworthiness through corroborative evidence of their safety and functional correctness. Key challenges include the complex and unpredictable nature of the real world in which assistant and service robots operate, the limitations on available V&V techniques when used individually, and the consequent lack of confidence in the V&V results. Our approach, called corroborative V&V, addresses these challenges by combining several different V&V techniques; in this paper we use formal verification (model checking), simulation-based testing, and user validation in experiments with a real robot. This combination of approaches allows V&V of the human–robot interaction task at different levels of modeling detail and thoroughness of exploration, thus overcoming the individual limitations of each technique. We demonstrate our approach through a handover task, the most critical part of a complex cooperative manufacturing scenario, for which we propose safety and liveness requirements to verify and validate. Should the resulting V&V evidence present discrepancies, an iterative process between the different V&V techniques takes place until corroboration between the V&V techniques is gained from refining and improving the assets (i.e., system and requirement models) to represent the human–robot interaction task in a more truthful manner. Therefore, corroborative V&V affords a systematic approach to “meta-V&V,” in which different V&V techniques can be used to corroborate and check one another, increasing the level of certainty in the results of V&V.

In this paper we present the findings of a usability study for a monitoring robotic unit tele-operated via a virtual fixtures (VF) based control framework. The study aims at investigating the impact of VF on the robot navigation as well as the impact of multimodal feedback on the user performance in a static inspection task. The findings will help in the design of the monitoring control framework to inspect a robotised welding process, as it has been researched in previous work. The study has been conducted with untrained participants, involved in four different test scenarios. The experiments treated a static case in which users were asked to navigate the monitoring robot in the workspace to find a lit LED of a test-piece. The statistical analysis of the experiment metrics showed a positive impact of the VF control on the navigation of the monitoring robot even for users with no previous experience. Moreover, from the analysis of the task load index forms (TLX) it emerged that the combination of VF control and additional multimodal feedback improved the user performance without negatively impacting the effort required to accomplish the task.

As autonomous robots move towards ubiquity, the need for robots to make decisions under risk that are trustworthy becomes increasingly significant; both to aid acceptance and to fully utilise their autonomous capabilities. We propose that incorporating a human approach to risk assessment into a robot's decision making process will increase user trust. This work investigates four robotic approaches to risk and, through a user study, explores the levels of trust placed in each. These approaches are: risk averse, risk seeking, risk neutral and a human approach to risk. Risk is artificially stimulated through performance-based compensation, in line with previous studies. The study was conducted in a virtual nuclear environment created using the Unity games engine. Forty participants were asked to complete a robot supervision task, in which they observed a robot making risk based decisions and were able to question the robot, question the robot further and ultimately accept or alter the robot's decision. It is shown that a robot that is risk seeking is significantly less trusted than a risk averse robot, a risk neutral robot and a robot utilising human approach to risk. There was found to be no significant difference between the levels of trust placed in the risk averse, risk neutral and human approach to risk. It is also found that the level to which participants question a robot's decisions does not form an accurate measure of trust. The results suggest that when designing a robot that must make risk based decisions during teleoperation in a hazardous environment, an engineer should avoid a risk seeking robot. However, that same engineer may choose whichever of the remaining risk profiles best suits the implementation, with knowledge that the trust in their system is unlikely to be significantly affected.

This letter presents a method for constrained motion planning from vision, which enables a robot to move its end-effector over an observed surface, given start and destination points. The robot has no prior knowledge of the surface shape, but observes it from a noisy point cloud. We consider the multi-objective optimisation problem of finding robot trajectories which maximise the robot's manipulability throughout the motion, while also minimising surface-distance travelled between the two points. This work has application in industrial problems of rough robotic cutting, e.g., demolition of legacy nuclear plant, where the cut path needs not be precise as long as it achieves dismantling. We show how detours in the path can be leveraged to increase the manipulability of the robot at all points along the path. This helps to avoid singularities, while maximising the robot's capability to make small deviations during task execution. We show how a sampling-based planner can be projected onto the Riemannian manifold of a curved surface, and extended to include a term which maximises manipulability. We present the results of empirical experiments, with both simulated and real robots, which are tasked with moving over a variety of different surface shapes. Our planner enables successful task completion, while ensuring significantly greater manipulability when compared against a conventional RRT* planner.

Force feedback in current teleoperation systems typically only provides haptic/kinaesthetic feedback to operators in the form of normal forces. However, it is proposed in this paper that the introduction of shear (or lateral) force feedback to operators' fingertips may provide an enhanced degree of control in manipulating remote objects for a wide variety of applications. A lightweight and wearable prototype device is developed and tested to demonstrate this hypothesis.

VENTURER was one of the first three UK government funded research and innovation projects on Connected Autonomous Vehicles (CAVs) and was conducted predominantly in the South West region of the country. A series of increasingly complex scenarios conducted in an urban setting were used to: (i) evaluate the technology created as a part of the project; (ii) systematically assess participant responses to CAVs and; (iii) inform the development of potential insurance models and legal frameworks. Developing this understanding contributed key steps towards facilitating the deployment of CAVs on UK roads. This paper aims to describe the VENTURER Project trials, their objectives and detail some of the key technologies used. Importantly we aim to introduce some informative challenges that were overcame and the subsequent project and technological lessons learned in a hope to help others plan and execute future CAV research. The project successfully integrated several technologies crucial to CAV development. These included, a Decision Making System using behaviour trees to make high level decisions; A pilot-control system to smoothly and comfortably turn plans into throttle and steering actuation; Sensing and perception systems to make sense of raw sensor data; Inter-CAV Wireless communication capable of demonstrating vehicle-to-vehicle communication of potential hazards. The closely coupled technology integration, testing and participant-focused trial schedule led to a greatly improved understanding of the engineering and societal barriers that CAV development faces. From a behavioural standpoint the importance of reliability and repeatability far outweighs a need for novel trajectories, while the sensor-to-perception capabilities are critical, the process of verification and validation is extremely time consuming. Additionally, the added capabilities that can be leveraged from inter-CAV communications shows the potential for improved road safety that could result. Importantly, to effectively conduct human factors experiments in the CAV sector under consistent and repeatable conditions, one needs to define a scripted and stable set of scenarios that uses reliable equipment and a controllable environmental setting. This requirement can often be at odds with making significant technology developments, and if both are part of a project’s goals then they may need to be separated from each other.

AbstractBackgroundPeople in the later stages of dementia often express their needs and distress through their behaviour in ways that challenge care staff to respond appropriately. Technological solutions may help staff to recognise this distress at an early stage and take preventative action. The aim of this project is to develop and test the feasibility of using wearable technologies and machine learning algorithms to identify stress and agitation.MethodTwo laboratory‐based feasibility studies were undertaken with healthy adult volunteers. In study 1, 20 participants took part in a computer‐based exercise designed to elicit stress. In study 2, 20 participants completed eight different physical activities which a person living with dementia might undertake during a typical day, plus two additional activities which were designed to be positively‐ and negatively‐stimulating. In both studies, participants’ physiological responses were recorded using wearables – in study 1, using commercially‐available devices worn around the wrist and waist, and in study 2, using an additional device developed by the researcher, worn in a sock. Participants also provided information on their affective states – in study 1 they rated their stress, and in study 2 they completed the Positive and Negative Affect Schedule (PANAS) before and after each activity. Participants’ responses were used to train and evaluate different binary classifiers for their ability to recognise reported stress in the physiological data.ResultsIn study 1, the K‐nearest neighbours algorithm and an ensemble classifier called bagged trees recognised periods when participants reported feeling most stressed with accuracy rates exceeding 90% (98% sensitivity, >99% specificity), using five‐fold validation. However, in study 2 which used a different group of participants, the algorithms trained in study 1 failed to reliably identify the negatively‐stimulating activity from the other activities.ConclusionTechnological solutions can facilitate a proactive approach to the management of stress and agitation in dementia. However, for these kinds of solutions to be effective in practice, they must be able to generalise to new users and different environments including uncontrolled environments. These kinds of solutions also raise ethical considerations, such as privacy and data security, which are particularly important when users may lack capacity.

Underactuated gripping mechanisms allow a wide range of objects to be grasped, with relatively simple control and input. Current 3D printed underactuated grippers are often composed of multiple parts that need assembly before use. Consolidating many of these parts allows the gripper to be manufactured more quickly for less money, and allows custom gripping devices to become more accessible. A novel one-shot printed underactuated gripping mechanism was developed, which was manufactured using HP’s MJF 3D printing process. The conventional tendon lines were replaced with a band which was 3D printed as part of the gripper. Finite Element Analysis was used to model the gripper behaviour, and 3D printed prototypes were manufactured and tested, which were to grip a range of objects.

We present a responsibility predictor, based on the adaptive filter model of the cerebellum, to provide feed-forward selection of cerebellar calibration models for robot Sound Source Localization (SSL), based on audio features extracted from the received audio stream. In previous work we described a system that selects the models based on sensory feedback, however, a drawback of that system is that it is only able to select a set of calibrators a-posteriori, after action (e.g. orienting a camera toward the sound source after a position estimate is made). The responsibility predictor improved the system performance compared to that without responsibility prediction. We show that a trained responsibility predictor is able to use contextual signals in the absence of ground truth to successfully select models with a performance approaching that of a system with full access to the ground truth through sensory feedback.