This document presents the outcomes of the Dagstuhl Seminar "Roadmap for Responsible Robotics," held in September 2023 at the Leibniz Centre for Informatics, Schloss Dagstuhl, Germany. The seminar brought together researchers from Robotics, Computer Science, Social and Cognitive Sciences, and Philosophy with the aim of charting a path towards improving responsibility in robotic systems. Through intensive interdisciplinary discussions centered on the various values at stake as robotics increasingly integrates into human life, the participants identified key priorities to guide future research and regulatory efforts. The resulting roadmap outlines actionable steps to ensure that robotic systems co-evolve with human societies, promoting human agency and humane values rather than undermining them. Designed for diverse stakeholders---researchers, policymakers, industry leaders, practitioners, NGOs, and civil society groups---this roadmap provides a foundation for collaborative efforts toward responsible robotics.

This document presents the outcomes of the Dagstuhl Seminar “Roadmap for Responsible Robotics,” held in September 2023 at the Leibniz Center for Informatics, Schloss Dagstuhl, Germany. The seminar brought together researchers from the fields of robotics, computer science, social and cognitive sciences, and philosophy with the aim of charting a path toward improving responsibility in robotic systems. Through intensive interdisciplinary discussions centered on the various values at stake as robotics increasingly integrates into human life, the participants identified key priorities to guide future research and regulatory efforts. The resulting road map outlines actionable steps to ensure that robotic systems coevolve with human societies, promoting human agency and humane values rather than undermining them. Designed for diverse stakeholders—researchers, policy makers, industry leaders, practitioners, nongovernmental organizations (NGOs), and civil society groups—this road map provides a foundation for collaborative efforts toward responsible robotics.

The robotics and autonomous systems communities have seen a significant and rapid increase in both the development of robots and vehicles for commercial use and in using these systems across a wide range of novel applications. As these robots, vehicles, software, and even embedded devices move toward much greater autonomy, techniques for verification, providing much higher confidence than usual, a...

We present an approach for the verification and validation (V&V) of robot assistants in the context of human–robot interactions, to demonstrate their trustworthiness through corroborative evidence of their safety and functional correctness. Key challenges include the complex and unpredictable nature of the real world in which assistant and service robots operate, the limitations on available V&V techniques when used individually, and the consequent lack of confidence in the V&V results. Our approach, called corroborative V&V, addresses these challenges by combining several different V&V techniques; in this paper we use formal verification (model checking), simulation-based testing, and user validation in experiments with a real robot. This combination of approaches allows V&V of the human–robot interaction task at different levels of modeling detail and thoroughness of exploration, thus overcoming the individual limitations of each technique. We demonstrate our approach through a handover task, the most critical part of a complex cooperative manufacturing scenario, for which we propose safety and liveness requirements to verify and validate. Should the resulting V&V evidence present discrepancies, an iterative process between the different V&V techniques takes place until corroboration between the V&V techniques is gained from refining and improving the assets (i.e., system and requirement models) to represent the human–robot interaction task in a more truthful manner. Therefore, corroborative V&V affords a systematic approach to “meta-V&V,” in which different V&V techniques can be used to corroborate and check one another, increasing the level of certainty in the results of V&V.

Current manufacturing applications are subject to constant changes in production orders for their robotic systems to adapt to the dynamic nature of the market. Hence, reprogramming robots needs to be a fast, easy and effective process. In this demonstration, we present an augmented reality (AR) interface using HoloLens. Our interface provides an intuitive platform to re-program a robotic packing application through simple hand gestures and the information gathered by the HoloLens' spatial mapping functionality.

The adoption of robots for collaborative tasks strongly depends on ensuring the safety of the operators, through both internal sensors for collision detection and protective stop, and external sensors to monitor human presence. External industrial safety sensors (e.g. laser scanners) are expensive, do not distinguish between a trained operator and a bystander, and the cycle time may be increased considerably as the robot stops or reduces its speed without considering the behaviour of the people around it in a more effective manner. We present a dynamic safety solution for human-robot collaboration that tracks human behaviour, based on on RGB-D cameras. Our solution updates in real-time the stopping and speed reducing areas of a robot, according to the robot's speed and the spatial relation between the robot and operators or bystanders. In our solution, the data from a commercial RGB-D camera provides more information about the people in the space, compared to industrial safety sensors. The presented solution is first evaluated through a case study of a collaborative robot in a pick-and-place task in a manufacturing workshop. Then, the solution is compared with off-the-shelf industrial safety sensors. Finally, we characterize the system's capabilities experimentally. The results indicate that using the proposed system significantly reduces the total average cycle time of the task, compared to traditional industrial safety setups.

Autonomous robots must operate in complex and changing environments subject to requirements on their behaviour. Verifying absolute satisfaction (true or false) of these requirements is challenging. Instead, we analyse requirements that admit flexible degrees of satisfaction. We analyse vague requirements using fuzzy logic, and probabilistic requirements using model checking. The resulting analysis method provides a partial ordering of system designs, identifying trade-offs between different requirements in terms of the degrees to which they are satisfied. A case study involving a home care robot interacting with a human is used to demonstrate the approach.

To design trustworthy robots, we need to understand the impact factors of trust: people's attitudes, experience, and characteristics; the robot's physical design, reliability, and performance; a task's specification and the circumstances under which it is to be performed, e.g. at leisure or under time pressure. As robots are used for a wide variety of tasks and applications, robot designers ought to be provided with evidence and guidance, to inform their decisions to achieve safe, trustworthy and efficient human-robot interactions. In this work, the impact factors of trust in a collaborative manufacturing scenario are studied by conducting an experiment with a real robot and participants where a physical object was assembled and then disassembled. Objective and subjective measures were employed to evaluate the development of trust, under faulty and non-faulty robot conditions, and the effect of previous experience with robots, and personality traits. Our findings highlight differences when compared to other, more social, scenarios with robotic assistants (such as a home care assistant), in that the condition (faulty or not) does not have a significant impact on the human's perception of the robot in terms of human-likeliness, likeability, trustworthiness, and even competence. However, personality and previous experience do have an effect on how the robot is perceived by participants, even though that is relatively small.

This article presents an implementation of an adaptive control architecture, which provides the combined advantages of better dynamic performance compared to other conventional industrial controllers, and the use of widely available hardware in process industry. Adaptive control architecture uses proportional–integral action and dynamic computation of the controller’s gains (self-tuning regulator), to maintain performance specifications, even in the presence of parametric disturbances. This architecture offers advantages over other advanced embedded control systems implemented on industrial programmable logic controllers and other hardware platforms. Implementation of controllers on industrial hardware platforms is possible through the Object Linking and Embedding (OLE) for process control communication standard. The implementation for an adaptive controller here proposed was evaluated through experiments using first-order and overdamped second-order systems emulated by hardware-in-the-loop, with a programmable automation controller. Performance of the adaptive controllers was compared to that of conventional proportional–integral controllers, and effectiveness of the former over the latter was demonstrated through the experiments carried out.

Safety in autonomous systems has been mostly studied from a human-centered perspective. Besides the loads they may carry, autonomous systems are also valuable property, and self-preservation mechanisms are needed to protect them in the presence of external threats, including malicious robots and antagonistic humans. We present a biologically inspired risk-based triggering mechanism to initiate self-preservation strategies. This mechanism considers environmental and internal system factors to measure the overall risk at any moment in time, to decide whether behaviours such as fleeing or hiding are necessary, or whether the system should continue on its task. We integrated our risk-based triggering mechanism into a delivery rover that is being attacked by a drone and evaluated its effectiveness through systematic testing in a simulated environment in Robot Operating System (ROS) and Gazebo, with a variety of different randomly generated conditions. We compared the use of the triggering mechanism and different configurations of self-preservation behaviours to not having any of these. Our results show that triggering self-preservation increases the distance between the drone and the rover for many of these configurations, and, in some instances, the drone does not catch up with the rover. Our study demonstrates the benefits of embedding risk awareness and self-preservation into autonomous systems to increase their robustness, and the value of using bio-inspired engineering to find solutions in this area.

The widespread adoption of autonomous systems depends on providing guarantees of safety and functional correctness, at both design time and runtime. Information about the extent to which functional requirements can be met in combination with non-functional requirements (NFRs) -- i.e. requirements that can be partially complied with -- , under dynamic and uncertain environments, provides opportunities to enhance the safety and functional correctness of systems at design time. We present a technique to formally define system attributes that can change or be changed to deal with dynamic and uncertain environments (denominated weakened specifications) as a partially ordered lattice, and to automatically explore the system under different specifications, using probabilistic model checking, to find the likelihood of satisfying a requirement. The resulting probabilities form boundaries of "optimal specifications", analogous to Pareto frontiers in multi-objective optimization, informing the designer about the system's capabilities, such as resilience or robustness, when changing its attributes to deal with dynamic and uncertain environments. We illustrate the proposed technique through a domestic robotic assistant example.

Self-adaptive systems change their operational behaviour for instance to accommodate variations in their environment, while preserving functional requirements and maintaining acceptable conformance to non-functional requirements (NFRs). While conformance with functional requirements is clear-cut, it is more challenging to specify acceptable behaviours when considering multiple NFRs, which are permitted to be partially satisfied. We propose a method for formalising how conformance to NFRs degrades and an automatic technique to compute a partially ordered set of levels of conformance. We then introduce a technique to systematically analyse the level of conformance to functional requirements that is achievable, allowing ranking and discriminating between system configurations (instantiation of system variables). This enables, at design time, understanding the impact that degrading NFRs may have on the overall system behaviour. Our technique offers design space exploration thus enabling the designer to analyse trade-offs between NFRs. We demonstrate the use and potential that this technique has on an autonomous robot case study.

The widespread adoption of autonomous adaptive systems depends on provided guarantees of safety and functional correctness, at both design time and runtime. Specifying adaptive systems is cognitively difficult when their aspects are in a large number and have complicated dependencies. We present a technique to construct and automatically explore a specification for systems that can degrade and/or adapt, towards analysis at design time for verification and validation. This technique combines and constructs sections of a lattice (or Hasse diagram) of all the possible ordered system degradations/adaptations of interest, limited by desirability or risk thresholds. The lattice allows the designer to understand the different levels and combinations of system degradations/adaptations. We use the lattices (or sections) to systematically explore whether a system is able to fulfil its task goals under a dynamic and uncertain environment, through probabilistic model checking. We illustrate the proposed specification technique through a domestic robotic assistant example. Systematically exploring the lattice allowed comparing the probabilities of task success/failure, to determine which degradation/adaptation combinations can be allowed in the final system implementation.

Industries such as flexible manufacturing and home care will be transformed by the presence of robotic assistants. Assurance of safety and functional soundness for these robotic systems will require rigorous verification and validation. We propose testing in simulation using Coverage-Driven Verification (CDV) to guide the testing process in an automatic and systematic way. We use a two-tiered test generation approach, where abstract test sequences are computed first and then concretized (e.g., data and variables are instantiated), to reduce the complexity of the test generation problem. To demonstrate the effectiveness of our approach, we developed a testbench for robotic code, running in ROS-Gazebo, that implements an object handover as part of a humanrobot interaction (HRI) task. Tests are generated to stimulate the robot's code in a realistic manner, through stimulating the human, environment, sensors, and actuators in simulation. We compare the merits of unconstrained, constrained and model-based test generation in achieving thorough exploration of the code under test, and interesting combinations of human-robot interactions. Our results show that CDV combined with systematic test generation achieves a very high degree of automation in simulation-based verification of control code for robots in HRI.

The challenges of robotic software testing extend beyond conventional software testing. Valid, realistic and interesting tests need to be generated for multiple programs and hardware running concurrently, deployed into dynamic environments with people. We investigate the use of Belief-Desire-Intention (BDI) agents as models for test generation, in the domain of human-robot interaction (HRI) in simulations. These models provide rational agency, causality, and a reasoning mechanism for planning, which emulate both intelligent and adaptive robots, as well as smart testing environments directed by humans. We introduce reinforcement learning (RL) to automate the exploration of the BDI models using a reward function based on coverage feedback. Our approach is evaluated using a collaborative manufacture example, where the robotic software under test is stimulated indirectly via a simulated human co-worker. We conclude that BDI agents provide intuitive models for test generation in the HRI domain. Our results demonstrate that RL can fully automate BDI model exploration, leading to very effective coverage-directed test generation.

Robotic code needs to be verified to ensure its safety and functional correctness, especially when the robot is interacting with people. Testing real code in simulation is a viable option. However, generating tests that cover rare scenarios, as well as exercising most of the code, is a challenge amplified by the complexity of the interactions between the environment and the software. Model-based test generation methods can automate otherwise manual processes and facilitate reaching rare scenarios during testing. In this paper, we compare using Belief-Desire-Intention (BDI) agents as models for test generation with more conventional automata-based techniques that exploit model checking, in terms of practicality, performance, transferability to different scenarios, and exploration (`coverage'), through two case studies: a cooperative manufacturing task, and a home care scenario. The results highlight the advantages of using BDI agents for test generation. BDI agents naturally emulate the agency present in Human-Robot Interactions (HRIs), and are thus more expressive than automata. The performance of the BDI-based test generation is at least as high, and the achieved coverage is higher or equivalent, compared to test generation based on model checking automata.