Despite recent technological progress in the development of autonomous vehicles (AVs), their societal acceptability remains a subject of debate as recent research findings point to psychological roadblocks. Concerns arise not only for physical safety but also for potential psychological risks resulting from human interaction with AVs. Psychological concepts such as trust, and perceived safety are well-studied in this context and are found to be determinant factors for the intention to use AVs. Unfortunately, there has been no formalization of the mechanism by which human interaction with AVs may lead to psychological hazards, threatening trust, perceived safety, and acceptability. Furthermore, there has been little prior research that conceptualizes the severity of psychological risk in AVs, and there are no clear guidelines for a systems designer on how to assess and address psychological risk in the AV development context. To address these limitations, this paper extends a theoretical framework for AV psychological safety risk assessment based on an early proposal. The proposed framework consists of an extended risk model for psychological safety including all the key concepts related to psychological safety in AVs, and an assessment method based on the Systems-Theoretic Accident Model and Processes (STAMP). We demonstrate the usefulness of the theoretical framework through a highly automated AV use case scenario, uncovering factors which may lead to psychological risk for an occupant. The use cases provide examples of how to use the framework to extensively evaluate psychological risk and determine vehicle behaviour that could lead to this risk. By developing a theoretical framework for AV psychological safety risk assessment, we provide a foundation and a method that were previously lacking to better enable responsible AV development regarding psychological safety.

In this paper, we critically examine the current "humanoid hype" in robotics, questioning its alignment with responsible robotics principles. While technical challenges drive internal fascination, the pervasive public image of humanoids demands deeper HRI engagement. We explore how responsible robotics concepts, such as privacy, dignity, and trust, are uniquely challenged or overlooked in the pursuit of anthropomorphic robot forms. By dissecting this hype, and mapping the main findings of the recently-published Roadmap for Responsible Robotics to the humanoids field, we aim to move beyond technical form-factor obsessions to understand the true societal implications and identify potential blind spots for the HRI community.

Despite rapid technological advances, the societal acceptability of autonomous vehicles (AVs) remains limited by psychological barriers that extend beyond traditional concerns of physical safety. While factors such as trust and perceived safety are known to influence user acceptance, there is a lack of formalized mechanisms and engineering methods to systematically identify, assess, and mitigate psychological risks arising from human-AV interactions. To address this gap, this work proposes and validates a systems-theoretic framework for the assessment of psychological safety in autonomous vehicles. First, a comprehensive psychological safety risk model is defined, extending the Systems-Theoretic Accident Model and Processes (STAMP) to incorporate key psychological constructs such as trust, perceived control, predictability, and perceived support. Based on this model, a hazard analysis method (AV-PsySafe) is developed to systematically identify psychological hazards, unsafe control actions, and loss scenarios, while introducing a Psychological Safety Integrity Level (PsySIL) to support risk prioritization. Second, the applicability and relevance of the framework are evaluated through its deployment in realistic autonomous vehicle scenarios. A structured validation approach is implemented, including a methodological guide, standardized analysis templates, and the collection of analyst feedback. The results demonstrate that the framework can be consistently applied by practitioners, producing meaningful insights into psychological risks. Overall, this work establishes both the theoretical foundations and practical feasibility of a unified approach to co-assessing psychological and physical safety in autonomous systems, contributing to more human-centred and trustworthy AV development.

This document presents the outcomes of the Dagstuhl Seminar "Roadmap for Responsible Robotics," held in September 2023 at the Leibniz Centre for Informatics, Schloss Dagstuhl, Germany. The seminar brought together researchers from Robotics, Computer Science, Social and Cognitive Sciences, and Philosophy with the aim of charting a path towards improving responsibility in robotic systems. Through intensive interdisciplinary discussions centered on the various values at stake as robotics increasingly integrates into human life, the participants identified key priorities to guide future research and regulatory efforts. The resulting roadmap outlines actionable steps to ensure that robotic systems co-evolve with human societies, promoting human agency and humane values rather than undermining them. Designed for diverse stakeholders---researchers, policymakers, industry leaders, practitioners, NGOs, and civil society groups---this roadmap provides a foundation for collaborative efforts toward responsible robotics.

AI-enhanced reasoning enables robots to create detailed accounts of their own situated behaviour as well as the behaviour of other people. This capability is currently employed by robot designers to achieve transparency, trust, and enhance robot social and communicative capabilities. Furthermore, robots may be designed to resemble humans both in their physical appearance and their behaviour. This approach is intended to facilitate more effective interactions with people. In this article we identify and examine some of the ethical, social and legal implications of these capabilities for the investigation of robot accidents. We consider two aspects in particular. The first of these is the role of robots as subjects in a testimony regarding an incident in which they are directly or indirectly involved. This can be described as a case of robots acting as witnesses. The second aspect is the role of robots as objects in a human testimony. This can be described as a case of robots being witnessed.

This document presents the outcomes of the Dagstuhl Seminar “Roadmap for Responsible Robotics,” held in September 2023 at the Leibniz Center for Informatics, Schloss Dagstuhl, Germany. The seminar brought together researchers from the fields of robotics, computer science, social and cognitive sciences, and philosophy with the aim of charting a path toward improving responsibility in robotic systems. Through intensive interdisciplinary discussions centered on the various values at stake as robotics increasingly integrates into human life, the participants identified key priorities to guide future research and regulatory efforts. The resulting road map outlines actionable steps to ensure that robotic systems coevolve with human societies, promoting human agency and humane values rather than undermining them. Designed for diverse stakeholders—researchers, policy makers, industry leaders, practitioners, nongovernmental organizations (NGOs), and civil society groups—this road map provides a foundation for collaborative efforts toward responsible robotics.

The rapid advancement of artificial intelligence and autonomous driving technologies has significantly propelled the development of autonomous vehicles (AVs). However, psychological barriers continue to impede widespread AV adoption, despite technological progress. This paper addresses the critical yet often overlooked aspect of psychological safety in AV design and operation. While traditional safety standards focus primarily on physical safety, this paper emphasizes the psychological implications that arise from human interactions with autonomous vehicles, highlighting the importance of trust and perceived risk as significant factors influencing user acceptance. The paper makes a methodological proposal, a framework for addressing AVs psychological safety consisting of three key contributions. First, it introduces a definition of psychological safety in AVs context. Secondly, it proposes a risk model for identifying and assessing AVs psychological hazards and risks. PsySIL (Psychological Safety Integrity Level), a classification of AV psychological risk levels is developed. Thirdly, an adapted system-theoretic analysis method for AVs psychological safety is proposed. The paper illustrates the application of the framework for assessing potential psychological hazards using a scenario involving a family's experience with an autonomous vehicle, pioneering a systems approach towards evaluating situations that could lead to psychological harm. By establishing a framework that incorporates psychological safety alongside physical safety, the paper contributes to the broader discourse on the safe deployment of autonomous vehicle, aiming to guide future developments in user-centred design and regulatory practices, while acknowledging the limitations brought by the application of the proposals on a rather simple but pedagogical illustrative example.

Remote driving is a promising strategy for helping Autonomous Vehicles (AVs) navigate many environments where edge cases may otherwise limit their abilities. For some companies, remote driving is an alternative to AVs altogether. Much remote driving research has taken place in simulated or controlled environments with novice operators, leaving the needs of operators with real-world experience under-explored. This research aims to understand if experienced operators are satisfied with current production remote driving systems, if they adapt to the difference in control, and how their job satisfaction compares to in-vehicle safety driving. This paper briefly overviews recent remote driving research and presents results from a questionnaire and a semi-structured interview with experienced teleoperators. The findings indicate that operators do adjust to the new domain, but latency and network reliability remain a challenge. Likewise, standardised training practices for operators are found to be lacking.

This article reports results from a public survey designed to evaluate public attitudes and perceptions towards data recorders in Autonomous Vehicles (AVs). Our study indicated that road users are willing to make compromises about their privacy in and around AVs, as long as the data recorded is used to improve vehicle safety. Our study also indicated that more vulnerable road-users such as pedestrians, cyclists and horse-riders are willing to be recorded by on-board devices, however this willingness is linked to the data from these devices being accessible to determine liability and the cause of an accident or near-miss. However, the type of data recording currently mandated by international legal frameworks does not accord with these public expectations. While the results of our survey highlights a gap between the international legal obligations of manufacturers and the expectations of the public, it is also relevant to inform policy makers at the national level on the public's view about the importance of data recorders in the development of trustworthy autonomous vehicles. The failure of AVs to meet societal expectations on transparent data recording frameworks, and the use of that data to improve safety, may impact the uptake and acceptance of AVs.

Automotive companies are investing in automated vehicles, conducting real world testing, and many countries are setting the pre-conditions for being the first to receive the benefits from this technology. In this study, we conducted a content analysis of the German Act on Autonomous Driving (2021) – one of the most advanced regulatory frameworks on driverless vehicles - to evaluate the role of regulation in achieving three socially desirable promises related to driverless cars: reducing road accidents, improving environmental sustainability, and ensuring equal access to road mobility.The main finding from our analysis of the German legal framework is that there was evidence of the three promises, but no provisions to operationalize them.

Physically embodied artificial agents, or robots, are being incorporated into various practical and social contexts, from self-driving cars for personal transportation to assistive robotics in social care. To enable these systems to better perform under changing conditions, designers have proposed to endow robots with varying degrees of autonomous capabilities and the capacity to move between them—an approach known as variable autonomy. Researchers are beginning to understand how robots with fixed autonomous capabilities influence a person’s sense of autonomy, social relations, and, as a result, notions of responsibility; however, addressing these topics in scenarios where robot autonomy dynamically changes is underexplored. To establish a research agenda for variable autonomy that emphasises the responsible design and use of robotics, we conduct a developmental review. Based on a sample of 42 papers, we provide a synthesised definition of variable autonomy to connect currently disjointed research efforts, detail research approaches in variable autonomy to strengthen the empirical basis for subsequent work, characterise the dimensions of variable autonomy, and present design guidelines for variable autonomy research based on responsible robotics.

This paper describes the enactment of a simulated (mock) accident involving an upper-body exoskeleton and its investigation. The accident scenario is enacted by role-playing volunteers, one of whom is wearing the exoskeleton. Following the mock accident, investigators - also volunteers - interview both the subject of the accident and relevant witnesses. The investigators then consider the witness testimony alongside robot data logged by the ethical black box, in order to address the three key questions: what happened?, why did it happen?, and how can we make changes to prevent the accident happening again? This simulated accident scenario is one of a series we have run as part of the RoboTIPS project, with the overall aim of developing and testing both processes and technologies to support social robot accident investigation.

Many sources propose that Autonomous Vehicles (AVs) could offer societal benefits in the future. Although such claims frequently relate to safety, AVs will undoubtedly also create new types of incidents and accidents. Data related to failure or accidents will therefore be a fundamental requirement for ensuring safety, accountability and public trust. Regulations and standards are being developed for equipping AVs with data recorders, also known as Black Boxes. These devices can log different types of parameters related to the vehicle status and collect large volumes of data relating to the vehicle and the surrounding environment. Although data retrieval is vital to understanding the causes of an accident and contributing to ongoing safety developments, there can be ethical risks as well as legal, social and political implications related to collection, storage, processing, access and use of data. In this work, we took a responsible innovation approach to these questions, seeking to establish the current practice with regard to technology, people, and institutions involved with AV data recorders, evaluate the usefulness of present regulations for defining safety-critical scenarios, and identify gaps that could have significant consequences further down the line. We close with recommendations for future work and practice.

In this study, we analyse the German Act on Autonomous Driving (2021) – one of the most advanced regulatory frameworks on fully autonomous vehicles - to evaluate the role of regulation in achieving the following three promises related to self-driving cars: reducing road accidents, improving environmental sustainability, and ensuring equal access to road mobility. We argue that these three promises have become the leitmotivs running through the autonomous driving technology narrative. The main finding from our analysis of the German legal framework is that there was evidence of the three promises, but no provisions to operationalize them. We discuss these findings and offer suggestions on how the German regulations on autonomous vehicles could be improved.

This paper draws on three case studies to examine some of the challenges and tensions involved in the use of Autonomous Decision-Making Systems (ADMS). In particular, the paper highlights: (i) challenges around the shifting “locale” of the decision, and the associated consequences for stakeholders; (ii) potential implications for stakeholders from regulation such as the General Data Protection Regulation (GDPR); (iii) the different values that stakeholder groups bring to the “decision” question; (iv) how complex pre-existing webs of stakeholders and decision-making authorities may be disrupted or disempowered by the use of an automated system and the lack of evaluation of possible consequences; (v) how ADMS for non-technical users can lead to circumvention of the boundaries of intended system use. We illustrate these challenges through case studies in three domains: adult social care, aviation, and vehicle driver monitoring systems. The paper closes with recommendations for both practice and policy in the deployment of ADMS.

Autonomous Vehicles (AVs) collect a vast amount of data during their operation (MBs/sec). What data is recorded, who has access to it, and how it is analysed and used can have major technical, ethical, social, and legal implications. By embedding Responsible Innovation (RI) methods within the AV lifecycle, negative consequences resulting from inadequate data logging can be foreseen and prevented. An RI approach demands that questions of societal benefit, anticipatory governance, and stakeholder inclusion, are placed at the forefront of research considerations. Considered as foundational principles, these concepts create a contextual mindset for research that will by definition have an RI underpinning as well as application. Such an RI mindset both inspired and governed the genesis and operation of a research project on autonomous vehicles. The impact this had on research outlines and workplans, and the challenges encountered along the way are detailed, with conclusions and recommendations for RI in practice.

This paper introduces a draft open standard for the robot equivalent of an aircraft flight data recorder, which we call an ethical black box. This is a device, or software module, capable of securely recording operational data (sensor, actuator and control decisions) for a social robot, in order to support the investigation of accidents or near-miss incidents. The open standard, presented as an annex to this paper, is offered as a first draft for discussion within the robot ethics community. Our intention is to publish further drafts following feedback, in the hope that the standard will become a useful reference for social robot designers, operators and robot accident/incident investigators.

Risk Assessment is a well known and powerful method for discovering and mitigating risks, and hence improving safety. Ethical Risk AssessmentEthical Risk Assessment uses the same approach, but extends the scope of risk to cover ethical risks in addition to safety risks. In this paper we outline Ethical Risk AssessmentEthical Risk Assessment (ERA), and set ERA within the broader framework of Responsible RoboticsResponsible Robotics. We then illustrate ERA, first with a hypothetical smart robotRobots teddy bear (RoboTed), and later with an actual smart robot toySmart Robot Toy (Purrble). Through these two case studies this paper demonstrates the value of ERA and how consideration of ethical risks can prompt design changes, resulting in more ethical and sustainable robotsRobots.

The slogan “robots will pervade our environment” has become a reality. Drones and ground robots are used for commercial purposes while semi-autonomous driving systems are standard accessories to traditional cars. However, while our eyes have been riveted on dangers and accidents arising from drones falling and autonomous cars’ crashing, much less attention has been ported to dangers arising from the imminent arrival of robots that share the floor with pedestrians and will mix with human crowds. These robots range from semi or autonomous mobile platforms designed for providing several kinds of service, such as assistant, patrolling, tour-guide, delivery, human transportation, etc. We highlight and discuss potential sources of injury emerging from contacts of robots with pedestrians through a set of case studies. We look specifically at dangers deriving from robots moving in dense crowds. In such situations, contact will not only be unavoidable, but may be desirable to ensure that the robot moves with the flow. As an outlook toward the future, we also offer some thoughts on the psychological risks, beyond the physical hazards, arising from the robot’s appearance and behaviour. We also advocate for new policies to regulate mobile robots traffic and enforce proper end user’s training.

Since 2014, a specific standard has been dedicated for the safety certification of personal care robots, which operate in close proximity to humans. These robots serve as information providers, object transporters, personal mobility carriers, and security patrollers. In this article, we point out the shortcomings concerning EN ISO 13482:2014, which encompasses guidelines regarding the safety and design of personal care robots. In particular, we argue that the current standard is not suitable for guaranteeing people's safety when these robots operate in public spaces. Specifically, the standard lacks requirements to protect pedestrians and bystanders. The guideline implicitly assumes that private spaces, such as households and offices, present the same hazards as in public spaces. We highlight the existence of at least three properties pertaining to robots’ use in public spaces. These properties include (1) crowds, (2) social norms and proxemics rules, and (3) people's misbehaviours. We discuss how these properties impact robots’ safety. This article aims to raise stakeholders’ awareness on individuals’ safety when robots are deployed in public spaces. This could be achieved by integrating the gaps present in EN ISO 13482:2014 or by creating a new dedicated standard.