A number of concerns have been recently raised regarding the possibility of human agents to effectively maintain control over intelligent and (partially) autonomous artificial systems. These issues have been deemed to raise “responsibility gaps.” To address these gaps, several scholars and other public and private stakeholders converged towards the idea that, in deploying intelligent technology, a meaningful form of human control (MHC) should be at all times exercised over autonomous intelligent technology. One of the main criticisms of the general idea of MHC is that it could be inherently problematic to have high degrees of control and high degrees of autonomy at the same time, as the two dimensions appear to be inversely related. Several ways to respond to this argument and deal with the dilemma between control and autonomy have been proposed in the literature. In this paper, we further contribute to the philosophical effort to overcome the trade-off between automation and human control, and to open up some space for moral responsibility. We will use the instrument of conceptual engineering to investigate whether and to what extent removing the element of direct causal intervention from the concept of control can preserve the main functions of that concept, specifically focusing on the extent it can act as a foundation of moral responsibility. We show that at least one philosophical account of MHC is indeed a conceptually viable theory to absolve the fundamental functions of control, even in the context of completely autonomous artificial systems.

The urban traffic environment is characterized by the presence of a highly differentiated pool of users, including vulnerable ones. This makes vehicle automation particularly difficult to implement, as a safe coordination among those users is hard to achieve in such an open scenario. Different strategies have been proposed to address these coordination issues, but all of them have been found to be costly for they negatively affect a range of human values (e.g. safety, democracy, accountability…). In this paper, we claim that the negative value impacts entailed by each of these strategies can be interpreted as lack of what we call Meaningful Human Control over different parts of a sociotechnical system. We argue that Meaningful Human Control theory provides the conceptual tools to reduce those unwanted consequences, and show how “designing for meaningful human control” constitutes a valid strategy to address coordination issues. Furthermore, we showcase a possible application of this framework in a highly dynamic urban scenario, aiming to safeguard important values such as safety, democracy, individual autonomy, and accountability. Our meaningful human control framework offers a perspective on coordination issues that allows to keep human actors in control while minimizing the active, operational role of the drivers. This approach makes ultimately possible to promote a safe and responsible transition to full automation.

Novel wearable neurotechnology is able to provide insight into its wearer's cognitive processes and offers ways to change or enhance their capacities. Moreover, it offers the promise of hands-free device control. These brain-computer interfaces are likely to become an everyday technology in the near future, due to their increasing accessibility and affordability. We, therefore, must anticipate their impact, not only on society and individuals broadly but also more specifically on sectors such as traffic and transport. In an economy where attention is increasingly becoming a scarce good, these innovations may present both opportunities and challenges for daily activities that require focus, such as driving and cycling. Here, we argue that their development carries a dual risk. Firstly, BCI-based devices may match or further increase the intensity of cognitive human-technology interaction over the current hands-free communication devices which, despite being widely accepted, are well-known for introducing a significant amount of cognitive load and distraction. Secondly, BCI-based devices will be typically harder than hands-free devices to both visually detect (e.g., how can law enforcement check when these extremely small and well-integrated devices are used?) and restrain in their use (e.g., how do we prevent users from using such neurotechnologies without breaching personal integrity and privacy?). Their use in traffic should be anticipated by researchers, engineers, and policymakers, in order to ensure the safety of all road users.

The paper presents a framework to realise "meaningful human control" over Automated Driving Systems. The framework is based on an original synthesis of the results of the multidisciplinary research project "Meaningful Human Control over Automated Driving Systems" lead by a team of engineers, philosophers, and psychologists at Delft University of the Technology from 2017 to 2021. Meaningful human control aims at protecting safety and reducing responsibility gaps. The framework is based on the core assumption that human persons and institutions, not hardware and software and their algorithms, should remain ultimately-though not necessarily directly-in control of, and thus morally responsible for, the potentially dangerous operation of driving in mixed traffic. We propose an Automated Driving System to be under meaningful human control if it behaves according to the relevant reasons of the relevant human actors (tracking), and that any potentially dangerous event can be related to a human actor (tracing). We operationalise the requirements for meaningful human control through multidisciplinary work in philosophy, behavioural psychology and traffic engineering. The tracking condition is operationalised via a proximal scale of reasons and the tracing condition via an evaluation cascade table. We review the implications and requirements for the behaviour and skills of human actors, in particular related to supervisory control and driver education. We show how the evaluation cascade table can be applied in concrete engineering use cases in combination with the definition of core components to expose deficiencies in traceability, thereby avoiding so-called responsibility gaps. Future research directions are proposed to expand the philosophical framework and use cases, supervisory control and driver education, real-world pilots and institutional embedding.

The notion of “responsibility gap” with artificial intelligence (AI) was originally introduced in the philosophical debate to indicate the concern that “learning automata” may make more difficult or impossible to attribute moral culpability to persons for untoward events. Building on literature in moral and legal philosophy, and ethics of technology, the paper proposes a broader and more comprehensive analysis of the responsibility gap. The responsibility gap, it is argued, is not one problem but a set of at least four interconnected problems – gaps in culpability, moral and public accountability, active responsibility—caused by different sources, some technical, other organisational, legal, ethical, and societal. Responsibility gaps may also happen with non-learning systems. The paper clarifies which aspect of AI may cause which gap in which form of responsibility, and why each of these gaps matter. It proposes a critical review of partial and non-satisfactory attempts to address the responsibility gap: those which present it as a new and intractable problem (“fatalism”), those which dismiss it as a false problem (“deflationism”), and those which reduce it to only one of its dimensions or sources and/or present it as a problem that can be solved by simply introducing new technical and/or legal tools (“solutionism”). The paper also outlines a more comprehensive approach to address the responsibility gaps with AI in their entirety, based on the idea of designing socio-technical systems for “meaningful human control", that is systems aligned with the relevant human reasons and capacities.

Passive BCIs can be used to measure brain processes that take place without necessarily having the intention to communicate, or even while being unaware that specific information about mental states is being collected. This type of symbiotic neurotechnology has the potential to create new and philosophically fascinating cases where the question of "was that me?" will make sense from both an individual and a societal perspective. We think that symbiotic technology is philosophically interesting in that it enables subconscious brain states to influence actions in a new, technology-mediated way. We will examine some of these cases and make a plea for a more systematic use of symbiotic technology in experimentally guided thought experiments aimed at studying the sense of agency. Our guiding questions are: What could technology-induced agency confusions tell us about the experience of ownership of action? What theoretical (e.g., conceptual) and practical implications (e.g., related to identity and responsibility) might this have?

The target article (Schonau et al. 2021) recognizes four key ethical dimensions, or values, that are affected by neurotechnology, and proposes that the notion of agency can provide a unifying conce...

In this chapter, we analyze the ethical and societal implications of AI-powered neuroimaging. Our examination takes ongoing research as a starting point, but we aim to analyze issues that can emerge in parallel with technological developments. We will suggest that the rapid progress in AI, as applied to brain reading, requires a careful consideration of various forms of expiry dates, especially for informed consent, data storage, and data analysis.

Vehicle cooperation, not vehicle automation, will yield the greatest benefits on road traffic. However, satisfactory human control over platoons of cooperative vehicles still has a large number of uncertainties and issues to be addressed. This paper aims to address these broader issues of control over a cooperative vehicle platoon by focussing on a truck platooning system as a case example, and taking the perspective of Meaningful Human Control (MHC) as control concept. MHC goes further than mere operational control as it addresses issues that exist in current system designs, and proposes improvements based on a novel and more encompassing set of conditions for control. These issues are addressed in regard to the vehicles and their Operational Design Domains (ODD), the role and ability of the drivers (both leading and following) and how these exist in regard to their road environment. We conclude that current advances are making progress, but that from a MHC perspective, issues still remain for the operational and tactical implementation of truck platoons and cooperative driving that need to be addressed in regard to ODDs and drivers. Furthermore, consideration of responsibility and liability aspects is required that stretches beyond nominal appointment thereof, as this does not satisfy important ethical and societal standards. This is demonstrated in the paper through two hypothetical cases focussing on issues on a system level and one further analysis which is focussed on the role of the driver in the platooning system.

Increased on-road testing and market availability of partially automated vehicles (AV) offers researchers and developers the opportunity to evaluate the AV’s performance. The occurrence of new types of accidents involving AV’s has sparked questions in regard to who is actually in control over and responsible for AV control. In this contribution, we suggest a potential discrepancy in AV control with the review of recently documented accidents involving AV’s. The identification of a gap in control is performed using a recently formulated moral philosophical framework of Meaningful Human Control (MHC). This shows a discrepancy between the attribution of responsibility and the ability of a human to fulfil the role assigned to them. While a gap in control is not evident from the viewpoint of operational control, it requires the more intricate concept of MHC to expose it. Recommendations are further made that AV developers and vehicle approval authorities should consider control from a MHC perspective to avoid future gaps in control with the resulting consequences.

The introduction of automated vehicles means that some or all operational control over these vehicles is diverted away from a human driver to a technological system. The concept of Meaningful Human Control (MHC) was derived to address control issues over automated systems, allowing a system to explicitly consider human intentions and reasons. Applying MHC to technological systems, such as automated driving is a real challenge, and the main focus of this article. An approach with mathematical elaboration has been developed that offers a first quantifiable operationalisation of MHC for the traffic domain and for use with automated vehicles. A major contribution lies in the taxonomification of control for MHC in the broader traffic environment, including consideration of the driver, the vehicle, the traffic environment, considering behaviour, moral standards and societal values, which are considered in a case study. The demonstration case shows the validity of the developed approach for an automated vehicle overtaking a cyclist on an urban street. This article is one of the first to operationalise MHC to such a level of detail and opens the door to further development of the concept for technological implementation.

Rapid advancements in machine learning techniques allow mass surveillance to be applied on larger scales and utilize more and more personal data. These developments demand reconsideration of the privacy-security dilemma, which describes the tradeoffs between national security interests and individual privacy concerns. By investigating mass surveillance techniques that use bulk data collection and machine learning algorithms, we show why these methods are unlikely to pinpoint terrorists in order to prevent attacks. The diverse characteristics of terrorist attacks-especially when considering lone-wolf terrorism-lead to irregular and isolated (digital) footprints. The irregularity of data affects the accuracy of machine learning algorithms and the mass surveillance that depends on them which can be explained by three kinds of known problems encountered in machine learning theory: class imbalance, the curse of dimensionality, and spurious correlations. Proponents of mass surveillance often invoke the distinction between collecting data and metadata, in which the latter is understood as a lesser breach of privacy. Their arguments commonly overlook the ambiguity in the definitions of data and metadata and ignore the ability of machine learning techniques to infer the former from the latter. Given the sparsity of datasets used for machine learning in counterterrorism and the privacy risks attendant with bulk data collection, policymakers and other relevant stakeholders should critically re-evaluate the likelihood of success of the algorithms and the collection of data on which they depend.

In this paper, in line with the general framework of value-sensitive design, we aim to operationalize the general concept of “Meaningful Human Control” (MHC) in order to pave the way for its translation into more specific design requirements. In particular, we focus on the operationalization of the first of the two conditions (Santoni de Sio and Van den Hoven 2018) investigated: the so-called ‘tracking’ condition. Our investigation is led in relation to one specific subcase of automated system: dual-mode driving systems (e.g. Tesla ‘autopilot’). First, we connect and compare meaningful human control with a concept of control very popular in engineering and traffic psychology (Michon 1985), and we explain to what extent tracking resembles and differs from it. This will help clarifying the extent to which the idea of meaningful human control is connected to, but also goes beyond, current notions of control in engineering and psychology. Second, we take the systematic analysis of practical reasoning as traditionally presented in the philosophy of human action (Anscombe, Bratman, Mele) and we adapt it to offer a general framework where different types of reasons and agents are identified according to their relation to an automated system’s behaviour. This framework is meant to help explaining what reasons and what agents (should) play a role in controlling a given system, thereby enabling policy makers to produce usable guidelines and engineers to design systems that properly respond to selected human reasons. In the final part, we discuss a practical example of how our framework could be employed in designing automated driving systems.

The human species is combining an increased understanding of our cognitive machinery with the development of a technology that can profoundly influence our lives and our ways of living together. Our sciences enable us to see our strengths and weaknesses, and build technology accordingly. What would future historians think of our current attempts to build increasingly smart systems, the purposes for which we employ them, the almost unstoppable goldrush toward ever more commercially relevant implementations, and the risk of superintelligence? We need a more profound reflection on what our science shows us about ourselves, what our technology allows us to do with that, and what, apparently, we aim to do with those insights and applications. As the smartest species on the planet, we don't need more intelligence. Since we appear to possess an underdeveloped capacity to act ethically and empathically, we rather require the kind of technology that enables us to act more consistently upon ethical principles. The problem is not to formulate ethical rules, it's to put them into practice. Cognitive neuroscience and AI provide the knowledge and the tools to develop the moral crutches we so clearly require. Why aren't we building them? We don't need superintelligence, we need superethics.

As automated vehicles become increasingly common on the road, the call for an appropriate preparation for its drivers is becoming more urgent.Expert opinions and insights have been acquired via a focus group discussion with eleven Dutch driving examiners to assist in inventorying what types of preparations are needed.The concept of meaningful human control (MHC) as an integral part of the discussion lead to consensual findings regarding ADAS functionality and the drivers' tasks, as well as discussion topics on driver training and levels of automation.It was concluded to have more research into human factors to safeguard proper control over automated vehicles.

Meynen’s article (Meynen 2019) provides an analysis of the ethical risks that are correlated to the potential deployment of neurotechnology that might be able, currently or in the foreseeable futur...

The future adoption of automated vehicles poses many challenges, with one of the more important being the preservation of control over vehicles that are no longer (fully) operated by drivers. There is consensus that vehicles should not perform actions that are unacceptable to humans. In this paper, we introduce the concept of Meaningful Human Control (MHC) as a function of a framework of the Automated Driving System (ADS). This framework is constructed through the core components that make up the ADS, primarily considered within the categories of the vehicle and driver. Identification of these components and the chain of control allow traceability of MHC to be performed, and aids vehicle manufacturers, software developers, other vehicle component designers, and vehicle- and driver licensing authorities to address many challenges related to the design and preservation of human control in automated vehicles. Operationalisation of MHC is discussed in the paper including a suggested approach that should aid understanding and the application of the concept. Four application examples are given and recommendations are made in regard to vehicle design, human machine interaction, transition of control, driver training, vehicle approval, and other topics. The framework and presented concept also allow researchers to identify areas to perform more explicit and relevant research and develop models that can be applied to perform projections of future impacts.

The ethical discussion on automated vehicles (AVs) has for the most part focused on what morality requires in AV collisions which present moral dilemmas. This discussion has been challenged for its failure to address the various kinds of risk and uncertainty which we can expect to arise in AV collisions; and for overlooking certain morally relevant facts which are unique to the context of AVs. We take these criticisms as a starting point and outline four perspectives on what matters for the ethics of AVs: risk and uncertainty, value sensitive design, partiality towards passengers and meaningful human control.

Automated driving systems (ADS) with partial automation are currently available for the consumer. They are potentially beneficial to traffic flow, fuel consumption, and safety, but human behaviour whilst driving with ADS is poorly understood. Human behaviour is currently expected to lead to dangerous circumstances as ADS could place human drivers 'out-of-the-loop' or cause other types of adverse behavioural adaptation. This article introduces the concept of 'meaningful human control'T to better address the challenges raised by ADS, and presents a new framework of human control over ADS by means of literature-based categorisation. Using standards set by European authorities for driver skills and road rules, this framework offers a unique, quantified perspective into the effects of ADS on human behaviour. One main result is a rapid and inconsistent decrease in required skill- and rule-based behaviour mismatching with the increasing amount of required knowledge-based behaviour. Furthermore, the development of higher levels of automation currently requires different human behaviour than feasible, as a mismatch between supply and demand in terms of behaviour arises. Implications, discrepancies and emerging mismatches this framework elicits are discussed, and recommendations towards future design strategies and research opportunities are made to provide a meaningful transition of human control over ADS.