Complex autonomous robots such as autonomous vehicles or robotic guides are critical systems because their failures could have catastrophic and costly consequences on themselves and their immediate environment, including users and bystanders. Moreover, verification and validation of these systems, that includes decisional software, is a difficult and complex task, requiring high expertise. In practice, despite recent advances in formal verification techniques and intensive testing for autonomous vehicles, it is still not possible to guarantee elimination of all residual development faults. Another way to enhance the confidence placed in such software, is to consider tolerance mechanisms with regards to these faults. This article proposes such an approach for temporal planners which are a major class of decisional software components in complex autonomous systems. The proposed fault tolerance mechanisms focus on residual development faults in planning models and heuristics. They use four complementary detection mechanisms to detect planning errors. Recovery from possible errors is achieved using redundant diversified planning models. We present an implementation of the proposed architecture on an existing autonomous robot software architecture. We also describe a validation framework used to evaluate the cost and efficacy of the fault tolerance mechanisms using real robot software on simulated robot hardware, and fault injection in the declarative planning models. In this framework, the proposed fault tolerant mechanisms are shown to greatly improve the system reliability with no significant impact on performance.

Ensuring that safety requirements are respected is a critical issue for the deployment of hazardous and complex reactive systems. We consider a separate safety channel, called a monitor, that is able to partially observe the system and to trigger safety-ensuring actuations. We address the issue of correctly specifying such a monitor with respect to safety and liveness requirements. Two safety requirement synthesis programs are presented and compared. Based on a formal model of the system and its hazards, they compute a monitor behavior that ensures system safety without unduly compromising system liveness. The first program uses the model-checker NuSMV to check safety requirements. These requirements are automatically generated by a branch-and-bound algorithm. Based on a game theory approach, the second program uses the TIGA extension of UPPAAL to synthesize safety requirements, starting from an appropriately reformulated representation of the problem.

Ensuring that safety requirements are respected is a critical issue for the deployment of hazardous and complex reactive systems. We consider a separate safety channel, called a monitor, that is able to partially observe the system and to trigger safety-ensuring actuations. We address the issue of correctly specifying such a monitor with respect to safety and liveness requirements. Two safety requirement synthesis programs are presented and compared. Based on a formal model of the system and its hazards, they compute a monitor behavior that ensures system safety without unduly compromising system liveness. The first program uses the model-checker NuSMV to check safety requirements. These requirements are automatically generated by a branch-and-bound algorithm. Based on a game theory approach, the second program uses the TIGA extension of UPPAAL to synthesize safety requirements, starting from an appropriately reformulated representation of the problem.

Autonomous systems operating in the vicinity of humans are critical in that they potentially harm humans. As the complexity of autonomous system software makes the zero-fault objective hardly attainable, we adopt a fault-tolerance approach. We consider a separate safety channel, called a monitor, that is able to partially observe the system and to trigger safety-ensuring actuations. A systematic process for specifying a safety monitor is presented. Hazards are formally modeled, based on a risk analysis of the monitored system. A model-checker is used to synthesize monitor behavior rules that ensure the safety of the monitored system. Potentially excessive limitation of system functionality due to presence of the safety monitor is addressed through the notion of permissiveness. Tools have been developed to assist the process.

Robotic systems have to cope with various execution environments while guaranteeing safety, and in particular when they interact with humans during rehabilitation tasks. These systems are often critical since their failure can lead to human injury or even death. However, such systems are difficult to validate due to their high complexity and the fact that they operate within complex, variable and uncertain environments (including users), in which it is difficult to foresee all possible system behaviors. Because of the complexity of human-robot interactions, rigorous and systematic approaches are needed to assist the developers in the identification of significant threats and the implementation of efficient protection mechanisms, and in the elaboration of a sound argumentation to justify the level of safety that can be achieved by the system. For threat identification, we propose a method called HAZOP-UML based on a risk analysis technique adapted to system description models, focusing on human-robot interaction models. The output of this step is then injected in a structured safety argumentation using the GSN graphical notation. Those approaches have been successfully applied to the development of a walking assistant robot which is now in clinical validation.

Les reseaux bayesiens sont utilises dans plusieurs domaines pour estimer la probabilite d'un evene- ment non observable a partir des evenements observables. Pour chacun des nœuds du reseau bayesien, une Table de Probabilite est definie et souvent remplie de maniere manuelle. Pour une Table plus complexe, l'outil AgenaRisk permet de la remplir suivant une expression logique ou une loi de probabilite. A travers quatres exemples de reseaux bayesiens, nous etudions le fonctionnement de cet outil grâce a la technique d'ingenierie inverse.

Considering the new threats in medical robotics due to increasing complexity and autonomy, and the absence of dedicated standards, we present in this paper how we carried safety analyses for a rehabilitation robot. We combine several standards and research works for a safe design and to construct a safety case for regulatory bodies. We point out some challenges for standardization and future research.

Recent advances in robotics technologies have opened multiple opportunities for the use of robots to support various activities of our daily life and to interact with humans in different ways. In such contexts, it is crucial to identify potential threats related to physical human-robot interactions and to assess the associated risks that might affect safety and dependability. Because of the complexity of human-robot interactions, rigorous and systematic approaches are needed to assist the developers in: i) the identification of significant threats and the implementation of efficient protection mechanisms to cope with these threats, and ii) the elaboration of a sound argumentation to justify the level of safety that can be achieved by the system. To fulfil these objectives, we believe that risk analysis should be carried out based on system models as soon as possible in the development process and hence provide elements to reason about system safety using a structured argumentation. The risk analysis method HAZOP-UML presented in this paper is a guided method to identify potential occurrences of harm, their causes and their severity. The results from risk analysis are then used as input for safety case construction in which we structure an argument about system safety. This process is illustrated by a case study on a robotized rollator.

Traditionally, software in avionics has been totally separated from open-world software in order to avoid any interaction that could corrupt critical on-board systems. However, new aircraft generations need more interaction with off-board systems to offer extended services, which makes these information flows potentially dangerous. In a previous work, we have proposed the use of virtualiza-tion to ensure dependability of critical applications despite bidi-rectional communication between critical on-board systems and untrusted off-board systems. A comparison mechanism based on execution traces analysis is used to detect discrepancies between replicas supported by diverse virtual machines. We propose to strengthen the comparison mechanism at runtime by the use of an execution model, derived from a static analysis of the java bytecode.

A systematic process for eliciting safety trigger conditions is presented. Starting from a risk analysis of the monitored system, critical transitions to catastrophic system states are identified and handled in order to specify safety margins on them. The conditions for existence of such safety margins are given and an alternative solution is proposed if no safety margin can be defined. The proposed process is illustrated on a robotic rollator.

The progress of artificial intelligence techniques, particularly decisional mechanisms, has allowed reactive systems to become more autonomous. This allows new applications in domains such as service robotics in which failures can lead to human injury or death, or financial loss. To ensure safety of such systems, we propose in this paper a process, based on a HAZOP/UML risk analysis, to elicit safety rules that can be enforced on-line. We present a case study of safety rule elicitation for an assistive robot for strolling and discuss implementation of the safety rules in a practical safety monitor.

We present the AMORES project, which aims to provide an architecture for the provision of privacy preserving and resilient collaborative services in "mobiquitous" (i.e., mobile and ubiquitous) systems. The project is built around three use-cases from the area of public transportation: (1) dynamic carpooling, (2) real-time computation of multimodal transportation itineraries and (3) mobile social networking. Four main research tasks are presented in this paper. The first task deals with use-cases, prototypes and privacy assessment. The second task addresses geo-communication primitives: verified positioning, locanyms and geo-services. The third task deals with privacy-preserving communication means such as anonymous routing and geo-cryptography. Finally, the last task is devoted to collaborative behaviors.

The functional layer of an autonomous system such as a robot is required to carry out multiple real-time control activities in parallel. These activities are launched by asynchronous calls from clients situated at higher layers, so there is a need for the functional layer to provide built-in protection to ensure that it is robust with respect to requests that are issued at instants that are incompatible with its current state and could therefore cause catastrophic system failure. This paper addresses the testing of the robustness provided by such protection mechanisms. A hybrid black-box robustness testing approach is considered by which test cases are generated by random mutation of a valid sequence of requests and test verdicts are obtained by a set of property-based robustness oracles applied to a logged trace of requests and responses. An application of the proposed framework to an experimental planetary explorer robot is described.

Le developpement des systemes decisionnels a permis de rendre les systemes reactifs de plus en plus autonomes et l'emergence de nouvelles applications dans des domaines tels que la robotique de service. En revanche, les defaillances eventuelles dans ces nouvelles applications peuvent avoir des consequences catastrophiques. Afin d'assurer la securite-innocuite de tels systemes, nous proposons dans cet article un pro-cessus de generation des regles de securite verifiables en ligne implementables dans un moniteur de securite independant.

The idea that diverse or dissimilar computations could be used to detect errors can be traced back to Dynosius Lardner's analysis of Babbage's mechanical computers in the early 19th century. In the modern era of electronic computers, diverse redundancy techniques were pioneered in the 1970's by Elmendorf, Randell, Aviz̆ienis and Chen. Since then, the tolerance of design faults has been a very active research topic, which has had practical impact on real critical applications. In this paper, we present a brief history of the topic and then describe two contemporary studies on the application of diversity in the fields of robotics and security.

Recent advances in robotics technologies have opened multiple opportunities for the use of robots to support various activities of our daily life and to interact with humans in different ways. In such contexts, it is crucial to identify potential threats related to physical human-robot interactions and to assess the associated risks that might affect safety and dependability. Because of the complexity of human-robot interactions, rigorous and systematic approaches are needed to assist the developers in: i) the identification of significant threats and the implementation of efficient protection mechanisms to cope with these threats, and ii) the elaboration of a sound argumentation to justify the level of safety that can be achieved by the system. To fulfil these objectives, we believe that risk analysis should be carried out based on system models as soon as possible in the development process and hence provide elements to reason about system safety using a structured argumentation. The risk analysis method HAZOP-UML presented in this paper is a guided method to identify potential occurrences of harm, their causes and their severity. The results from risk analysis are then used as input for safety case construction in which we structure an argument about system safety. This process is illustrated by a case study on a robotized rollator.

The design of computer systems to be embedded in critical real-time applications is a complex task. Such systems must not only guarantee to meet hard real-time deadlines imposed by their physical environment, they must guarantee to do so dependably, despite both physical faults (in hardware) and design faults (in hardware or software). A fault-tolerance approach is mandatory for these guarantees to be commensurate with the safety and reliability requirements of many life- and mission-critical applications. A Generic Fault-Tolerant Architecture for Real-Time Dependable Systems explains the motivations and the results of a collaborative project(*), whose objective was to significantly decrease the lifecycle costs of such fault-tolerant systems. The end-user companies participating in this project currently deploy fault-tolerant systems in critical railway, space and nuclear-propulsion applications. However, these are proprietary systems whose architectures have been tailored to meet domain-specific requirements. This has led to very costly, inflexible, and often hardware-intensive solutions that, by the time they are developed, validated and certified for use in the field, can already be out-of-date in terms of their underlying hardware and software technology. The project thus designed a generic fault-tolerant architecture with two dimensions of redundancy and a third multi-level integrity dimension for accommodating software components of different levels of criticality. The architecture is largely based on commercial off-the-shelf (COTS) components and follows a software-implemented approach so as to minimise the need for special hardware. Using an associated development and validation environment, system developers may configure and validate instances of the architecture that can be shown to meet the very diverse requirements of railway, space, nuclear-propulsion and other critical real-time applications. This book describes the rationale of the generic architecture, the design and validation of its communication, scheduling and fault-tolerance components, and the tools that make up its design and validation environment. The book concludes with a description of three prototype systems that have been developed following the proposed approach. (*) Esprit project No. 20716: GUARDS: a Generic Upgradable Architecture for Real-time Dependable Systems.

Safety is a major concern for autonomous systems that physically interact with humans, such as service robots. However, modeling dynamics of such systems is hard so classical safety analysis methods need to be adapted. In this paper, we propose an approach based on a combination of well-known safety analysis techniques. We propose to describe scenarios of use with the common Unified Modeling Language. Risk analysis is then performed using a Preliminary Hazard Analysis, an adaptation of the HAZOP method and the classical Fault Tree Analysis. This paper explains the overal process and illustrates it through the exemple of the MIRAS projects which aims to develop a robotic strolling assistant that will help disabled persons to stand, sit and walk.