The interactions that occur in human-teaming are inspiring novel aerospace designs aimed at improving how humans and machines, or multiple machines, work together. Unfortunately, current Systems Engineering processes are ill-equipped to handle these complex relationships and are unable to design and assure the safety for these systems. To close part of this gap, this paper introduces a novel system-theoretic analytical process to identify unsafe collaborative control actions. It is part of a broader set of techniques that extend the state-of-the-art in hazard analysis, System Theoretic Process Analysis (STPA), to systematically address collaboration. The method rigorously expresses the different ways multiple commands may be unsafe together. Using Systems Theory, it employs abstraction to manage the combinatorial complexity in enumerating control contributions from multiple collaborating components. An algorithm integrates these concepts into an end-to-end process and is supported by automation to enumerate, refine, prune, and prioritize unsafe combinations of control actions. The output of the method feeds the specification of system requirements to implement safety-guided design starting early in concept development. The process is demonstrated on a manned-unmanned aircraft teaming exemplar and finds new causal factors that were not previously found in a past hazard analysis of the same system.

Verification and validation of increasingly autonomous aviation systems is a major challenge. Traditional techniques for the assurance of high-confidence, safety-critical systems are not equipped to handle the complexity, uncertainty, and lack of predictability inherent in non-deterministic systems. Techniques such as run time monitoring, formal methods, and testing and simulation have been applied to some effect, but it is difficult to properly assess the success of such measures. The authors propose the concept of Assurance Efficacy to address this gap. Assurance Efficacy is seen as a parameter, criteria, or perspective by which to evaluate, identify and explore safety risk mitigation strategies and operational assurance architectures. Validation of the utility of this concept through flight testing is a first step in determining its potential role in assessing the overall safety of complex, increasingly autonomous systems that cannot be fully assured in the design phase.

This paper presents a process used to develop appropriate scenarios and metrics for evaluating the maturity of intelligent contingency management algorithms. A benchmark scenario is a reference point against which something can be measured, compared, or assessed. Creating an accurate benchmark requires considerable research and expertise. The scenario itself is an artificial representation of a real-world event, designed to achieve a set of learning objectives through experiential learning. Designing an effective benchmark simulation scenario requires careful planning, including identification of clear objectives; capability assessment of the algorithm/tool being evaluated; assessment of necessary levels of fidelity; development of a process flow map of events and event interactions; and identification of metrics that map back to objectives. Thus, a benchmark scenarios for contingency management might consist of one or several commonly used functions taken from real world applications, used for evaluation, characterization and performance measurement of a contingency management algorithm. Behavior of the contingency management algorithm under different environmental conditions should then be able to be predicted using a set of benchmark functions. The paper describes the resulting benchmark problem as an illustration of the application of this process.

This work considers the scheduling problem for Urban Air Mobility (UAM) vehicles travelling between origin-destination pairs with both hard and soft trip deadlines. Each route is described by a discrete probability distribution over trip completion times (or delay) and over inter -arrival times of requests (or demand) for the route along with a fixed hard or soft deadline. Soft deadlines carry a cost that is incurred when the deadline is missed. An online, safe scheduler is developed that ensures that hard deadlines are never missed and that average cost of missing soft deadlines is minimized. The system is modelled as a Markov Decision Process (MDP) and safe model based learning is used to find the probabilistic distributions over route delays and demand. Monte Carlo Tree Search (MCTS) Earliest Deadline First (EDF) is used to safely explore the learned models in an online fashion and develop a near-optimal non-preemptive scheduling policy. These results are compared with Value Iteration (VI) and MCTS (Random) scheduling solutions.

The emergence and development of advanced technologies and vehicle types have created a growing demand for new forms of flight operations. These new and increasingly complex operational paradigms, such as Advanced and Urban Air Mobility (AAM/UAM), present regulatory authorities and the aviation community with several design-and-implementation challenges – particularly for highly autonomous vehicles. An overarching and daunting task is to develop protocols that can integrate these operations without compromising safety or disrupting traditional airspace operations. A shift toward a more predictive, autonomous, risk mitigation capability becomes critical to meet this challenge. This paper proposes and evaluates a computationally-efficient path planning approach to perform pre-flight planning and autonomous in-flight re-routing to minimize exposures to selected hazards. In our evaluation, hazards associated with degraded and missing critical GPS navigation data are considered. In this paper, we first present a high-performance computing path planning approach based on an adapted Bellman-Ford algorithm, developed in the CUDA programming language. Using the adapted path planning algorithm, we test this algorithm when encountering issues with GPS quality, and deliver an implementation that can produce flight paths that minimize exposure to risks, while maintaining a low computational burden. In our evaluation, the computation of periodic and aperiodic path updates are evaluated, prioritizing specific events as triggers for updates, based on changes to satellite availability. These critical events can lead to significant exposure to navigational hazards if not dealt with correctly.

The identification, modelling, and analysis of root causes of accidents and incidents dominate conventional safety management approaches. However, the effect of humans’ safety-producing behavior on the overall resilience of the system is often neglected. Additionally, emerging aviation markets are giving rise to concepts of operation, such as urban air mobility and optionally piloted air cargo operations, that are leading to a shift in locus of control between humans and automation. Without an understanding of the human contribution to safety, it is difficult to assess the effects of these novel role allocations on overall system safety. In this work, safety-producing behaviors are identified and abstracted into resilient performance strategies. Production rules that encapsulate these strategies are then generated and classified in the Soar cognitive architecture. The strategies are then applied to a remotely-operated air cargo example to demonstrate how safe learning is facilitated. The learned rules and strategies are then formally verified.

This paper describes a general "vision" concept of operations (ConOps) for intermediate-state, passenger-carrying urban air mobility (UAM) missions that has been developed jointly by NASA and Deloitte with input from stakeholders in the UAM ecosystem. This vision ConOps provides a broad overview of some of the high-level requirements for realizing the simultaneous operation of hundreds of aircraft over a single metropolitan area in a wide range of weather conditions as conceptualized by many visionaries at the time of its publication. The concepts contained in this vision ConOps are intended to provide a starting point for further discussions and investigations into how UAM operations can be best enabled. Consequently, we also describe some of the areas where additional research is required before a detailed baseline ConOps can be finalized.

This paper illustrates a proof of concept in coupling decision-making with traditional control algorithms for aerospace applications. Autonomy software sometimes requires augmenting conventional control algorithms with arbitrary logic code, rendering stability analysis difficult or impossible. With rigorous convergence results elusive, an alternative is to improve decision-making to the extent possible to maintain the aircraft's overall safety. To illustrate, we introduce a pitch control system that uses a system monitor (cognition) to cluster modules of similar intent to decide on a malfunctioning module's invalidity based on the pilot's intent.

The advent of third aviation revolution that is seeking to enable transportation where users have access to immediate and flexible air travel. The users dictate trip origin, destination and timing. One of the major components of this vision is urban air mobility (UAM) for the masses. UAM means a safe and efficient system for vehicles to move passengers and cargo within a city. In order to reach UAM’s full market potential the vehicle will have to be autonomous. One of the primary challenges of autonomous flight is dealing with off-nominal events, both common and unforeseen; thus, intelligent contingency management (ICM) is one of the enabling technologies. In this context, the vehicle has to be aware of its internal state and external environment at all times, ascertain its capability and make decisions about mission completion or modification. All of these functions require data to model and assess the environment and then take actions based on these models. Necessarily, there is uncertainty associated with the data and the models generated from it. Since we are dealing with safety-critical systems, one of the main challenges of ICM is to generate sufficient data and to minimize its uncertainty to enable practical and safe decision making. We propose an overall architecture that incorporates deterministic and learning algorithm together to assess vehicle capabilities, project these into the future and make decision on mission management level. A layered approach allows for mature parts and technologies to be integrated into early highly automated vehicle before the final state of autonomy is reached.

Critical software systems responsible for emergency management integrate advanced intelligent algorithms to successfully handle contingencies. These algorithms provide a diverse set of advanced functionalities that were previously performed by humans. Thus, to gain confidence in these systems it emphasizes the need to guarantee safety as these algorithms are executed. Additionally, multiple such algorithms might need to be executed sequentially or concurrently to accomplish a mission, or provide guidance for safety critical operations. The selection among the functionalities at runtime is a complex and critical activity that needs to be systematically designed and analyzed before deployment for real time applications. Towards this end, we discuss a formal architectural approach to address the problem, by designing the architecture with the abstract model of the complex algorithms, then identifying the key performance parameters of the complex algorithms, followed by formal verification of the composition of these algorithms with the assignment of formal contracts based on assumptions and guarantees that represent performance boundaries for the system. We then demonstrate our compositional approach in assuring behavior of an autonomous aerial system with advanced algorithms to handle critical emerging situations.

Urban Air Mobility (UAM) is the name chosen for the concept of using the airspace over urban areas to satisfy the public imperative for fast and efficient transportation within growing metropolitan areas. This Operational Concept (OpsCon) describes a community vision of the projected evolution of urban air service with vehicles capable of carrying one or more passengers.

The national airspace system is exceptionally safe, specifically in terms of commercial air traffic operations. The introduction of innovative aircraft (such as electric vertical takeoff and landing vehicles) undergoing novel operations (such as for an urban air passenger carrying mission) is a potential disruptor to the current means of regulating and ensuring the safety of air travel. The aviation industry and associated regulatory bodies are adapting their approaches to assuring system safety to enable these new paradigms. However, any system safety analysis requires that a hazard assessment be performed. In this work, we consider the Functional Hazard Assessment (FHA) and Systems Theoretic Process Analysis (STPA) techniques for hazard assessment and evaluate whether they can be used in a complementary fashion for regulatory approval purposes. We perform an FHA and an STPA on an electric vertical takeoff and landing (eVTOL) vehicle undergoing an urban air mobility (UAM) passenger carrying reference scenario and present excerpts of this analysis. We then draw parallels between the techniques and highlight elements where they naturally reinforce to each other's results, specifically in the consideration of hazard severity with respect to flight phases, the design of hazard mitigations, and the applicability of the results to all types of regulatory approvals (e.g., type certification, operational approval, and crew training).

Urban Air Mobility (UAM) - defined as safe and efficient air traffic operations in a metropolitan area for manned aircraft and unmanned aircraft systems - is being researched and developed by industry, academia, and government. Significant resources have been invested toward cultivating an ecosystem for Urban Air Mobility that includes manufacturers of electric vertical takeoff and landing aircraft, builders of takeoff and landing areas, and researchers of the airspace integration concepts, technologies, and procedures needed to conduct Urban Air Mobility operations safely and efficiently alongside other airspace users. This paper provides high-level descriptions of both emergent and early expanded operational concepts for Urban Air Mobility that NASA is developing. The scope of this work is defined in terms of missions, aircraft, airspace, and hazards. Past and current Urban Air Mobility operations are also reviewed, and the considerations for the data exchange architecture and communication, navigation, and surveillance requirements are also discussed. This paper will serve as a starting point to develop a framework for NASA's Urban Air Mobility airspace integration research and development efforts with partners and stakeholders that could include fast-time simulations, human-in-the-loop (HITL) simulations, and flight demonstrations.

Developing trust in intelligent agents requires understanding the full capabilities of the agent, including the boundaries beyond which the agent is not designed to operate. This paper focuses on applying formal verification methods to identify these boundary conditions in order to ensure the proper design for the effective operation of the human-agent team. The approach involves creating an executable specification of the human-machine interaction in a cognitive architecture, which incorporates the expression of learning behavior. The model is then translated into a formal language, where verification and validation activities can occur in an automated fashion. We illustrate our approach through the design of an intelligent copilot that teams with a human in a takeoff operation, while a contingency scenario involving an engine-out is potentially executed. The formal verification and counterexample generation enables increased confidence in the designed procedures and behavior of the intelligent copilot system.

A whole host of applications for the use of small Unmanned Aerial Systems (UAS) have been enabled by new regulation for this class of system. However, much work remains to establish appropriate regulatory infrastructure that allows UAS to operate for commerce over populated areas. The magnitude of that effort is compounded by the wide-ranging variety of UAS types and possible Concepts of Operation (ConOps), as well as the diversity in quality and provenance of UAS components. This paper introduces a case study to investigate hazards posed by a small unmanned rotorcraft operating in proximity to a suburban environment for the purpose of package delivery. A potential mitigation to some of these hazards involves the use of an assured containment system, which is different from a common geofencing application. The proposed assured containment system is expected to reduce the risk posed by the operation, which may then be conducted in compliance with the parameters for operation outlined in 14 CFR Part 107, Small Unmanned Aircraft Systems [1].

There is a current drive towards enabling the deployment of increasingly autonomous systems in the National Airspace System (NAS). However, shifting the traditional roles and responsibilities between humans and automation for safety critical tasks must be managed carefully, otherwise the current emergent safety properties of the NAS may be disrupted. In this paper, a verification activity to assess the emergent safety properties of a clearly defined, safety critical, operational scenario that possesses tasks that can be fluidly allocated between human and automated agents is conducted. Task allocation role sets were proposed for a human-automation team performing a contingency maneuver in a reduced crew context. A safety critical contingency procedure (engine out on takeoff) was modeled in the Soar cognitive architecture, then translated into the Hybrid Input Output formalism. Verification activities were then performed to determine whether or not the safety properties held over the increasingly autonomous system. The verification activities lead to the development of several key insights regarding the implicit assumptions on agent capability. It subsequently illustrated the usefulness of task annotations associated with specialized requirements (e.g., communication, timing etc.), and demonstrated the feasibility of this approach.

This paper discusses results from a recent study that investigates certification requirements for an unmanned rotorcraft performing agricultural application operations. The process of determining appropriate requirements using a risk-centric approach revealed a number of challenges that could impact larger UAS standardization efforts. Fundamental challenges include selecting the correct level of abstraction for requirements to permit design flexibility, transforming human-centric operational requirements to aircraft airworthiness requirements, and assessing all hazards associated with the operation.

While incremental steps are being taken to integrate unmanned aircraft systems (UAS) into the various national airspace systems, much work remains to establish appropriate regulatory infrastructure that allows UAS larger than 55 lb to operate for commerce or hire. The magnitude of that effort is compounded by the wide-ranging variety of UAS types and possible applications, as well as the diversity in quality and provenance of UAS components. The FAA has suggested developing design standards tailored to specific applications and operating environments as an approach to facilitate integration and safe operation of some UAS. This paper introduces a case study to investigate design standards for a midsize unmanned rotorcraft operating in a rural environment. A key aspect of this study is the concept of using a certifiable containment system, different from a conventional geofencing application, to ensure that the unmanned aircraft does not escape its intended operational area. The proposed assured containment system is expected to reduce the effort needed to regulate some UAS that could not currently meet rigorous aircraft design standards and fall outside of the parameters for operation outlined in the proposed small UAS rule. This paper discusses how assured containment may be a useful approach to limiting risk and reducing an otherwise prohibitive certification burden to enable UAS operations in confined areas. The case study examines the potential effect the assured containment approach might have on airworthiness certification requirements.