To increase the autonomy of future air taxis, machine learning is necessary for a lot of areas such as vision-based tasks. However, the safety aspect of any machine learning application is of significant concern for users, experts, and certification authorities. To mitigate the risk to passengers or people on the ground, any machine learning-enabled component requires demonstration of rigorous compliance to safety and development assurance standards. Standardization organizations and authorities are currently developing and establishing new guidelines for the safe use of machine learning applications in the aviation domain. This work showcases the concept of runtime monitoring for enabling the safe integration of an example machine learning application in the urban air mobility context: the detection of humans during a landing approach of an air taxi via an onboard camera. Such an application may be useful in the context of autonomous landing to ensure that no person on the ground is endangered. In particular, the concept of operational design domain monitoring is discussed in the context of the recent European Union Aviation Safety Agency guidance and demonstrated in flight testing. The operational design domain monitor is composed of several sub-monitors that supervise different parameters in the operational domain and detect out-of-distribution inputs. Through the development of this component, this work further extends our work on safe operation monitor and runtime assurance for machine learning applications. The flight test results indicate that monitoring the operational design domain can support performance as well as the safety of the operation.

Maintenance of offshore wind farms requires the transportation of tools and spare parts in close coordination with the deployment of technicians and the cost-intensive shutdown of the wind turbines. In addition to ships and helicopters, drones are envisioned to support the offshore transportation system in the future. For cost-efficient and scalable offshore drone operations, autonomy is key to minimize the required infrastructure and personnel. In this work, we present a system architecture that integrates the key onboard capabilities for autonomous offshore drone operations: onboard mission and contingency management, en-route trajectory planning, robust flight control, safe landing, communication management, and runtime monitoring. We also present technical solutions for each of these capabilities and discuss their integration and interaction within the autonomy architecture. Furthermore, remaining challenges and the feasibility of autonomous drone operations for offshore wind farm cargo delivery are addressed, contributing to the realization of this vision in the near future. The work presented here summarizes the results of autonomous cargo drone operations within the UDW research project, a joint project between the German Aerospace Center (DLR) and the energy supplier EnBW.

A key necessity for the safe and autonomous flight of Unmanned Aircraft Systems (UAS) is their reliable perception of the environment, for example, to assess the safety of a landing site. For visual perception, Machine Learning (ML) provides state-of-the-art results in terms of performance, but the path to aviation certification has yet to be determined as current regulation and standard documents are not applicable to ML-based components due to their data-defined properties. However, the European Union Aviation Safety Agency (EASA) published the first usable guidance documents that take ML-specific challenges, such as data management and learning assurance, into account. In this paper, an important concept in this context is addressed, namely the Operational Design Domain (ODD) that defines the limitations under which a given ML-based system is designed to operate and function correctly. We investigated whether synthetic data can be used to complement a real-world training dataset which does not cover the whole ODD of an ML-based system component for visual object detection. The use-case in focus is the detection of humans on the ground to assess the safety of landing sites. Synthetic data are generated using the methods proposed in the EASA documents, namely augmentations, stitching and simulation environments. These data are used to augment a real-world dataset to increase ODD coverage during the training of Faster R-CNN object detection models. Our results give insights into the generation techniques and usefulness of synthetic data in the context of increasing ODD coverage. They indicate that the different types of synthetic images vary in their suitability but that augmentations seem to be particularly promising when there is not enough real-world data to cover the whole ODD. By doing so, our results contribute towards the adoption of ML technology in aviation and the reduction of data requirements for ML perception systems.

In the evolving landscape of robotic applications, there is an increasing demand for advanced path planning algorithms capable of navigating dynamic environments. Estab-lished sampling-based algorithms like Probabilistic Roadmaps (PRM/PRM*) or Rapidly-exploring Random Trees (RRT/RRT*) and their derivatives have marked significant progress in this domain. However, there remains room for improvements when it comes to effectively planning cost-optimal paths through a priori known time-varying environments without resorting to online replanning. Addressing limitations of related algorithms and extending the range of considered dynamic factors, we propose Time-Aware Probabilistic Roadmaps (TA-PRM*), combining PRM with the concept of time-dependent graphs. Our approach distinctively incorporates dynamic weighting into PRM and introduces a temporal dimension to the A* algorithm, while preserving both asymptotic optimality and probabilistic completeness properties. We also propose techniques for pruning in the temporal dimension and heuristic tuning to improve runtime efficiency, while slightly compromising optimality. Through comprehensive evaluations, we benchmark TA-PRM*'s efficacy against that of its best-known contemporaries and demonstrate its applicability in a real-world scenario for unmanned aerial vehicle trajectory planning.

With the introduction and implementation of the new EU drone regulations over the past years, the quantification of the operation-specific ground risk based on population density maps has been established as a key step in the approval process for drone operations. Currently, census data is used as the main data source for the ground risk assessment. However, census maps are static and of limited spatial resolution. Also, they fail to capture aspects of mobility, commuting patterns, seasonal changes and events. In this work, we propose the use of time-annotated population density maps obtained from mobile devices for planning minimum ground risk trajectories. We outline an end-to-end approach to compute minimum risk trajectories including data processing, planning algorithms and software implementation details. To achieve planning capabilities with response times of only a few seconds on a single CPU core, the planning framework is based on the quasi-random generation of cyclic search graphs also known as probabilistic roadmaps. We present results from a set of example trajectory planning scenarios which indicate the feasibility and usefulness of the algorithmic approach in combination with time-annotated population density maps. Furthermore, we discuss how the presented approach could be used to increase safety of future drone operations.

In order to respond effectively in the aftermath of a disaster, emergency services and relief organizations rely on timely and accurate information about the affected areas. Remote sensing has the potential to significantly reduce the time and effort required to collect such information by enabling a rapid survey of large areas. To achieve this, the main challenge is the automatic extraction of relevant information from remotely sensed data. In this work, we show how the combination of drone-based data with deep learning methods enables automated and large-scale situation assessment. In addition, we demonstrate the integration of onboard image processing techniques for the deployment of autonomous drone-based aid delivery. The results show the feasibility of a rapid and large-scale image analysis in the field, and that onboard image processing can increase the safety of drone-based aid deliveries.

View Video Presentation: https://doi.org/10.2514/6.2023-1124.vid The introduction of machine learning in the aviation domain is an ongoing process. This is also true for safety-critical domains, especially for the area of Urban Air Mobility. A significant growth in number of air taxis and an increasing level of autonomy is to be expected allowing for operating a large number of air taxis in complex urban environments. Due to the complexity of the tasks and the environment, key autonomy functions will be realized using machine learning, for example the camera-based detection of objects. However, the safety assurance for avionics systems using machine learning components is challenging. This work investigates safety and verification aspects of machine learning components. A camera-based detection of humans on the ground, e.g. to assess a potential landing area, serves as an example for an machine learning-based autonomy functio and was integrated into an Unmanned Aircraft. In the context of this exemplary machine learning component, the concept of Operational Design Domain as recently adapted European Aviation Safety Agency in the context of machine learning assurance is described along with other key concepts of machine learning assurance. Furthermore, runtime assurance is used to monitor conformance to the Operational Design Domain during flight. The presented flight test results indicate that monitoring the Operational Design Domain can support performance as well as the safety of the operation.

: Machine learning has become one of the most widely used techniques in artificial intelligence, especially for image processing. One of the biggest challenges in developing an accurate image processing model is to collect large amounts of data that are sufficiently close to the real-world scenario. Ideally, real-world data is therefore used for model training. Unfortunately, real-world data is often insufficiently available and expensive to generate. Therefore, models are trained using synthetic data. However, there is no standardized method of how training data is generated and which properties determine the data quality. In this paper, we present first steps towards the generation of large amounts of data for human detection based on aerial images. To create labeled aerial images, we are using Unreal Engine and AirSim. We report on first impressions of the generated labeled aerial images and identify future challenges – current simulation tools can be used to create realistic and diverse images including labeling, but native support would be beneficial to ease their usage.

View Video Presentation: https://doi.org/10.2514/6.2022-1133.vid Machine learning (ML) has proven to be the tool of choice for achieving human-like or even super-human performance with automation on specific tasks. As a result, this data-driven approach is currently experiencing massive interest in all industry domains. This increased use also applies for the safety critical aviation domain. With no human pilot on board, the potential use cases of ML for unmanned aircraft are particularly promising. Even upcoming Urban Air Mobility (UAM) concepts are planning to remove the onboard pilot and instead use ML to support a remote pilot, possibly supervising a fleet of vehicles. However, the verification of ML algorithms is a challenging problem, since established safety standards and assurance methods are not applicable. Thus, this work comprises a literature study on the topic of ML verification and safety. This research paper uses a systematic approach to map and categorize the research and focus on specific subtopics that are of particular interest in the context of existing guidance documents.

The communication protocol used to command and control (C2) unmanned aircraft from the ground is vital in regards to the overall system's architecture and capabilities. It directly affects the degree of situational awareness available on the ground through telemetry and the capabilities to remotely guide and control the unmanned aircraft. Consequently, the C2 link protocol may be a limiting factor in regards to system capabilities and autonomy and and may also directly or indirectly impact the design of onboard systems such as the autopilot. In this work, we analyze and compare two well-known C2 link protocols in regards to these aspects. We specifically focus on the requirements of the protocols concerning guidance, control mode design and mission transfer and highlight certain factors that conceptually limit interoperability between the two C2 link protocols. We discuss advantages and disadvantages of both C2 link protocols and outline potential improvements for both. With this work, we provide our perspective towards the development of future C2 communication protocols and the interoperable design of unmanned aircraft systems with extended autonomous capabilities.

A key challenge for low-altitude unmanned air transportation is to minimize operational risks by all means. Besides many other measures to be considered, the aircraft's trajectory must be planned carefully and optimized as there are inevitable remaining risks which should be minimized when flying over sparsely populated areas. The risk may be mitigated by a safe termination of the flight if circumstances permit. Also, the probability of violating any operational constraints that would lead to a flight termination should be reduced as much as possible. Adequate risk models and efficient algorithmic risk assessment techniques are required to perform such optimizations. Furthermore, the aircraft may have to react to certain events such as high-priority traffic by changing its trajectory online during flight. As command and control (C2) links may have limited reliability, it must be possible to perform trajectory re-planning onboard with limited computational resources. This poses high demands on the runtime efficiency of the planning algorithms. In this work, we present conceptual approaches to risk modeling and assessment based on geospatial datasets and aircraft dynamic models. We further present the design and experimental results of a software framework for onboard and online trajectory planning. Our results demonstrate that risk-based motion planning for unmanned aircraft can be performed with limited onboard computational resources allowing for safe autonomous flight.

With increasing onboard intelligence and planning capabilities, autonomous unmanned aircraft system (UAS) can execute a wide spectrum of tasks ranging from low-level flight tasks to complex abstract mission tasks. Regardless of their onboard intelligence, most state-of-the-art UAS are commanded with waypoint-based command and control (C2) interfaces. However, waypoint-based interfaces are not well suited to represent mission tasks at different levels of abstraction. In this work, we address the problem of designing a C2 interface for autonomous UAS at a conceptual level. We propose a C2 interface that allows operating UAS at different levels of autonomy using task decomposition and a hierarchical data format for transmission. This interface is designed to enable transparent and consistent communication of mission tasks between the ground control station and the onboard systems. We demonstrate the application of the interface for two different UAS mission scenarios in detail. The proposed interface is designed to supersede waypoint-based C2 interfaces for the next generation of UAS providing high-level autonomy through onboard intelligence.

The design and development of unmanned aircraft is a complex multi-disciplinary task. In the context of risk-based and operation-centric assurance and certification, operational constraints play a key role. Consequently, technologies that impact operational risks such as flight termination, command-and-control links, and onboard autonomy have to be addressed holistically and within the context of the operation. A scenario simulation combines these various aspects originating from different disciplines into one versatile tool. It allows making informed design trade-offs and validating operational requirements at early development stages. In this work, we give insight into the scenario simulation framework developed within the research project ALAADy (Automated Low-Altitude Air Delivery) which addresses unmanned freight operations at low-altitudes within the context of operation-centric certification as introduced by EASA with the so-called specific category. We describe use-cases and requirements derived from research objectives. Furthermore, we present the simulation architecture as well as details on implementation and modelling. Results from an exemplary simulation study regarding safe flight termination in the event of an operational constraint violation are shown to demonstrate the applicability and usefulness of the simulation framework as a tool for holistic unmanned aircraft design.

We present the architecture, implementation and benchmark results of a minimum-risk path planning framework targeted towards long-range unmanned flights at low altitudes. Application scenarios, such as transportation, remote sensing or surveillance missions, often require long-range flights beyond the visual line-of-sight of a pilot on the ground. Autonomy is a key enabler for such operations as it decreases the dependency on command-and-control links and human interaction. An essential technology for autonomous unmanned aircraft is onboard and online path planning. It requires highly efficient planning algorithms to be integrated in onboard systems with limited computational resources. In this work, we demonstrate how semantic geospatial datasets and sampling-based planning algorithms can be used to calculate long-range and low-altitude flight paths within seconds. We present benchmark results from a set of mission scenarios of realistic complexity and extent. The results indicate that in-flight and onboard path planning for autonomous aircraft can be realized through effective pre-processing of input data and careful design, integration and tuning of planning algorithms.

In real-world applications, unmanned aircraft may be subject to short-term configuration changes such as new payloads, additional batteries or antennas. Furthermore, degradation of materials, system failures, icing and weather conditions may impact performance. As a consequence, performance parameters may often be unknown or subject to uncertainty. To handle such performance uncertainties within an autonomous system, the system must exhibit a high degree of self-awareness and be able to react and adapt to false assumptions and inaccurate parameter values. Within this work, an adaptive path planning approach based on in-flight estimation of kinematic model parameters is presented. In order to achieve this, the parameters of the kinematic model used for path planning are continuously monitored and the flight path is re-planned correspondingly in order to fully utilize the aircraft's turn and climbing performance. At the same time the path tracking error and required safety distance along the flight path are minimized. Simulation results for multiple scenarios indicate that the approach improves performance and safety in the presence of uncertain or changing kinematic model parameters.

On-board path planning is a key capability for safe autonomous unmanned flight. Recently, it has been shown that using trochoids for turn segments allows for run-time efficient path planning under consideration of the prevailing wind. However, with varying wind conditions and uncertainty in the estimation of wind speed and direction, paths optimized for a reference wind condition may become infeasible to track for the aircraft. In this work, we discuss how to calculate conservative turn rate limits and we present a novel approach to calculate safety distances along trochoidal turn segments in order to account for an unknown wind speed and airspeed component of bounded magnitude. This allows to plan flight paths that are optimized for the currently expected wind condition but are still safe in case the aircraft experiences different wind conditions. We present results from simulation and flight tests which demonstrate the impact of uncertain and varying wind conditions on the tracking performance of paths with circular and trochoidal turn segments. The results show that trochoids can be used to reduce path tracking errors even if the prevailing wind changes significantly. Furthermore, the proposed method to calculate safety distances conservatively over-approximates path deviations in all considered cases. Thus, it can be used to plan safe paths in the presence of uncertain wind conditions without solely relying on conservative performance limits.

We present a runtime efficient approach to sampling-based path planning for fixed-wing unmanned aerial vehicles (UAV) based on line-graphs. Accounting for flight performance limits and the local prevailing wind, path planning is computationally expensive especially in 3D obstacle environments. A common approach is to solve the problem successively, i.e. to plan collision free paths, which are then transformed into feasible paths. However, this may compromise planning completeness if path smoothing fails. We show that line-graphs based on 3D probabilistic roadmaps can be used to effectively decouple the planning problem. The roadmap serves as a persistent free space representation of the environment and the corresponding line-graph is used to incorporate kinematic constraints to respect the fixed-wing flight performance limits in wind. Applying the A* graph-search on the line-graph instead of the 3D roadmap allows to efficiently find paths that respect these kinematic constraints without relying on path smoothing. Our results show that the presented approach can be used for near-realtime multi-query planning with varying wind conditions and flight performance constraints.

High altitude platforms, also known as pseudo-satellites, are envisioned as unmanned aircraft flying at altitudes above 15 km to provide observation, remote sensing or communication services. A challenging yet recurring requirement for such aicraft is to be able to perform long-endurance missions over multiple days or even weeks. A sophisticated onboard energy management system including solar panels and batteries is needed to achieve this. The energy balance of such an aircraft depends on many factors which should be considered in the design process. In this work, we propose a systematic approach to evaluate the energy balance of high altitude platform designs for specific mission scenarios. This approach can be employed at very early design stages and incrementally extended to follow the design process. We demonstrate the methodology with an exemplary parameter study for a generic fixed-wing aircraft. In particular, the impact and correlations of the mission latitude, wind conditions, flight trajectory optimization, sizing of the platform and solar panel coverage of the main wing was evaluated. A key result is that battery mass can be reduced significantly, especially for missions at high latitudes, by optimizing the holding pattern that is flown throughout the mission or by increasing the solar panel coverage. Generally, the results indicate that the method allows to evaluate mission-specific effects on the energy balance of high altitude platforms.

Onboard and online flight path planning for small-scale unmanned rotorcraft requires efficient algorithms in order to meet real-time constraints. In a priori unknown environment, rapid replanning is necessary in order to maintain a safe clearance when new obstacles are detected. The complexity of the planning problem varies vastly with the required flight path qualities and the complexity of the environment. In most scenarios flight paths should be smooth and time-efficient and always feasible to fly. Therefore, the rotorcraft's flight dynamics must be accounted for. Decoupled planning approaches have been proven to solve this problem very efficiently by dividing the problem into sequentially solvable subproblems. However, this computational efficiency comes at the cost of having to compromise on the flight path quality. In this work, we present a decoupled planning approach that has been integrated with our midiARTIS helicopter in order to perform onboard path planning when flying through a priori unknown environment. The approach involves roadmap-based global path planning and local path refinement with cubic splines. It allows to plan safe, dynamically feasible and time-efficient flight paths with limited onboard processing power. We present simulation results from a set of benchmark scenarios in complex urban terrain as well as results from flight testing of a closed-loop obstacle avoidance maneuver with virtual obstacle mapping. Our results demonstrate that close-to-optimal flight paths can be planned with a decoupled planning approach, if heuristics and simplifications for each planning step are carefully chosen.