This paper addresses the positioning quality of Simultaneous Localization And Mapping (SLAM) based on Light Detection and Ranging (LiDAR) sensors within urban road traffic. Based on the assumption of functional capability of existing SLAM implementations, the paper evaluates specific details of urban car drives that arise when SLAM is to be used for automatic car control. In the presented case, LiDAR-based positioning is done with the Google Cartographer software which generates real-time updates that are compared to GNSS reference. The evaluation is done by using own Light Detection And Ranging (LiDAR) sensor recordings from urban driving. Next to the overall GNSS-free path estimation, the paper zooms into some typical situations (e.g. waiting at busy intersection, driving curves) where SLAM might be inaccurate.

This paper is about automated condition monitoring of critical railway infrastructure using unmanned aircraft systems as flying sensors. As far as possible, automation shall include flight guidance and management as well as automated processing of large sensor data sets. Since a commercial solution must consider the regulatory framework on remotely piloted aircraft systems, the paper discusses legal issues to make allowance for flights beyond visual line of sight. The work described here is focused on Europe and Germany, however, the major principles are likely to be adaptable to other countries. Next to that, the paper presents a strategy for automated image and video data processing. It consists of a super-resolution approach where onboard video camera data from typical offthe-shelf drones can replace higher-resolution still imagery and thus avoid the necessity to use special flight systems, and a deeplearning approach where specific elements are to be detected in the images. With data from flight tests over railway overhead lines, the paper shows an automated detection of rod insulators. Moreover, it presents resolution improvements from video data so that off-the-shelf camera drones can be qualified for the detection of small defects.

After the TransAID scenarios have been identified in D2.2, and after first simulations of the TransAID ideas have taken place, this deliverable describes the steps taken to get from the scenario definition to system architectures of the connected and automated vehicles and the road side cooperating with them. In detail, the scenarios are discussed and requirements are extracted. The requirements lead to the creation of the system architecture which is described in an abstract way. Finally, all potential vehicles, test sites and mobile infrastructure components are presented. While this deliverable sketches the work to be done in WP7 in each project iteration, the exact setup and the results of the feasibility assessment are discussed in detail later on in D7.2.

The objective of the TransAID (Transition Areas for Infrastructure-Assisted Driving) project is to deal with situations that cooperative and automated vehicles (CAV) might face when they are approaching to traffic conditions or zones that their automated systems are not able to handle by themselves. In those cases, the driver will be required to take control of the vehicle; this is the socalled Transition of Control (ToC). TransAID develops and demonstrates traffic management procedures and protocols to increase the overall traffic safety and efficiency specially at transition areas (i.e. zones where ToCs should take place) considering the coexistence of CAVs, autonomous vehicles (AVs), cooperative vehicles (CVs) and legacy vehicles (LV). TransAID measures require the use of communications between vehicles (V2V), and between vehicles and the road infrastructure (V2I) which are mainly used to gather information about the traffic stream through cooperative sensing and to support in the coordination of the vehicles maneuvers through cooperative maneuvers. In this context, this document shows the sensor devices and techniques to fuse their data that are being developed in TransAID. This includes techniques implemented at camera-equipped infrastructures that are able to detect, create bounding boxes and uniquely track objects using optical flow, and at the vehicle employing a hybrid sensor fusion strategy which contains a low-level LIDAR fusion module, that transforms the sensor data of multiple laser scanners into a common coordinate system, and an object-level fusion module, that fuses in-vehicle sensor data with data coming from neighbouring vehicles. The document also shows the cooperative techniques that are being designed to enable the Collective Perception Service (CPS) in line with ETSI. The ETSI's CPS entails the continuous exchange of Collective Perception Messages (CPM) that include a logic representation of the objects detected by the sensors and which are useful to improve the vehicles' and the infrastructure's perception of the driving environment. A key aspect for the efficient execution of the CPS is the definition of appropriate generation rules for the transmission of the CPMs, i.e. how often they are transmitted and what information do they include. This document includes a comprehensive analysis of the effect on the communications performance and information awareness of different CPM generation rules that are being considered in ETSI. In particular, the CPM generation rules follow a periodic policy (at 10Hz or 2Hz) where all detected objects are included, or a dynamic one where only the objects fulfilling some requirements are included. The conducted analysis has shown that there is a trade-off between perception capabilities and communications performance/scalability: vehicles detecting the same object(s) and including them in their CPMs create redundant detection which can help improve the perception capabilities but generate higher channel load levels and therefore impact the performance of V2X networks. The obtained results show that the ETSI's dynamic generation policy significantly reduces the communications channel load compared with the periodic ones, without compromising the perception capabilities. In the framework of TransAID, advanced policies will be proposed to further optimize the CPM, both its content and transmission triggering conditions, in order to achieve the necessary levels of redundancy and minimize the impact of the implementation of CPM in the stability and scalability of future V2X networks. In addition, this document investigates existing cooperative driving mechanisms, and specially the ETSI approach on manoeuvre coordination. The ETSI's Manoeuvre Coordination Service (MCS) is defining new concepts and messages which can be used to coordinate manoeuvres between vehicles. TransAID is actively participating in this process, e.g., by means of the definition of the Manoeuvre Coordination Message (MCM) and extending the role of the infrastructure to support the vehicles' manoeuvres coordination under certain scenarios and conditions. In this context, this document presents the message flow for the set of services that are being considered in TransAID. First, this document has analysed the traffic management measures defined by the different services of the TransAID project, and the required message flow for each service has been defined. Each message flow describes how, when, and where the vehicles communicate between them, and between them and the infrastructure, to execute the traffic management measures. Then, this document provides a preliminary analysis of the MCM generation rules. As highlighted for the CPMs, MCM messages should be transmitted with a frequency high enough to guarantee that the vehicles' manoeuvre coordination is possible. However, a too frequent exchange of MCM messages can increase the channel load to the point that it can negatively impact the performance and scalability of the V2X network. The conducted analysis has shown the importance of considering the vehicular context for the generation of the MCM messages in order to achieve a good balance between channel load and reliability for a safe execution of the cooperative manoeuvres.

This paper presents an architecture for Automated Valet Parking (AVP) connected to cloud-based IoT services and mobile user interfaces. The goal is to enable AVP services for automatic vehicles. From the user perspective, automatic car drop-off and pick-up are activated via smart phone application, and the user will be able to continuously monitor the vehicle status together with additional services as cleaning or recharge during the parking phase. Further, the IoT platform allows the integration of live services that will interact with automatic driving and parking. As an example, the presented AVP setup includes the operation of service drones to automatically guide a vehicle to the best parking spot. The demonstration in this paper comprises a parking car and a micro aerial vehicle (MAV) connected in real-time through the IoT platform as well as the smart phone application where the car is controlled and supervised.

In the near future Automated Vehicles (AVs) will be part of the vehicular traffic on the roads. Normally, all automation levels will be granted on the road based on the different road situations, but challenging situations will still exist that AVs will not be able to handle safely and efficiently. AVs driving at a high automation level may step down to the lower automation level and handover the partial/full control to the driver when the automation system reaches its functional system limits or encounters unexpected situations. This paper briefly explains the H2020 TransAID project covering the transition phases between different levels of automation. It will review related work and introduce the concept to investigate automation level changes. Furthermore, the collective sensor data processing architecture using for demonstrators and the selected use cases are presented.

Since 2010 the German Aerospace Center is working on the project Autonomous Terrain-based Optical Navigation (ATON). Its objective is the development of technologies which allow autonomous navigation of spacecraft in orbit around and during landing on celestial bodies like the Moon, planets, asteroids and comets. The project developed different image processing techniques and optical navigation methods as well as sensor data fusion. The setup—which is applicable to many exploration missions—consists of an inertial measurement unit, a laser altimeter, a star tracker and one or multiple navigation cameras. In the past years, several milestones have been achieved. It started with the setup of a simulation environment including the detailed simulation of camera images. This was continued by hardware-in-the-loop tests in the Testbed for Robotic Optical Navigation (TRON) where images were generated by real cameras in a simulated downscaled lunar landing scene. Data were recorded in helicopter flight tests and post-processed in real-time to increase maturity of the algorithms and to optimize the software. Recently, two more milestones have been achieved. In late 2016, the whole navigation system setup was flying on an unmanned helicopter while processing all sensor information onboard in real time. For the latest milestone the navigation system was tested in closed-loop on the unmanned helicopter. For that purpose the ATON navigation system provided the navigation state for the guidance and control of the unmanned helicopter replacing the GPS-based standard navigation system. The paper will give an introduction to the ATON project and its concept. The methods and algorithms of ATON are briefly described. The flight test results of the latest two milestones are presented and discussed.

In this paper, we introduce a fast and lightweight method based on several combined filters to detect and track an object in images recorded by a moving camera. Assuming we know nothing about the intruders shape, color or other geometric appearance, we focus with our work on change detection in the image, caused by movement of the object against the background. The method is evaluated with image data from experimental flights with two unmanned aircraft performing different flight maneuvers. The correctness of the intruder detection is evaluated by comparison with hand labeled ground truth from different sequences of the test flight. Additionally, we evaluate the performance of our implementation on architectures with low computational power with regard to a practical onboard solution for small unmanned aerial vehicels (UAV).

This paper presents an optical navigation method for unmanned flights where satellite navigation might be disturbed. Core is an inertial-based navigation filter that provides high-frequent flight state updates and where satellite positioning can be replaced with updates from optical sensors in case of satellite signal dropouts. This alternative positioning is determined by a simultaneous localization and mapping (SLAM) algorithm that can generally handle 2D and 3D feature inputs from arbitrary sources. In the presented setup, 2D features are generated from camera images where 3D information from laser range is added if available. Within a simulation environment, visual SLAM is fed with emulated inputs. The architecture provides strong separation, i.e. the sensor pre-processing, visual SLAM, state estimation, and flight control modules are exchangeable and can be run and tested independently. This concept may prevent a very tight coupling of all components, but with regard to future certification, validation and verification will be easier once single components are assured. The navigation method is tested in two ways: first within a flight test of an 85-kg helicopter where only the quality of optical-aided state estimation is tested, and second within a closed-loop simulation where mutual interactions between navigation and flight control are critical in terms of stability. The tests underline the applicability of the presented approach, making this method ready for automatic camera-based flights.

This paper presents an optical-aided navigation method for automatic flights where satellite navigation might be disturbed. The proposed solution follows common approaches where satellite position updates are replaced with measurements from environment sensors such as a camera, lidar or radar as required. The alternative positioning is determined by a localization and mapping (SLAM) algorithm that handles 2D feature inputs from monocular camera images as well as 3D inputs from camera images that are augmented by range measurements. The method requires neither known landmarks nor a globally flat terrain. Beside the visual SLAM algorithm, the paper describes how to generate 3D feature inputs from lidar and radar sources and how to benefit from both monocular triangulation and 3D features. Regarding state estimation, the approach decouples visual SLAM from the filter updates. This allows software and hardware separation, i.e. visual SLAM computations on powerful hardware while the main filter can be installed on real-time hardware with possible lower capabilities. The localization quality in case of satellite dropouts is tested with data sets from manned and unmanned flights with different sensors while keeping all parameters constant. The tests show the applicability of this method in flat and hilly terrain and with different path lengths from few hundred meters to many kilometers. The relative navigation achieves an accumulation error of 1–6 % of distance traveled depending on the flight scenario. In addition to the flights, the paper discusses flight profile limitations when optical navigation methods are used.

In the context of optical-aided navigation and visual Simultaneous Localization And Mapping (SLAM) for satellite-denied aircraft navigation, this paper extends the monocular SLAM approach by the use of multiple sensors with different viewing directions. Downward optical sensors see other movements than forward-looking cameras, hence it is straightforward to combine the benefits of both. This combination helps to estimate all the six motion components with increased robustness, yielding a more stable and accurate state estimation for optical-aided navigation solutions. The method is evaluated with aerial data from manned and unmanned flights. In the data analysis, a satellite navigation dropout is simulated, and the flight trajectory is then reconstructed just by the optical data. The method is tested in small-scale scenarios as well as in longer flights with several kilometers of flight range. The results show some increased performance of an additional forward camera in comparison to a setup with only downward sensors. It is proposed to use such multi-sensor configurations wherever motion estimation ambiguities with a single camera are probable, especially when larger distances have to be flown with optical navigation.

Dieser Bericht umfasst die Beschreibung und die Ergebnisse der Studie Stabile Navigation und Gelandefolgeflug fur VTOL UAS, welche im Zeitraum 2013-2017 durchgefuhrt wurde. Zielrichtung dieser Studie ist die Untersuchung und Beschreibung von Methoden zur Steigerung der Automation im Bereich der Flugfuhrung von VTOL UAS. Dadurch kann der Operateur entlastet und z.B. in die Lage versetzt werden, mehr Kapazitat auf den Einsatz der UAV-Nutzlast verwenden zu konnen. Neben der Methodenentwicklung erfolgt die Erprobung und Validierung der neuen Verfahren auf einem unbemannten Versuchstrager.

This paper presents a model-based shadow estimation method that aims at identifying self-cast shadows of aerial vehicles in on-board images. The self-cast shadow poses a non-negligible problem in any kind of on-board processing, e.g. remote sensing, visual odometry, or target tracking. Therefore, it often is necessary to exclude the image regions that contain this self-cast shadow. The presented method achieves this exclusion by using data from an INS (Inertial Navigation System) combined with the knowledge of the appearance of the shadow-casting object. This paper will present the self-cast shadow detection algorithm in detail. Further, the algorithm will be tested on flight data that have been recorded by an unmanned helicopter which is operated by the German Aerospace Center. The results show that it is possible to forecast the position of the shadow with an accuracy of over 95 %, thus this method is capable of finding an image region where typical image motion estimation algorithms are likely to fail.

This paper explores the state estimation problem for an autonomous precise landing approach on celestial bodies. As part of the project “Autonomous Terrain-based Optical Navigation” (ATON) of the German Aerospace Center (DLR) this paper describes the central state estimation algorithm. This algorithm combines high rate inertial navigation with low rate sensor fusion. The description includes the software architecture of the developed navigation system and the estimator, which is based on an Unscented Kalman Filter (UKF). The UKF equations are presented as well as the specific transition and observation models. Additionally, different image processing modules, providing the UKF with position updates, are described shortly. Finally, the evaluation of the implemented system based on performed flight tests imitating a landing on the Moon is presented. These tests show that the method is capable of providing a robust navigation solution during the landing approach.

For flight automation tolerable to satellite navigation dropouts, this paper presents a simultaneous localization and mapping method based on radar altimeter measurements and monocular camera images. The novelty within mapping is the combination of radar distance and image triangulation. This approach verifies whether the radar measurement fits to a specific horizontal plane in the map, yielding the sub-set of image features that do most probably correspond with the radar measurement. With this map match of the radar altitude, ambiguities in the radar measurement can be resolved. Since unusable radar measurements are suppressed, this method is suitable for positioning in non-flat terrain, e.g. in mountain areas. For matched data, the method estimates a scale correction factor for the image projection rays in order to remove scale ambiguities of the monocular navigation. Together with mapping, vehicle localization is done which is essentially camera resectioning. Localization can be parameterized with the required number of degrees of freedom depending on the availability of additional position sensors. The incremental positioning is tested in kilometer-scale outdoor flights of a 30 kg unmanned airplane as well as in flights with a Cessna 172R equipped with camera and radar sensors. The tests show the benefits of the proposed method in flat and hilly terrain, and demonstrate reduction of accumulation errors down to 2-6% over the distance flown. Some constraints of the method for the altitude range are existent, however it is highlighted that this method will generally work on typical flight profiles.

This paper explores the state estimation problem for an autonomous precision landing approach on celestial bodies. This is generally based on sensor fusion from inertial and optical sensor data. Independent of the state estimation filter, a remaining problem is the provision of position updates without the use of known absolute support information as it appears when the vehicle navigates within unknown terrain. Visual odometry or simultaneous localization and mapping (SLAM) approaches typically provide relative position. This is quite suitable, but it can be adverse due to error accumulation. The presented method combines monocular camera images with laser distance measurements to allow visual SLAM without errors from increasing scale uncertainty. It is shown that this reduces the accumulated error in comparison to sole monocular visual SLAM. Further, the presented method integrates the matching to known landmarks if they are available in the beginning of a landing approach so that the relative optical navigation can be initialized without systematic errors. Finally, tests with a simulated moon landing are performed and it is shown that the method is capable of navigating down to the ground impact.

This paper presents a visual Simultaneous Localization And Mapping (SLAM) method for temporary satellite dropout navigation for an unpowered fixed-wing aircraft. It is designed for flight altitudes beyond typical stereo ranges, but within the range of distance measurement sensors. The proposed visual SLAM method consists of a common localization step with monocular camera resectioning, and a mapping step which incorporates radar altimeter data for absolute scale estimation. With that, there will be no scale drift of the map and of the estimated flight path. The method does not require simplifications like known landmarks and it is thus suitable for unknown and nearly arbitrary terrain. The method is tested with sensor datasets from a manned Cessna 172 aircraft. With 5% absolute scale error from radar measurements causing approximately 2-6% accumulation error over the flown distance, stable positioning is achieved over several minutes of flight time. The main limitations are flight altitudes above the radar range of 750 m where the monocular method will suffer from scale drift, and, depending on the flight speed, flights below 50 m where image processing gets difficult with a downwards-looking camera due to the high optical flow rates and the low image overlap.

Unbemannte Luftfahrzeuge (UAS) leisten bereits heute wichtige Dienste fur Erkundung und Aufklarung. Am Institut fur Flugsystemtechnik des DLR Braunschweig werden 3D–Sensoren in die Flugplanung und Flugsteuerung integriert und damit die Einsatzmoglichkeiten derartiger Systeme durch Tiefflugfahigkeit, Orientierung in unbekanntem oder schwierigen Terrain sowie die Landeplatzbewertung erweitert.