Visual observation of objects is essential for many robotic applications, such as object reconstruction and manipulation, navigation, and scene understanding. Machine learning algorithms constitute the state-of-the-art in many fields but require vast data sets, which are costly and time-intensive to collect. Automated strategies for observation and exploration are crucial to enhance the efficiency of data gathering. Therefore, a novel strategy utilizing the Next-Best-Trajectory principle is developed for a robot manipulator operating in dynamic environments. Local trajectories are generated to maximize the information gained from observations along the path while avoiding collisions. We employ a voxel map for environment modeling and utilize raycasting from perspectives around a point of interest to estimate the information gain. A global ergodic trajectory planner provides an optional reference trajectory to the local planner, improving exploration and helping to avoid local minima. To enhance computational efficiency, raycasting for estimating the information gain in the environment is executed in parallel on the graphics processing unit. Benchmark results confirm the efficiency of the parallelization, while real-world experiments demonstrate the strategy's effectiveness.

Offline reinforcement learning (RL) provides a framework for learning decision-making from offline data and therefore constitutes a promising approach for real-world applications such as automated driving (AD). Especially in safety-critical applications, interpretability and transferability are crucial to success. That motivates model-based offline RL approaches, which leverage planning. However, current state-of-the-art (SOTA) methods often neglect the influence of aleatoric uncertainty arising from the stochastic behavior of multi-agent systems. Further, while many algorithms state that they are suitable for AD, there is still a lack of evaluation in challenging scenarios. This work proposes a novel approach for Uncertainty-aware Model-Based Offline REinforcement Learning Leveraging pl A nning (UMBRELLA), which jointly solves the prediction, planning, and control problem of the self-driving vehicle (SDV) in an interpretable learning-based fashion. A trained action-conditioned stochastic dynamics model captures distinctively different future evolutions of the traffic scene. The analysis provides empirical evidence for the effectiveness of our approach and SOTA performance in challenging AD simulations and using a real-world public dataset.

Motion planning and control are crucial components of robotics applications like automated driving. Here, spatio-temporal hard constraints like system dynamics and safety boundaries (e.g., obstacles) restrict the robot's motions. Direct methods from optimal control solve a constrained optimization problem. However, in many applications finding a proper cost function is inherently difficult because of the weighting of partially conflicting objectives. On the other hand, Imitation Learning (IL) methods such as Behavior Cloning (BC) provide an intuitive framework for learning decision-making from offline demonstrations and constitute a promising avenue for planning and control in complex robot applications. Prior work primarily relied on soft constraint approaches, which use additional auxiliary loss terms describing the constraints. However, catastrophic safety-critical failures might occur in out-of-distribution (OOD) scenarios. This work integrates the flexibility of IL with hard constraint handling in optimal control. Our approach constitutes a general framework for constraint robotic motion planning and control, as well as traffic agent simulation, whereas we focus on mobile robot and automated driving applications. Hard constraints are integrated into the learning problem in a differentiable manner, via explicit completion and gradient-based correction. Simulated experiments of mobile robot navigation and automated driving provide evidence for the performance of the proposed method.

Offline reinforcement learning (RL) provides a framework for learning decisionmaking from offline data and therefore constitutes a promising approach for realworld applications as automated driving. Self-driving vehicles (SDV) learn a policy, which potentially even outperforms the behavior in the sub-optimal data set. Especially in safety-critical applications as automated driving, explainability and transferability are key to success. This motivates the use of model-based offline RL approaches, which leverage planning. However, current state-of-theart methods often neglect the influence of aleatoric uncertainty arising from the stochastic behavior of multi-agent systems. This work proposes a novel approach for Uncertainty-aware Model-Based Offline REinforcement Learning Leveraging plAnning (UMBRELLA), which solves the prediction, planning, and control problem of the SDV jointly in an interpretable learning-based fashion. A trained action-conditioned stochastic dynamics model captures distinctively different future evolutions of the traffic scene. The analysis provides empirical evidence for the effectiveness of our approach in challenging automated driving simulations and based on a real-world public dataset.

SummaryCollaborative robots have to adapt its motion plan to a dynamic environment and variation of task constraints. Currently, they detect collisions and interrupt or postpone their motion plan to prevent harm to humans or objects. The more advanced strategy proposed in this article uses online trajectory optimization to anticipate potential collisions, task variations, and to adapt the motion plan accordingly. The online trajectory planner pursues a model predictive control approach to account for dynamic motion objectives and constraints during task execution. The prediction model relates reference joint velocities to actual joint positions as an approximation of built‐in robot tracking controllers. The optimal control problem is solved with direct collocation based on a hypergraph structure, which represents the nonlinear program and allows to efficiently adapt to structural changes in the optimization problem caused by moving obstacles. To demonstrate the effectiveness of the approach, the robot imitates pick‐and‐place tasks while avoiding self‐collisions, semistatic, and dynamic obstacles, including a person. The analysis of the approach concerns computation time, constraint violations, and smoothness. It shows that after model identification, order reduction, and validation on the real robot, parallel integrators with compensation for input delays exhibit the best compromise between accuracy and computational complexity. The model predictive controller can successfully approach a moving target configuration without prior knowledge of the reference motion. The results show that pure hard constraints are not sufficient and lead to nonsmooth controls. In combination with soft constraints, which evaluate the proximity of obstacles, smooth and safe trajectories are planned.

Menschliche Fahrfehler stellen die Hauptursache für Unfälle im Straßenverkehr dar. Die automatische Verkehrsüberwachung bietet einen Beitrag, um die Vision des unfallfreien Straßenverkehrs zu erreichen. Eine solche Infrastruktur erhöht unmittelbar die Verkehrssicherheit insbesondere vor dem Hintergrund einer langwierigen Durchdringung des Fahrzeugbestandes durch neue Assistenzsysteme. Das Multi Funktionale Detektions System erkennt an Autobahnabfahrten sowie Park- und Rastanlagen potenzielle Falschfahrer bei der Auffahrt auf die falsche Richtungsfahrbahn. Typischerweise werden dabei ortsfeste Messplattformen eingesetzt, hierfür bieten Kamerasensoren besonders günstige Voraussetzungen. Der vorliegende Beitrag betrachtet die Fahrzeugdetektion in Kamerabildern für die Integration in ein Falschfahrerwarnsystem. Für diese Anwendung ist die Realisierung von möglichst schnellen Verarbeitungszeiten entscheidend. Gängige Methoden der kamerabasierten Objektdetektion führen zu diesem Zweck eine vollständige Abtastung des aufgezeichneten Bildes durch. Aufgrund der feststehenden Kameraposition und der bekannten statischen Verkehrselemente lässt sich jedoch der Suchraum deutlich einschränken. Der Ansatz basiert auf einer reduzierten Suchstrategie, welche die vorteilhaften Eigenschaften herkömmlicher Verfahren zur Objektdetektion erhält.

Zusammenfassung Der vorliegende Beitrag untersucht die Erkennung benachbarter Fahrstreifen auf Grundlage von Kamerabildern. Hierbei wird sowohl die Anzahl befahrbarer Fahrstreifen als auch deren Verlauf innerhalb eines festgelegten Bereichs vor dem Fahrzeug bildbasiert geschätzt. Die Erkennung erfolgt durch Convolutional Neural Networks. Der Beitrag bewertet die Güte der Schätzung und Genauigkeit der Fahrstreifenerkennung. Zur Interpretation der Entscheidungen werden die durch das neuronale Netz gelernten Merkmale und Zwischenrepräsentationen visualisiert.

The article presents an overview of the status quo in benchmarking in classification and nonlinear regression. It outlines guidelines for a comparative analysis in machine learning, benchmarking principles, accuracy estimation, and model validation. It provides references to established repositories and competitions and discusses the objectives and limitations of benchmarking. Benchmarking is key to progress in machine learning as it allows an unprejudiced comparison among alternative methods. This article presents guidelines and best practices for benchmarking in classification and regression. It reviews state-of-the-art approaches in machine learning, establishes benchmarking principles and discusses performance metrics for a sound statistical comparative analysis. This article is categorized under: Technologies > Computational Intelligence Fundamental Concepts of Data and Knowledge > Key Design Issues in Data Mining Technologies > Machine Learning Technologies > Classification

Traffic scene understanding involves the perception and accurate representation of a vehicles surroundings within a comprehensive environment model. Generally, automotive environment models incorporate multiple different and complementary representations each focusing on specific aspects of traffic scenes such as the drivable terrain, dynamic objects, the lane layout, etc. Considering camera based environment perception, multi-task models offer an efficient approach capable of generating complementary representations in an integrated manner. Towards this objective, we propose a convolutional neural network (CNN) comprised of a shared encoder stage and specific decoders for road segmentation and object detection. In comparison to exclusive models for each separate task, the redundant computation of common feature maps is avoided resulting in fast inference times even on low cost embedded systems. Furthermore, the detection stage is extended to predict the orientation of detected objects. The orientation estimate guides a consecutive 3D bounding box estimation based on analytic geometry. Thus, the CNN suffices to generate a basic environment model consisting of the spatial information of dynamic objects as well as the free space area obtained from road segmentation.

This tutorial chapter provides a comprehensive step-by-step guide on the setup of the navigation stack and the teb_local_planner package for mobile robot navigation in dynamic environments. The teb_local_planner explicitly considers dynamic obstacles and their predicted motions to plan an optimal collision-free trajectory. The chapter introduces a novel plugin to the costmap_converter ROS package which supports the detection and motion estimation of moving objects from the local costmap. This tutorial covers the theoretical foundations of the obstacle detection and trajectory optimization in dynamic scenarios. The presentation is designated for ROS Kinetic and Lunar and both packages will be maintained in future ROS distributions.

When working together, human and robots complement each other’s abilities yielding more efficient task execution. The cognitive abilities of humans are complemented by physical robot support, reducing fatigue and stress, particularly in repetitive tasks. A robot-human handover is an example of interaction in which human and robot act in temporal and spatial concurrence, with particular relevance in scenarios without temporary object placement, such as service robotics. This physical and cognitive interaction between human and robot requires coordination and effective communication. This work concerns a robot-human object handover with a robot arm equipped with a two-finger robotic gripper. Mutual task awareness between human and robot is attained by both haptic and visual feedback. A force/torque sensor measures the grasp forces imposed by the human which eventually trigger the release of the object. The handover behavior releases the object once force and torque thresholds are exceeded. The decision rule aggregates direct and integral measures of the force/torque signal. In other words, the gripper releases either if the human imposes a significant force or a medium force over an extended period of time. Furthermore, a lighting ring in the robot wrist provides the user with visual feedback on the force exerted and indicates a forthcoming grip release. The evaluation of the algorithm considers the qualitative individual human perception of the naturality of the handover with and without visual feedback. Results show that merely static thresholds w.r.t. direct force and torque measures provide an effective handover but parameterization proved difficult as the algorithm does not adapt to the conditions of the handover. An adaptive threshold based on integral measures attained better results when parameterized for maximum adaptability. Furthermore, visual feedback has a positive impact on the human perception of the naturality of the handover as users receive information on the required force for attaining an object release.

This paper proposes a hypergraph formulation for solving MPC problems. The hypergraph approach exploits the sparse structure in the calculation of derivatives. It is therefore computationally more efficient in case of multiple-shooting, collocation and full-discretization methods compared to a dense formulation. Recent advances in realtime optimization rely on automatic differentiation (AD) to compute derivatives. An extensive analysis compares MPC variants with both hypergraph and AD on two benchmark control problems. Even though AD requires a computational overhead to set up the problem structure, solving the nonlinear program at each iteration is fast. The overhead in the hypergraph approach is negligible, and computational effort in the solving phase is inferior but comparable to AD. This observation favors the hypergraph representation for MPC problems with non-static problem structure.

This paper presents a purely visual semantic mapping framework using omnidirectional images. The approach rests upon the robust segmentation of the robot's local free space, replacing conventional range sensors for the generation of occupancy grid maps. The perceptions are mapped into a bird's eye view allowing an inverse sensor model directly by removing the non-linear distortions of the omnidirectional camera mirror. The system relies on a place category classifier to label the navigation relevant categories: room, corridor, doorway, and open room. Each place class maintains a separated grid map that are fused with the range-based occupancy grid for building a dense semantic map.

The perception of traffic related objects in the vehicles environment is an essential prerequisite for future autonomous driving. Cameras are particularly suited for this task, as the traffic relevant information of a scene is inferable from its visual appearance. In traffic scene understanding, semantic segmentation denotes the task of generating and labeling regions in the image that correspond to specific object categories, such as cars or road area. In contrast, the task of scene recognition assigns a global label to an image, that reflects the overall category of the scene. This paper presents a deep neural network (DNN) capable of solving both problems in a computationally efficient manner. The architecture is designed to avoid redundant computations, as the task specific decoders share a common feature encoder stage. A novel Hadamard layer with element-wise weights efficiently exploits spatial priors for the segmentation task. Traffic scene segmentation is investigated in conjunction with road topology recognition based on the cityscapes dataset [1] augmented with manually labeled road topology ground truth data.

This paper presents a novel generic formulation of Timed-Elastic-Bands for efficient online motion planning of car-like robots. The planning problem is defined in terms of a finite-dimensional and sparse optimization problem subject to the robots kinodynamic constraints and obstacle avoidance. Control actions are implicitly included in the optimized trajectory. Reliable navigation in dynamic environments is accomplished by augmenting the inner optimization loop with state feedback. The predictive control scheme is real-time capable and responds to obstacles within the robot's perceptual field. Navigation in large and complex environments is achieved in a pure pursuit fashion by requesting intermediate goals from a global planner. Requirements on the initial global path are fairly mild, compliance with the robot kinematics is not required. A comparative analysis with Reeds and Shepp curves and investigation of prototypical car maneuvers illustrate the advantages of the approach.