Teleoperation allows a human operator to remotely interact with and control a mobile robot in a dangerous or inaccessible area. Besides well-known applications such as space exploration or search and rescue operations, the application of teleoperation in the area of automated driving, i.e., teleoperated driving (ToD), is becoming more popular. Instead of an in-vehicle human fallback driver, a remote operator can connect to the vehicle using cellular networks and resolve situations that are beyond the automated vehicle (AV)'s operational design domain. Teleoperation of AVs, and unmanned ground vehicles in general, introduces different problems, which are the focus of ongoing research. This paper presents an open source ToD software stack, which was developed for the purpose of carrying out this research. As shown in three demonstrations, the software stack can be deployed with minor overheads to control various vehicle systems remotely.
Traffic crashes claim more than 36,000 lives every year in the US, with economic costs amounting to $836B. Automated Vehicles (AVs) promise to usher in a future with near-zero crashes. It seems likely that widespread deployment of AVs will eliminate the large number of crashes caused by impaired, distracted or reckless drivers. However, it is unclear whether AVs will be able to avoid a significant fraction of the remaining crashes for which no driver is directly responsible. As such, deploying AVs without adequately assessing their safety capabilities might lead to an increase in crashes rather than a reduction. In this paper, we discuss how an analysis of human crashes can provide insights about the types of crashes that remain challenging for AVs and the role of connected infrastructure in addressing them. We also discuss how human crashes and driving data can be valuable in inferring safety capabilities of AVs in diverse driving contexts. Based on these observations, we provide suggestions for policies and regulations governing the deployment of AVs.
While intermittent renewable power generation in the form of wind and solar PV has become a significant presence on the worlds electricity grids, the intermittency associated with this power supply introduces uncertainty into the grid operators task of matching supply with demand. A consequence of this intermittency is the large costs that accrue when supply is lower than forecasted and this cost is increasingly being laid at the feet of the power producers running intermittent generators. These uncertainties present a significant obstacle to renewable investment and future growth. In this paper we investigate the opportunities arising from the projected growth in electric vehicles (EV's) and vehicle-to-grid enabled charging infrastructure to hedge against the large price risk associated with spot markets by accessing the stored energy in the form of unused battery capacity in EV's. Specifically, we investigate the scale of an electric car fleet needed to effectively provide this service as opposed to the economics of correctly pricing the service. By employing Quality of Service (QoS) based metrics we show that the number of participating vehicles needed to mitigate production risk is small and the impact on these vehicles minimal.
Heavy duty vehicle platooning under highway operating conditions, has been projected to provide significant fuel economy gains based on aerodynamic drag improvements of the platooning vehicles. Realizing these benefits under real-world operating conditions presents several challenges. This paper (the first as part of series) provides a framework to quantify the fuel economy benefits of both conventional and electrified powertrains operating across the U.S. Interstate highway system. In addition to the powertrain, key interactions that are explored include the vehicle separation distance, baseline vehicle aerodynamic properties, vehicle weight, number of vehicles in the platoon, excursions from road speed limits, and particular traffic interactions. While the approach makes use of a limited fidelity vehicle model for longitudinal dynamics, the narrative provides application decision personnel with a mechanism and well-defined set of fuel economy impact factors to consider as part of their architecture selection process.
This paper uses deep learning in the field of public transport, where it is of immense interest to know the occupation of vehicles for their coordination and scheduling. Counting people on a bus or train is very often still done manually, whereas this paper presents a modern approach with just the use of camera images and deep learning. The people counter presented in this work consists of two parts, in each of which a neural network has been optimized. In the first step, persons are recognized by means of a person detector while this information is used in the second step for counting people. For the detection of people at the bus entrance, RetinaNet was selected as the model and optimized. The output of the person detector was then used to optimize a novel architecture of a neural network. This allows determining the number of people getting on and off a bus in a video.
A realistic data basis is crucial for the development of Advanced Driver Assistance Systems and traffic research. Traffic cameras at urban intersections provide a way to efficiently obtain information about complex traffic behavior, with the trajectories of road users being particularly relevant. For this purpose, the established Deep Neural Network Mask R-CNN is trained with a self-generated dataset to segment vehicles, cyclists and pedestrians frame-by-frame achieving an Average Precision of 94.07 % for vehicles. For the tracking of objects, the tracking-algorithm SORT was found to be a suitable method. In order to minimize the perspective error of the vehicle position due to the lateral camera perspective, a novel method is presented that estimates ground planes for vehicles based on segmentation. The estimated trajectories are evaluated with data from real measurement runs with an reference vehicle over the intersection area, resulting in an average accuracy of 0.57 m. The generated data can be used in traffic simulation software and to create fully defined scenarios for virtual vehicle testing. Furthermore we provide an open-source implementation of the proposed work at https://github.com/jul095/TrafficMonitoring.
The increasing capability of ingesting crowdsensing data from vehicle fleets for a wide variety of applications on the basis of road conditions, is producing ever-increasing amounts of data. This needs to be aggregated in order to add value and reduce the quantity of data. In the case of spatial data, various clustering methods are available that serve this purpose. The main objective of this paper is to develop a method that provides lane-accurate spatio-temporal data on road anomalies on the basis of fleet data for the first time. To achieve this, a DBSCAN algorithm is applied in conjunction with an Affinity Propagation algorithm to cluster additional parameters. 95.7% of all single obstacles were classified as true positives using the DBSCAN algorithm. Moreover, additional metadata for classification using Affinity Propagation, resulted in a true positive rate of 99%.
Investment decisions in research and development include risks due to the uncertainty if the developed technology reaches market readiness and generates profit. Testing of Connected and Automated Vehicles (CAVs) is evolving quickly along the progress that is made in the fields of sensor technologies, computer vision, artificial intelligence, and control engineering. Open and manufacturer-specific proving ground owners need to prioritize their investments in infrastructure to test CAVs in order to increase their performance (i.e. profit, gain of knowledge, usability, and customer satisfaction). This theoretical paper elaborates on attributes proving ground owners could consider for investment decisions. These attributes are the increase in the proving ground's versatility, the adaptability, the rate of return, external costs, the date of requirement, and the importance of tests, which the infrastructure enables.
Battery degradation in electric vehicles is detrimental and keeping track of the State of Health (SOH) of the battery is essential. Current battery SOH estimation techniques (such as monitoring voltage decrease with increasing cycles) have several drawbacks, such as having monotonically decreasing predictions in non-monotonic systems and susceptibility to noise effects. There is a simpler and more reliable technique by monitoring the charging times of the battery in every cycle. This method is proven to be accurate and to have low susceptibility to noise effects. Results show nearly identical results in scenarios with and without noise.
We introduce a new color-based fiducial marker system-RainbowTag (RT)-for detection and identification that is suitable for autonomous navigation due to robustness to varying lighting conditions, motion blur, partial occlusion and folding. This system uses cameras already present on the vehicles to complement spatial information estimated from other sensors (e.g., Global Positioning System, inertial measurement, radar). RT is composed of a fiducial marker design and its adapted detection algorithm. Numerous real-world experiments demonstrate that markers can be reliably detected in various lighting conditions, in the presence of large motion blur, and even when folded or partially occluded. In all test conditions, RT outperforms the fiducial markers Aruco and ChromaTag. Compared to other blur-resistant fiducials that are circularly symmetric [1], [2], RT has the advantage that it encodes orientation information. Our detection algorithm is powered by a novel color segmentation approach that carefully orchestrates information from the hue constant IPT, the perceptually uniform CIELAB, and the Bradford LMS cone response color spaces.
Vehicular crowdsensing (VCS) is a subset of crowd-sensing where data collection is outsourced to group vehicles. Here, an entity interested in collecting data from a set of Places of Sensing Interest (PsI), advertises a set of sensing tasks, and the associated rewards. Vehicles attracted by the offered rewards deviate from their ongoing trajectories to visit and collect from one or more PsI. In this win-to-win scenario, vehicles reach their final destination with the extra reward, and the entity obtains the desired samples. Unfortunately, the efficiency of VCS can be undermined by the Sybil attack, in which an attacker can benefit from the injection of false vehicle identities. In this paper, we present a case study and analyze the effects of such an attack. We also propose a defense mechanism based on generative adversarial neural networks (GANs). We discuss GANs' advantages, and drawbacks in the context of VCS, and new trends in GANs' training that make them suitable for VCS.
Commercial applications of Unmanned Aerial Vehicles (UAVs) are expected to be one of the disruptive technologies that can shape many activities spans from goods delivery to surveillance. To maximize the effectiveness of such UAV applications, it is very important to enable beyond-line-of-sight (BLoS) communications. Hence, integrated UAV with LTE network can be used to extend UAV operations beyond visual line-of-sight (BVLOS) communications. This paper investigates the ability of Long-Term Evolution (LTE) network to provide coverage for UAV in such rural and urban area, in particular for the uplink video transmission. The system design takes into consideration the dependency of the large-scale path loss, shadowing and line-of-sight probability on the height of the UAV in the simulation environment, which is obtained from industrial measurements, and a real-world communication infrastructure layout and configuration. Results show that UAV height is a very critical factor in terms of delay and jitter performance for urban micro scenarios, and less effective in urban macro scenarios. Nevertheless, average throughput performance is less sensitive to the change in parameters and communication environments when the application type is set as a 1080p-quality video feed. Besides, mobility performance is explored for different system parameters such as hysteresis margin, time-to-trigger under various communications scenarios. Finally, the finding presented in this paper can be a roadmap to facilitate UAV-LTE integration in the near future.
Automated vehicles are expected to cause a paradigm shift in mobility and the way we travel. Particularly, the increasing penetration rates of vehicles with automated driving functions will introduce new challenges such as excessive rutting. Lack of wheel wander and keeping to the lane center perfectly is expected to lead to excessive road surface depression induced by automated vehicle platoons. Inspired by this, the EU project ESRIUM investigates infrastructure assisted routing recommendations. In this respect, specially designed ADAS functions are being developed with capabilities to adapt their behavior according to specific routing recommendations, which will help to reduce rutting as well as to improve safety. The current paper presents three rule-based trajectory planner design alternatives for a representative use case and, their performance comparisons based on a simulation framework.
The amount of information which is gathered, pro cessed and sent by vehicles increases permanently. Thereby, V2X communication is subject to various limitations such as limited bandwidth and hardware constraints. Furthermore, processing and analyzing vehicle data as well as training artificial neural networks on this enormous data amount is highly computational expensive. In conclusion, there is a need of system-wide optimization of data processing, data transmission, and data mining to reduce environmental burdens with respect to the named limitations. Therefore, we have defined the following research question: How to optimize vehicle communication under consideration of limited bandwidth, computational constraints, real time capability as well as the subsequent utilization of the vehicle data in data mining methods? To answer this research question, we developed a lightweight but extremely powerful compression scheme, which we applied on multivariate vehicle sensor time series. Our approach achieved Pareto-optimal compression results regarding the quality measures compression ratio and compression speed. The results demonstrated that our proposed method enables an efficient linkage of data compression and data mining within a holistic and a real time capable context.
With numerous new comfort functions such as infotainment or navigation systems being added to modern car cockpits, touchscreens gradually grew to be the primary input system for automotive UI. However, screen-based UI navigation requires extended visual attention and can thus be a dangerous distraction while driving. In this paper, we describe an innovative few-shot-learning real-time gesture control system that allows the driver to define custom dynamic hand gestures by recording a small set of samples for each gesture. By employing a deep convolutional neural network to extract 2D hand poses from a continuous stream of depth data, we can make use of a k-NN classifier to match gestures with a small set of previously recorded samples. Utilizing a state-of-the-art network architecture, our pose estimation network achieves an average keypoint distance accuracy of 87%–98.6%. Based on that, our overall system is able to classify gestures from as few as three samples with an accuracy of 93.69%. The system runs at 20fps in an unoptimized Python implementation and therefore meets the real-time requirements to a sufficient amount.
Connected Vehicles (CVs) make transportation safe by communicating with vehicles and the infrastructure in their neighborhood. CVs are embedded with onboard units (OBUs) which transmit basic safety messages (BSMs) containing location, heading, velocity information of the vehicle using either Dedicated Short-Range Communications (DSRC) or Cellular Vehicle-to-Everything (C-V2X) technology. BSMs can be used to warn the drivers in various hazardous vehicle-to-vehicle (V2V) scenarios such as Forward Collision Warning (FCW), Emergency Electronic Brake Light Assist (EEBL), Blind Spot Warning (BSW), etc. Many existing approaches employ computer vision techniques which are costly in processing power and suffer from several drawbacks arising from poor visibility. Further, they assume single lane scenario which is over-simplifying. We propose 1 1 A provisional US patent on this work is under consideration. a novel approach that overcomes these limitations by leveraging only BSMs and no cameras. It can handle multiple lane scenarios as well which requires accurate determination of the relative position of CVs. The host vehicle implements logic that computes relative angle with the remote vehicle based on which three types of hazardous conditions (FCW, EEBL, BSW) are detected and warned. The proposed approach is implemented in widely used CARLA simulator. We successfully demonstrate the three V2V safety applications in various scenarios.
Nefarious users or attackers may target vehicular networks for malicious objectives, such as unlawfully collecting data or tampering with communications. There are several coun-termeasures to these challenges in the literature; however, they are tailored to a specific scenario with infrastructure or focused on new routing protocols. To mitigate the effects of malicious attacks on vehicular network communications, this work proposes the development of a group-based security framework, which is independent of the routing protocol with vehicular-to-vehicular communication and capable of forming groups with trusted nodes and adding security to vehicular network communications between group members. This framework should be able to work in connected network settings while ensuring the security and integrity of the information transferred over it. To employ a hybrid cryptography system, the framework needs to have a trust management method for forming groups and establishing group-shared keys in a decentralized manner.
Environment perception sensors play a crucial role for advanced and autonomous driving. Lidar sensors in particular provide three-dimensional data with high temporal and spatial resolution, down to a few centimeters. Sensor models play a major role in autonomous driving simulations, virtual testing and validation. Ray tracing is an ideal method to model pulsed automotive Lidar sensor. We present a stand-alone Lidar sensor model, based on the Optix ray tracing engine, generating raw point cloud data. The model can be updated using Open Simulation Interface messages to simulate a full scenario. The focus of this paper is the structure of the model including an internal dynamic object update.
Numerous different road marking patterns are required to test advanced driver assistance systems and automated vehicles. These patterns are described in test protocols, e.g. from certifications, ratings and standards. Since proving ground operators possess only a confined space to provide these scenarios, solutions like agile alteration of road marking patterns are advisable. Furthermore, a high precision is necessary to comply with the specifications from the test protocols and to enable a high test-reproducibility. To achieve these goals mobile robots that follow determined trajectories can be used for pre-marking. Real-Time Kinematic (RTK) positioning can facilitate a high accuracy. Since proving ground operators or their users commonly have access to automated vehicles such as passenger cars or robotic target platforms, I recommend utilizing these for highly accurate pre-marking for temporary maneuver markings on test tracks. In order to test this concept, a truck equipped with GPS, UHF, LTE antennas, a driving controller and a pre-marking apparatus, was setup to follow predetermined trajectories on the Mercedes-Benz proving ground in Immendingen, Germany. Measurements of the specified pre-marked lane width returned mean accuracies of 98.74 and 99.72 % for two individual maneuver markings. Due to a pre-marking velocity of 4 m/s, the time needed to apply the pre-marker is low. The results indicate application potentials for proving ground operators and testing service providers.
Heavy duty (HD) vehicles platooning under highway operating conditions, have been projected to provide significant fuel economy (FE) gains based on aerodynamic drag reduction of the platooning vehicles. Realizing these benefits under real-world operating conditions have several challenges. This paper (the second as part of series) quantifies the minimum admissible separation distance between platooning vehicles in HD commercial vehicle (CV) applications. This distance must be set up such that no collision occurs with leading vehicles under all deceleration conditions. In this paper key interactions to characterize this distance are demonstrated. These include, road surface conditions, grade, inter-vehicle communication delays, air-brake lag time, vehicle speed, weight, and aerodynamic drag. It is seen that characterizing the vehicle and road dynamics will be critical in establishing required deceleration or stopping distances for each vehicle in a platoon. The results of the study also necessitate vehicle control systems that can dynamically adjust not only the separation distances but also the braking reference points based on road conditions. The narrative provides application decision personnel with a well-defined set of admissible separation distance impact factors to consider as part of their architecture selection process.
