Advanced automated driving presents significant potential to improve modern automotive safety systems, but it depends highly on the reliable activation of restraint systems. Forward-looking sensors are crucial for immediate and precise object detection. Recent developments in automotive radar technology enable detailed environment detection and the recognition of high-resolution features, such as micro-Doppler signatures. Combined with advanced AI techniques, these features significantly enhance object detection and improve the accuracy of kinematic parameter estimation. This is essential for the early and reliable activation of irreversible safety systems, such as smart airbags and adaptive seat belts. Therefore, an anchor-based AI model is presented, designed to process high-resolution radar data with an explicit focus on micro-Doppler signatures to improve pre-crash object detection. Furthermore, these signatures can improve the accuracy of kinematic object parameter estimation and reduce false negatives, especially in the critical near-field. To address the challenges of sparse and fluctuating radar point clouds, an innovative radar-image dilation technique on the feature input channels was developed to amplify local radar patterns, like micro-Doppler features. Therefore, this approach increases the system's reliability and increases its ability to detect objects in pre-crash scenarios despite radar multipath reflections and ghost objects. In order to investigate the applicability and compare the model's performance with advanced automotive radar tracking methods, a radar data set using series sensors and pre-crash relevant scenarios was recorded. The results demonstrate the advantages of the anchor-based AI model over established tracking approaches. It excels at estimating object parameters in dynamic scenarios and underscores its ability to process different data sets effectively.
Dense direct visual odometry (DDVO) is a well-known robust model-based visual odometry method, that computes robot self-localization with visual data. However, to obtain a metric localization depth information is needed, which is not available in most applications, especially in outdoor scenarios. For this reason, we focus on a hybrid dense direct visual odometry (HDVO) that combines a model-based localization module and a data-based stereo depth estimation module. Since depth is estimated from a stereo sensor, the proposed HDVO can achieve better localization performance than methods using only monocular RGB data. Moreover, the learning part of HDVO is optimized with the photometric loss of DDVO without any depth supervision signal. For this loss, occlusion and low texture will affect the optimization convergence. This paper investigates a binary mask based on stereo-temporal consistency to address the occlusion problem, along with another binary mask based on local patch consistency to solve the low-texture problem. Finally, it is shown that HDVO can achieve a significant advantage in terms of both inference speed and odometry error on public benchmarks. The corresponding code is available at https://gitlab.inria.fr/acentauri-public/hdvo.
We propose a system featuring an omnidirectional camera mounted on an electric wheelchair to automatically detect Braille blocks in all directions using machine learning. This system assists in navigating the electric wheelchair along the detected Braille blocks. Initially, Braille blocks are identified using You Only Look Once version 8 (YOLOv8) from images captured by an onboard omnidirectional camera. The system then determines the travel direction based on these detection results. Subsequently, it estimates the distance between the wheelchair and the Braille block. For this distance estimation, we introduce a novel method based on the equidistant projection technique used for generating omnidirectional camera images. Utilizing this information, the joystick is controlled to maintain a specific distance from the Braille blocks, aiding in the wheelchair's navigation. In this study, we developed a new mechanism compatible with various types of electric wheelchairs. This mechanism operates the joystick, enabling autonomous navigation without the need to interact with the internal systems of the electric wheelchair. During the verification experiments, the system achieved a 97.3% accuracy rate in detecting Braille blocks in both indoor and outdoor settings, a 100% success rate in determining travel direction, and a low average distance measurement error between the wheelchair and Braille blocks. Our findings confirm that the proposed system, with its algorithms and mechanics, effectively assists wheelchair users.
Nano quadcopter unmanned aerial vehicles (nQUAVs) offer easy deployment, high manoeuvrability, scalability, and low cost, but their small size and lightweight structure make them highly susceptible to external disturbances and system uncertainties, while limited onboard computation constrains the complexity of control algorithms. This paper presents an Active Adaptive Bidirectional Fuzzy Brain Emotional Learning (AA-BFBEL) controller for precise trajectory tracking and disturbance rejection in uncertain environments. Unlike conventional passive methods, the AA-BFBEL integrates a computationally efficient disturbance observer based on an Extended State Observer (ESO) into the BFBEL framework to estimate wind forces and system uncertainties in real time, which are then actively compensated without prior training. This design is particularly suited to nQUAVs with short flight durations and limited processing resources, in contrast to reinforcement or deep learning-based controllers. The controller was validated through simulations and real-world experiments on the Crazyflie nano quadcopter, tracking complex three-dimensional figure-8 trajectories-a demanding test scenario for small UAVs. Across all cases, the AA-BFBEL significantly outperformed the baseline BFBEL, conventional PID, sliding mode control (SMC), and active disturbance rejection control (ADRC), achieving up to 74% higher tracking accuracy and nearly 61% faster settling time under strong wind conditions. These results demonstrate that the AA-BFBEL delivers high precision, robustness, and adaptability, providing an effective and reliable control solution for nQUAV operations in dynamic and challenging environments.
Reinforcement learning (RL) has emerged as a promising approach for achieving high-level autonomous driving as its self-evolving ability without reliance on human rules. Although RL-driven methods have yielded fruitful demonstrations in driving domain, most of them are trained and validated only on simulation platforms. Due to the inherent difference between simulation and reality, the driving policy usually performs poorly when applied to a realistic environment. In this paper, we propose an adversarial training framework for driving policy to enhance practical performance, which introduces the adversarial policy to simulate the discrepancy of environments during training and incorporates the collected data from the real world to provide the discrepancy bound. Besides, the adversarial policy is iteratively updated with gradient-based optimization, which enables the automatic generation of diversified discrepancies for different traffic participants. The action projection is developed to ensure that the output of adversary satisfies the discrepancy bound given by data, so as to prevent an aggressive adversary making the overly conservative policy. We evaluate the trained policy on a fully sized vehicle at an urban intersection with mixed traffic flows. Results indicate that our driving policy can handle unseen behaviors of traffic actors meanwhile realizing the safe and smooth control for the automated vehicle. Our work provides a feasible solution for RL implementation in the field of real-world autonomous driving.
Lane changes are common maneuvers in daily and natural driving on public roads. Automated vehicles require information about the predicted motion of surrounding vehicles to be able plan their motion. This survey gives an overview on the state of the art on lane change prediction with respect to the datasets, methods for classification and features as inputs that are most frequently used, the most common outputs in terms of the classification problem they solved (such as lane keeping and left and/or right lane change), definition of occurrence of a lane change and metrics used to evaluate the estimation results. Overall, 76 articles were included in the analysis, with 58 individual and 13 cumulative features being identified and assigned percentages with which they were used in the publications. Five classes of outputs were analysed in terms of accuracy of estimation results and its relation to the prediction time. The analysis of the latest publications revealed an increasing interest in an already established prediction method, recurrent neural networks, but revealed also that newer methodologies like transformer networks have been relatively often used in the last five years showing that there is still room for experimenting with this task. In general, the prediction time was below 5 seconds and with increasing prediction time, the estimation accuracy decreased.
Albeit model predictive control (MPC) has been broadly applied for the trajectory tracking of unmanned ground vehicles (UGVs), certain issues still remain to be further investigated, particularly with regard to the tracking stability analysis and computational efficiency. This paper formulates the trajectory tracking for Ackerman-steering UGV as an optimization-constrained control problem within the MPC framework, incorporating stability analysis through the specification of terminal ingredients. To relax the computational burdens of MPC in network transmission, this paper integrates historical triggered signals into the event-triggered scheme synthesis, ensuring that critical moments, such as peaks and troughs in the system dynamics, are triggered. Additionally, an upper limit on untriggered signals is imposed to safeguard against potential abnormal behaviors on event generators. The effectiveness of the proposed resilient memory-event-triggered MPC algorithm is validated through both computer simulations and hardware experiments of tracking circular and “8”-shaped trajectories.
This paper considers deploying a multi-rotor Un manned Aerial Vehicle (UAV) for spraying pesticides precisely and evenly, for completely covering a bounded crop field. Due to varying wind, the multi-rotor UAV can spray inaccurate amount of pesticides on wrong areas, which leads to unwanted pollution of nature. To resolve this problem, this paper addresses a Complete Coverage Route (CCR) planning approach to spray pesticides precisely and evenly on a bounded crop field, considering the effect of wind explicitly. Considering the wind effect, we plan the multi-rotor UAV's CCR so that there exists no holes in pesticide coverage of a cluttered bounded crop field which may contain conservative area (ex: non-target vegetation or clean pond). The multi-rotor UAV can fly above conservative areas, but we need to avoid spraying pesticide over conservative areas. To the best of our knowledge, our research is novel in proposing how to plan the multi-rotor UAV's CCR under varying wind, so that the UAV can spray pesticides inside the cluttered bounded crop field precisely and evenly.
Decision Planning (DP) is a critical module in achieving level 5 autonomy for Autonomous Vehicles (AVs). Existing state of-the-art DP methods each have distinct advantages and limitations. This paper presents a hybrid Decision-Making (DM) framework that integrates classical and deep learning methodologies to emulate human-like behavior in diverse driving scenarios. The proposed architecture includes three main components: (1) a dual-stream Convolutional Neural Network (CNN) that processes front- and rear-view camera inputs to classify driving scenarios as either simple (highway) or complex (urban), (2) a Hierarchical Finite State Machine (HFSM) for handling simple scenarios, and (3) a Deep Learning (DL) module for managing complex environments. The system was evaluated in the CARLA simulator using diverse traffic scenarios. The human driver baseline was derived from a survey of 50 drivers, who assessed the optimal decisions for simulated scenes. The hybrid model outperformed baselines, achieving a 30% improvement in decision optimality over human drivers and a 45% improvement over a standalone DL approach. It also maintained a safety rate of 92%, an efficiency of 66%, and an 85.5% lane idleness rate. These results demonstrate the potential of the proposed framework to enhance safety, generalization, and performance in fully autonomous driving systems.
The ability of efficient path planning and obstacle avoidance is a key requirement for mobile robots in most real-world applications. This paper presents a novel Bug-based path-planning algorithm for mobile robots. Building upon the Bug-2 algorithm, the proposed Shortcut Bug algorithm generates shorter paths compared to existing deterministic methods by reducing unnecessary boundary following around obstacles. The proposed method does not require advanced capabilities such as distance estimation or obstacle edge detection and allows the robot to leave the boundary of the obstacle earlier than the existing algorithms. Comprehensive simulations across various maps and obstacles revealed that the proposed algorithm produces paths that are, on average, 7.93% shorter than its closest counterpart. Additionally, the percentage deviation from the optimal path is at least 19.4% lower than that of existing algorithms, demonstrating the proposed algorithm's ability to generate paths closer to optimal solutions.
Path planning and tracking of terrain vehicles are key to navigation safety in the field considering complex working conditions such as uncertain rough surfaces and deformable soil. This paper proposes a hierarchical path-planning and tracking framework to address inherent stochasticity and safety-critical problems in deformable terrain navigation. Firstly, a safe planning method based on distributional reinforcement learning is proposed, where route safety is strengthened by Conditional Value at Risk (CVaR) optimization based on terrain risk evaluation and constraints. Then, the terramechanics establishes the dynamics model for terrain vehicles driving on soft surfaces. A tracking controller based on the Adaptive Prescribed Performance Sliding Mode Control (APPSMC) is developed, where an improved prescribed performance function and a Fuzzy Logic System (FLS) are introduced to estimate the ground vehicle dynamics as well as the environmental disturbances. Simulation conducted on high-fidelity terrain environments shows satisfactory planning and tracking performances. It exhibits engineering transferability for autonomous operations such as planetary exploration and precision agriculture, providing modular design for seamless integration into off-road robotic platforms, and safety-critical tasks in unpredictable, unstructured environments.
Panoptic perception forms the foundation for decision-making in autonomous vehicles. This comprehensive perception includes essential functions such as lane line detection, drivable area recognition and vehicle detection. However, existing methods mainly focus on normal weather conditions, resulting in a significant degradation of Panoptic perception performance under inclement weather conditions including snow rain and haze. To improve the accuracy and robustness of panoramic perception under inclement Weather conditions, a multi-task network is proposed termed UniPerception, which uses a hybrid architecture of Transformer and CNN and a multi-stage learning strategy for parameter updating. Due to the lack of a dataset for severe weather, we developed the BDD100 K dataset using image enhancement techniques. Experimental results indicate that the UniPerception model consistently outperforms advanced multitasking and single-tasking networks in a variety of tasks under inclement weather conditions.
Implementing energy-saving practices requires accurately modeling electric vehicle (EV) energy consumption. This research proposes a data-centric approach to predict the energy consumption of EVs using the nonlinear autoregressive with exogenous inputs (NARX) model structure with the motor torque and vehicle speed as inputs. The model was identified and validated with on-road driving data from four trips between Southampton and various U.K. cities, achieving at least 99.2% accuracy in total energy consumption predictions. The NARX model achieved good accuracy in both simulation and prediction configurations, while reducing the mean squared error (MSE) by 30% compared to the Long Short-Term Memory (LSTM) network, which is a machine learning model structure with feedback and feedforward components. Although a feedforward neural network (NN) performed comparable to the NARX model when using past inputs and outputs, the NARX model proved more computationally efficient due to its more straightforward structure with fewer parameters. These results also demonstrate potential benefits of integrating system identification techniques to enhance machine learning models' performance for dynamic systems.
In this paper, we present a motion planning algorithm designed to guide agents, termed as player agents, optimally through multi-agent 3D urban air environments. The method integrates a sampling-based path planner, model-free optimal control, and a cognitive hierarchy model to predict the motion of other agents. Each player constructs a path through the environment, which is dynamically re-planned as the obstacle space of the environment evolves based on its online observations and the observations of cooperating players. The cognitive hierarchy model predicts the behavior of each agent in the environment, while a Gaussian process classification method estimates an unknown agent's level of rationality in real-time by observing each agent's kinodynamic distance. Once another agent's motion planning strategy is inferred, the player agents construct a predicted obstacle space based on each agent's expected motion to avoid potential collisions. Each player then traverses its planned path using a Q-learning controller. We validate the effectiveness of the proposed method in numerical experiments of a 3D urban air environment containing four and ten agents. We demonstrate that this approach is effective for reducing distance traveled by agents to reach their goals, mitigating the risk of collisions, and preventing deadlocks.
Ride-hailing platforms have a profound impact on urban transportation systems, and their performance largely depends on how intelligently they dispatch vehicles in real time. In this work, we develop a new approach to online vehicle dispatch that strengthens a platform's ability to serve more requests under demand uncertainty. We introduce a novel measure called sink proximity, a network-science-inspired measure that captures how demand and vehicle flows are likely to evolve across the city. By integrating this measure into a shareability-network framework, we design an online dispatch algorithm that naturally considers future network states, without depending on fragile spatiotemporal forecasts. Numerical studies demonstrate that our proposed solution significantly improves the request service rate under peak hours within a receding horizon framework with limited future information available.
Low-speed vehicles such as bicycles and e-scooters mark a potential shift in modern transportation, distancing themselves from traditional modes by offering several advantages. The appeal of micromobility lies in its efficiency, flexibility, and eco-friendliness. This paper provides a comprehensive overview of the concept of micromobility, with a focus on its seamless integration into Cooperative Intelligent Transportation Systems (C-ITS) through the adoption of advanced technologies and emerging processes. This study emphasizes the critical role of the Internet of Vehicles (IoV) and Vehicular Ad Hoc Network (VANET) applications. It highlights their significance in facilitating real-time communication through Vehicle-to Everything (V2X) message exchanges and efficient traffic management. This is ensured by communication modes such as ITS-G5, based on the IEEE 802.11p, a predefined standard of vehicular communication, and mobile networks, enabling seamless connectivity and data exchange. Furthermore, this paper discusses the integration of advanced technologies, such as geofencing and geolocation, to enhance the management and supervision of micromobility vehicles. Finally, in such an environment, considering energy consumption is essential to ensure efficient and sustainable operation of the system. Addressing these challenges is crucial for fully integrating micromobility into C-ITS. The scope of this work includes a multidisciplinary exploration of technologies addressing various aspects of micromobility, providing a holistic perspective.
As various 3D light detection and ranging (LiDAR) sensors have been introduced to the market, research on multi-session simultaneous localization and mapping (MSS) using heterogeneous LiDAR sensors has been actively conducted. Existing MSS methods mostly rely on loop closure detection for inter-session alignment; however, the performance of loop closure detection can be potentially degraded owing to the differences in the density and field of view (FoV) of the sensors used in different sessions. In this study, we challenge the existing paradigm that relies heavily on loop detection modules and propose a novel MSS framework, called Multi-Mapcher, that employs large-scale map-to-map registration to perform inter-session initial alignment, which is commonly assumed to be infeasible, by leveraging outlier-robust 3D point cloud registration. Next, after finding inter-session loops by radius search based on the assumption that the inter-session initial alignment is sufficiently precise, anchor node-based robust pose graph optimization is employed to build a consistent global map. As demonstrated in our experiments, our approach shows substantially better MSS performance for various LiDAR sensors used to capture the sessions and is faster than state-of-the-art approaches. Our code is available at https://github.com/url-kaist/multi-mapcher.
Path-speed decomposition-based trajectory planning schemes have garnered widespread usage in real-world robotics applications due to their efficacy and computational efficiency. While a global route can be planned offline, generating a local path adaptive to real-time situations online remains essential. We propose a local path planning algorithm that prioritizes smoothness and low computational complexity, facilitating scalability to dense environments with various on-road entities. Our algorithm leverages a sparse graph structure to generate crucial obstacle-specific nodes and connect them via spline edges. Several conditional checks are introduced to maintain graph sparsity, boosting computational efficiency without compromising performance. The final path evaluation considers both the smoothness of the path and the risks to vulnerable road users. The effectiveness of the proposed algorithm is demonstrated through CARLA simulation studies and extensive comparative analysis against benchmarking methods. Finally, a scaled car demonstration with a dynamic vehicle on a curved road is presented to showcase the performance of the proposed method on a physical system.
Urban intersections are critical areas where traffic flows converge and conflict, significantly influencing traffic safety, economic benefits, and energy consumption. Effective management and control of intersections have become a central focus in transportation research. With the advancement of automation technology, intersection management methods combined with connected autonomous vehicles (CAVs) have been rapidly developed. However, a comprehensive analysis of these emerging approaches remains lacking, from intersection design to management. This paper systematically reviews recent research on cooperative intersection management (CIM). Firstly, it explores the design of intersections. Secondly, various management objectives and evaluation methods are outlined. Thirdly, relevant research on vehicle trajectory control at the micro level, intersection management at the meso level, and arterial traffic flow regulation and network management at the macro level are discussed in detail. Based on this analysis, this paper identifies future research themes, emphasizing the need for trade-offs, integration, and coordination. Key areas for further study include enhancing the alignment between abstract models and real-world applications, balancing the performance of control methods with their implementation efficiency, and integrating various intersection control strategies to collectively enhance traffic efficiency, sustainability, and safety cooperatively.
The objective of this paper is to propose a reliable control solution that addresses the issue of communication disconnections in a heterogeneous vehicle platoon. For each vehicle in the platoon, predecessor-leader following (PLF) communication is selected. In the event of a communication failure, the vehicle transitions to the predecessor-following (PF) topology until communication with the leader is reestablished. To address this issue, a controller has been developed with the objective of adapting to the current communication topology on each vehicle. Event-triggering is implemented to reduce the amount of control orders given to the throttle/brake actuator pedals on each vehicle. Constant time spacing is selected for vehicle separation. A novel method is proposed to ensure string stability of the platoon during the controller design phase. Closed-loop stability of the proposed controlled vehicle platoon is guaranteed under Lyapunov criterion. Robustness against external disturbances and sensor measurement errors is guaranteed under $\mathscr {H}_\infty$ criterion. Simulations demonstrate that the proposed platoon control methodology can enhance road safety in the event of network disconnections. In the most unfavorable circumstances, the separation error is reduced by 40% in comparison to model predictive control techniques and by 73% in comparison to the intelligent driver model.
