Unmanned aerial vehicles (UAVs) exhibit considerable potential in post-disaster search and rescue, environmental monitoring, and other related applications. However, achieving collaborative path planning to cover irregular regions remains a key challenge. To address this issue, this study designs a comprehensive pipeline to generate paths for UAVs to cover a nonconvex region with nonconvex obstacles, thereby minimizing not only path length but also the number of turns in the sky. The pipeline comprises three main modules, i.e., region division, task allocation, and path generation. In the first module, a region division module upon polygon decomposition is proposed for scenarios including nonconvex regions with nonconvex obstacles. The task allocation module is modeled as a graph node grouping problem, which is then solved by an iterative algorithm designed upon the graph structure to balance computing load and path cost. In the last module, different from traditional vertical scanning, a direction-optimized scanning strategy is proposed, which solves for the direction that minimizes path length and number of turns first, and then scans the region. Simulations and experiments using 3 UAVs are carried out. Results show the effectiveness of the proposed algorithm in various irregular regions and its applicability to real platforms.
Adverse weather poses a critical challenge to autonomous vehicles due to asymmetric sensor degradation: rain and fog severely degrade cameras, snow attenuates LiDAR by causing severe backscattering, while radar remains robust. This asymmetry breaks conventional fusion architectures that assume consistent sensor reliability—the "static fusion paradox." Existing surveys focus on clear-weather scenarios, leaving gaps in understanding how architectures handle dynamically degrading sensors. This survey provides the first comprehensive analysis of deep learning-based multimodal fusion for asymmetric degradation under adverse weather. We systematically reviewed 30 highly relevant multimodal fusion papers from the broader autonomous vehicle literature (2016–2025), presenting: (1) a degradation-centric fusion taxonomy categorizing architectures by how they handle unreliable sensors, revealing 70% (21/30) use static weighting despite dynamic conditions; (2) a unified restoration-fusion framework analyzing 8 integrated approaches; (3) critical benchmark evaluation of 12 datasets showing only 3 provide severe asymmetric conditions; and (4) a prioritized research roadmap toward weather-agnostic foundation models. Our synthesis establishes that weather-robust perception requires co-evolution of restoration and fusion, not separate optimization, providing both a definitive reference and strategic guidance for next-generation autonomous perception systems.
The security of Unmanned Aerial Vehicles (UAVs) is compromised by the UAVCAN protocol’s inherent vulnerabilities and the computational constraints that preclude resource-intensive Machine Learning (ML) and Deep Learning (DL) detection solutions. This paper proposes a novel rule-based Intrusion Detection System (IDS), specifically engineered for the UAVCAN bus, which we designate LUMI (Lightweight UAVCAN Message Window-Based IDS). We first construct a new, robust UAVCAN dataset featuring sophisticated, watchdog-mimicking attack scenarios. We then demonstrate the dataset’s complexity by applying adapted ML/DL models from Controller Area Network (CAN) literature, which show significant performance degradation. LUMI addresses this challenge by enforcing sequential Transfer ID (TID) continuity within a sliding message window. Experimental results demonstrate that LUMI achieves a near-perfect F1-score of 0.9999—matching the performance of the most complex models—while maintaining linear time complexity and delivering a 16ms detection latency. This resolves the critical performance-complexity trade-off, validating LUMI as a practical and effective solution for real-time, resource-constrained UAVCAN systems.
Automated vehicles need to drive at least as well as humans using sensors, data, and machine intelligence. Therefore, this paper investigates data quality and its impact on upstream machine learning. Clearly, poor data increases the risk of hazardous situations and requires additional cycles in the drive-gather-process-assess process, increasing costs and probably delays. A complicating factor for data quality is adverse weather, prevalent in Nordic conditions.This work introduces a novel measure for quantifying the data quality. Akin to Technical Readiness Levels, Data Readiness Levels have been developed within this work to quantify data quality. The idea is a simple overall measure of the dataset, ranging from 1-9. As well as a framework, DRLs normalise values and metrics from various assessment tools.To evaluate the concept, three autonomous driving datasets were used, augmented with additional weather conditions not present in some scenes. Two of the datasets are from open-road driving scenes, and one from a test track. Images were further distorted with rain and fog, and point clouds with fog. All source code is available and, one data set is, with the other two part of a larger release.We show that a simple, single metric can be derived from 46 image and 2 point cloud quality metrics tested across nearly 5 TB of data. Through considerable statistical rankings, two image quality metrics performed well, at some computational cost, whilst state of the art point cloud metrics were inconclusive across our data.
Narrow-space parallel parking is one of the most challenging tasks in autonomous driving, attributed to limited maneuvering space and strict kinematic constraints. Conventional rule-based planners (e.g., Reeds–Shepp (RS) curves) can generate feasible trajectories but suffer from high computational costs and limited adaptability in narrow environments. Recent reinforcement learning (RL) approaches offer enhanced flexibility, yet their performance degrades under severe spatial constraints and nonholonomic vehicle dynamics. To address these issues, this work proposes a learnable RS-integrated reinforcement learning framework. First, a learnable RS cost field based on attention networks is developed to encode kinematic feasibility and provide soft heuristic guidance for reward design. Then, by fusing the RS-guided reward with a direct parking reward under a dual-critic architecture, the framework effectively integrates rule-based priors with the adaptability of RL. Experimental results show that the proposed learnable RS field accurately generates RS cost distributions with a 161× computational speedup. Compared with baselines, the RS-RL framework improves the success rate by 8.7% and reduces the travel distance by 20.3%. Particularly for narrow parallel parking spaces with a length of only 1.3 times the vehicle length (6.5m), our method attains a 97.8% parking success rate with adequate safety margins, demonstrating superior safety performance and adaptability to narrow parking scenarios.
Applying game theory to model lane-change interaction between human-driven and automated vehicles in mixed traffic is not new. Yet many studies treat the game as a purely mathematical task and overlook key cognitive steps in human social behavior. This work presents a behavior planner that extends game theory with explicit models for lateral movement and longitudinal acceleration. The planner covers the full lane-change process and reflects psychological mechanisms that guide human drivers during decision and preparation. We evaluated the planner in mandatory and discretionary lane changes under three driving styles. Results showed that the planner captured the main features of human decision making and carried out lane changes in a stable and consistent way.
Timely detection of vulnerable road users (VRU) remains one of the toughest safety challenges as driving tasks shift from humans to automated systems. Beyond factors such as environmental visibility and road conditions, specific perceptual and behavioral characteristics of vulnerable road users could also compromise driver hazard perception. This study focuses on motion camouflage, a stealth strategy seen in animals in which a pursuer moves in such a way that its image on the target’s retina mimics the motion of a stationary object, thereby minimizing the optical cues that usually draw attention. Motion camouflage in driving scenarios refers to situations where, e.g., specific VRU movements make them less detectable to drivers. A virtual reality driving simulator was used to study how driver braking responses are affected by VRU motion trajectories under varying environmental conditions. Results show that VRU motion camouflage constitutes a fundamental gap in hazard perception, posing a risk that could potentially result in delayed detection and increased risk of accidents. Based on these findings, this paper provides a basis for developing more trustworthy advanced driver-assistance systems (ADAS) capable of recognizing and mitigating risks due to such perceptual vulnerabilities.
This paper presents a Koopman Operator-based Model Predictive Control (KMPC) framework for emergency maneuvers that incorporates nonlinear tire-road interactions to obtain solutions to harder situations characterized by shorter time-to-collision and distance. Current autonomous driving strategies struggle to strike a balance between computational efficiency and model fidelity. Previous KMPC algorithms for autonomous driving have demonstrated success at low slip angles, but are not generalizable for large slip angles that are often encountered during emergency maneuvers at high speeds. In contrast, nonlinear MPC strategies have been employed in many safety-critical applications, but struggle to meet real-time constraints as tire nonlinearities increase. Our approach for the first time expands the capabilities of KMPC methods to large slip ranges, allowing for the execution of aggressive lane-switching maneuvers with an order-of-magnitude solve time reduction compared to nonlinear methods. To generalize KMPC performance to a broader range of slip angles, Latin Hypercube Sampling is used to generate the training data with an evenly spread slip angle distribution, resulting in capabilities that KMPC formulations regressed from other considered training generation methods could not achieve. We compare the performance and computational efficiency of the proposed KMPC solution with locally linearized and nonlinear MPC algorithms as state-of-the-art benchmarks. Leveraging the global linearization properties and corresponding offline initializations that Koopman formulations can offer, the proposed framework performs double-lane-change maneuvers at speeds and distances similar to those of nonlinear methods, while achieving the lowest solve time of all methods considered.
Trailer sway, observed in experiments, is investigated in this work. A single-track nonlinear model is established which captures the tractor-trailer interaction and accommodates a time-dependent velocity profile. The nonlinear behavior is analyzed using bifurcation analysis and speed ranges of bistable behavior are identified where trailer sway may be induced by applying large enough perturbations. It is demonstrated that via appropriate steering control one may push these dangerous bistable regions to higher speed ranges, but this strategy is sensitive to the time delay in the control loop. It is shown that braking with appropriate deceleration may eliminate trailer sway but the vibration amplitude may increase significantly during braking. We demonstrate that there is an optimal way of combining braking and steering in order to mitigate the sway.
Semantic segmentation is essential for reliable scene understanding in autonomous driving, but generalization across unseen domains remains a major challenge. Domain Generalization (DG) addresses this issue by learning domain-invariant representations without access to the target domain. Image transformations, e.g. to the frequency space, are a popular technique in DG to obtain more domain-invariant input modalities. However, existing approaches lack in utilizing both the original RGB-modality and the domain-invariant auxiliary modality effectively together. For this reason, we propose a dual-encoder architecture that performs hierarchical feature fusion via cross-attention in an intra-modal setting to effectively harness both modalities. It combines RGB features with auxiliary representations derived directly from the input image, such as the Fast Fourier Transform (FFT), Discrete Cosine Transform (DCT), edge maps, and color transformations (HSV, Lab). The two encoders are fused repeatedly across multiple hierarchical levels via cross-attention and a learned gating mechanism, enabling effective integration of complementary information without requiring external sensors or additional data sources.Evaluated in the synthetic-to-real setting, our approach demonstrates improved robustness to domain shifts in several target-domain settings and achieves competitive or higher segmentation performance depending on the target domain, highlighting the potential of hierarchical intra-modal fusion for domain-generalized scene understanding in autonomous driving.
In the age of massive vehicle instrumentation and the growing use of telematic boxes that continuously transmit sensory data over the air, it has become inexpensive to monitor the condition and behavior of various critical car components. It also opened wide prospects for detecting early symptoms of malfunctions and, in turn, counteracting them. In this work, we demonstrate an unsupervised learning approach for building a domain-agnostic, auto-encoder-based anomaly detector. It can consume any multidimensional telematic signal carrying working parameters of a vehicle and transform it into an alert if the observed pattern sufficiently deviates from normality. Next to being feature extraction free, the advantage of the proposed solution is that it does not require historical failure records for normality modeling, but only post-training anomaly score threshold tuning, which we demonstrate in two real-life scenarios.
Imperfect prediction of driving environments imposes significant challenges to motion planning for autonomous road vehicles. Contingency planning addresses these uncertainties by maintaining multiple candidate trajectories that share the same initial segment and branch later to account for different evolutions of the driving environment. Existing approaches typically determine the branching time of the contingent trajectories heuristically, which may lead to conservative or even infeasible plans. We propose a mixed-integer programming formulation of contingency planning that optimizes the branching time as a decision variable. Our approach is implemented within a receding-horizon planning framework and evaluated using the CommonRoad benchmark suite. Numerical experiments demonstrate that optimizing the branching time ensures feasibility while improving the quality of contingency plans. Our contingency planner is published as open-source code at https://github.com/CommonRoad/miqp-contingency-planner.
Longitudinal vehicle motion control is essential for enhancing performance and optimizing a vehicle’s energy usage. However, it remains a challenging task due to the nonlinear and uncertain nature of vehicle dynamics, along with varying driving conditions. This paper presents a novel ultra-local optimal control approach based on Pontryagin’s Minimum Principle (PMP) that circumvents the need for detailed system identification by employing an ultra-local model. The control objective is to minimize the total energy consumption under boundary conditions while ensuring smooth traction force generation. The proposed approach is evaluated using a high-fidelity vehicle model in three representative scenarios: (i) nominal driving, (ii) a change in tire road friction coefficient (TRFC) from 0.5 to 0.65 and road slope from 0% to 5% during the maneuver, with target velocity unchanged, and (iii) a change in target velocity from 20 m/s to 0 m/s during the maneuver, while maintaining nominal TRFC and slope conditions. The simulation results demonstrate that the proposed method delivers robust performance, effectively balancing consumption and tracking accuracy in all tested scenarios.
Autonomous Vehicles (AVs) and Advanced Driver Assistance Systems (ADAS) rely on extensive scenario based testing to ensure reliable performance under diverse environmental and traffic conditions. However, traditional test design is labor intensive and often insufficient for capturing rare but safety critical edge cases such as sudden pedestrian crossings, poor visibility, or sensor perception failures in adverse weather. This paper presents EC-Detector (Edge Case), an AI assisted simulation framework that automates edge case discovery through a closed loop feedback process connecting scenario generation, simulation, and post-simulation analysis. The framework leverages large language models (LLMs) to iteratively refine scenario parameters based on simulation outcomes, enabling systematic exploration of high-risk conditions within the defined Operational Design Domain (ODD). Implemented in the CARLA simulator with an Automated Emergency Braking (AEB) feature under test, EC-Detector identifies braking failures caused by varying traffic dynamics, environmental factors, and actor behaviors. Experimental evaluation shows that EC-Detector achieved an edge-case rate of 61.6%, outperforming manual (32.9%) and random (26.9%) generation methods. These results confirm that feedback guided optimization enhances scenario diversity, test coverage, and edge case discovery, establishing EC-Detector as a scalable and effective approach for AI driven validation of ADAS and autonomous systems.
The Operational Design Domain (ODD) defines the conditions under which an Automated Driving System (ADS) can safely operate. Validating the ODD requires linking its taxonomy concepts to data that represent real operational environments. Current Operational Domains (CODs) provide this link by describing environmental conditions at specific times and locations using ODD-relevant taxonomy concepts. However, systematic methods for defining CODs and their data requirements are still lacking. This paper introduces the novel formal concept of Minimal Taxonomy Sets (MTS), which represent the smallest sets of taxonomy concepts required within a COD to assess whether an ADS operates inside its ODD. We propose a structured approach for identifying these MTS from nested ODDs and present a variant of the Recursive Prime Implicant Enumeration algorithm that enables their extraction regardless of ODD structure or complexity. We illustrate the effectiveness of the approach using ODDs of varying complexity. Our work shows that MTS provide a principled foundation for generating ODD-aligned CODs, supporting scalable, data-driven validation of ADS.
In autonomous head-to-head racing, maximizing hardware performance and executing collision-free overtaking maneuvers are critical for achieving competitive lap times. Model Predictive Control (MPC) is a powerful framework for such tasks, however, its reliance on a single set of parameters makes it difficult to adapt to diverse driving scenarios. This paper presents a Reinforcement Learning (RL)-guided Switching Model Predictive Control (SMPC) framework designed to address this challenge. The proposed system defines three types of specialized MPC controllers, each optimized for a different racing strategy. An RL-guided decision-making module dynamically selects the most suitable controller based on the current driving context. The reward function is carefully designed to consider overtaking success, collision avoidance, smooth control, and overall racing performance. Experiments conducted in the F1TENTH GYM simulator demonstrate that the proposed method achieves a 34% improvement in overtaking success rate compared to a baseline.
This paper presents a safety-critical lane-keeping controller for automated vehicles subject to actuator delay, modeled as a first-order lag. Such lags, arising from the steer-by-wire system, can significantly degrade safety but are often ignored in prior studies. To remedy this, we propose a lag-compensating control barrier function (LCCBF) by applying a state transformation to the CBF designed for the lag-free vehicle dynamics. Using LCCBF, we design a lag-aware safety filter that provides formal safety guarantees for lane keeping under lag. Comparative simulations demonstrate that, unlike high-order and backstepping CBFs, the proposed method features a simple formulation without additional parameters while consistently maintaining safety with less conservatism and control effort, ensuring robust and comfortable lane-keeping performance.
We propose an accident damage minimizing controller to mitigate collision damage during unavoidable accidents. Our damage minimizing controller consists of high-level decision making enforced by low-level actuation. The decision making level will determine the vehicle’s inability to avoid an imminent collision, which is based on our novel danger zone algorithm. If another vehicle enters the danger zone of the ego vehicle, indicating the risk of unavoidable collision, the damage minimizing (Model Predictive) controller will look up a Collision Impact Table, which includes the quantified collision impacts of different speeds and angles to attain the minimum possible impact during the rest of the drive while ensuring that the vehicle does not slip or roll due to excessive steering. We show an implementation in a driving simulator.
Reliable lane-changing decision-making remains a major challenge for autonomous driving, particularly in interactive and constrained scenarios such as mandatory merging. Existing rule-based methods and reinforcement learning models trained only in synthetic environments often generalize poorly to real-world driving. This work proposes an improved Generative Adversarial Imitation Learning (GAIL) framework that learns lane-changing policies directly from expert demonstrations without handcrafted reward functions or strong assumptions about traffic dynamics. We first introduce a high-precision mandatory lane change dataset captured by Tesla Full Self-Driving vehicles, providing the expert trajectories that form the foundation of our imitation learning framework. To improve training stability and generalization, the GAIL architecture incorporates dropout-based regularization, a variational discriminator bottleneck, and a leaky Proximal Policy Optimization objective. Trained in a simulation environment aligned with the field setup, the resulting policy closely reproduces expert behavior and achieves smooth, robust lane changes across diverse traffic conditions. This study demonstrates how combining structured real-world data with principled imitation learning enables scalable and reliable autonomous decision-making.
Semantic image segmentation provides extremely granular image understanding for intelligent and autonomous vehicle platforms. A major factor in the development of these algorithms comes courtesy of the sheer scale of labeled data publicly available. However, more niche domains, such as rail vehicles, face a shortfall of publicly available data. In this paper, we address shortages of semantically labeled data in such thinly-sampled domains by adapting modern diffusion based generative foundational models. Our technique, which we call Joint Augmented ControlNet-LoRA (JACL), combines two widely used diffusion fine-tuning methods in a novel manner to jointly bias the network towards the target domain. JACL differs from existing techniques which aim to label synthetic imagery during or after generation; instead, JACL flips the problem on its head, using the desired ground truth labels as the input, resulting in imagery that closely conforms with the input semantic labels. Furthermore, JACL can easily generate synthetic imagery in various weather and lighting conditions by modifying input text prompts, with no additional training. We qualitatively and quantitatively demonstrate how the output imagery from JACL conforms to the desired scene structure while exhibiting statistically significant differences that can augment downstream algorithm development and testing.
