Ensuring the safety of autonomous vehicles requires addressing not only hardware and software failures, but also functional insufficiencies arising from limitations in perception, decision-making, and sensor performance. While traditional Fault Tree Analysis (FTA) provides a systematic framework for identifying failure pathways, it does not account for the dynamic dependencies and uncertainties inherent in real-world driving scenarios. To overcome these limitations, this paper proposes an integrated methodology that combines FTA with BN from the perspective of the Safety of the Intended Functionality (SOTIF). The proposed framework enables both qualitative and quantitative safety analysis, capturing logical structural as well as probabilistic dependencies among triggering conditions, functional insufficiencies, and failure modes. A case study on object detection failure in autonomous vehicles is conducted to demonstrate the applicability of the methodology. The results indicate that environmental factors such as weather and occlusion are major contributors to object detection failures, which contributes to 45.76% and 58.72%, respectively. Additionally, posterior probability analysis highlights the influence of specific variables and provides actionable insights for prioritizing mitigation strategies. This study contributes a novel framework that enhances safety assessment capabilities in SOTIF contexts and provides system designers with practical guidance to improve the reliability and robustness of autonomous vehicle systems.
This paper studies the Fair Sequenced Group Trip Planning Query Problem (Fair-SGTP) , a variant of the Group Trip Planning Problem that arises in autonomous and intelligent transportation systems supporting shared group-based mobility services. Fair-SGTP focuses on selecting optimal sequences of Points-of-Interest in a road network to minimize total travel distance while ensuring fairness among group members, which is an increasingly important requirement in autonomous ride-sharing, tourist routing, and collaborative mobility planning. Despite the NP-hard nature of classical GTP queries, we present polynomial-time algorithms for Fair-SGTP under multiple fairness constraints. Drawing inspiration from fair resource allocation problems, we investigate three fairness notions: minimum-envy path, minimax-distance path, and ϵ-proportional path, and adapt them to the group trip planning context. We introduce the Price of Fairness (PoF) to quantify the trade-off between efficiency and fairness in autonomous route planning. Our theoretical analysis shows that while the PoF can be unbounded for general solutions, it is bounded for Pareto-optimal solutions by (2 b − 1) for minimum-envy path, b for minimax-distance path, and (2 b − 1) for ϵ-proportional path with b agents, and these bounds are provably tight. We evaluate the proposed algorithms using real-world urban road network datasets, demonstrating superior performance compared to existing approaches. Excluding the one-time shortest-path preprocessing step, the algorithms compute fair routes in \(\mathcal {O}(bn^4 + (kn)^3) \) time with k being the number of categories, and experimental results indicate that the PoF remains below 2 across all tested city networks and fairness criteria. These results highlight the practicality of Fair-SGTP for deployment in autonomous transportation systems requiring equitable and efficient group decision-making.
The core challenge of autonomous driving is achieving real-time decision-making and safety control in complex, dynamic environments, especially in long-tail scenarios where conventional systems often fail due to limited adaptability, poor human-vehicle interaction, and low interpretability. This paper introduces MLLM-CLAD, a closed-loop decision-making framework powered by a multimodal large language model, which addresses semantic gaps and feedback limitations through a co-optimization strategy that integrates behavior-state alignment and language instruction fusion. MLLM-CLAD integrates three core modules: (1) The Mapping-Enhanced Language-State Alignment Module employs a deterministic mapping pipeline from language instructions to control signals, which enables highly accurate mapping from LLM semantic outputs to executable control signals. (2) The Hierarchical Dynamic Instruction Integration Module robustly categorizes and dynamically prioritizes human language instructions into different semantic types. (3) The Multimodal Fusion Module leverages a lightweight spatiotemporal Q-Former to jointly encode images and LiDAR point clouds, aligned with language in latent space for improved temporal reasoning and cross-modal consistency. Evaluated in the CARLA Town05 Long, MLLM-CLAD achieves a 78.3% Driving Score and reduces collision rates by 35% in safety-critical scenarios. On the LangAuto benchmarks, MLLM-CLAD attains a 78.5% long-tail scenario pass rate. Furthermore, the end-to-end latency is reduced to 128 milliseconds. This work establishes a scalable path toward safe, low-latency, and interpretable autonomous driving in simulation, with a clear roadmap for real-world extension.
In this study, we present the design and evaluation of PlenaMap—a natural language-supported geospatial learning tool aimed at closing the gap between student learners utilizing geospatial learning tools. To achieve this, we constructed a scaffold based on Wenger's “windows” construct and integrated a large language model. We then evaluated its effectiveness in supporting high school students in Puerto Rico as they identified and engaged with local problems using geospatial data and visualizations. The evaluation was conducted through two independent workshops with six students from different regions of Puerto Rico. Using conjecture maps, we detail the design process, user interactions, and intended outcomes to provide readers with a holistic view of the design work. Through thematic analysis, we highlight students' interactions and experiences with PlenaMap. We found that students engaged with the tool in ways consistent with geospatial practices (e.g., asking geographic questions, gathering geographic data). Students reported that the natural language-based interface made problem-solving and the research process more approachable and less intimidating. We conclude by discussing the design implications of LLM-based geospatial learning tools as scaffolds to better support the needs of high school students.
Understanding the real-time behaviour of pedestrians and their surrounding context is essential for road safety. Unfortunately, current perception systems struggle to capture how multiple cues jointly influence behavior in diverse traffic scenarios. Most existing models in Intelligent Transportation Systems (ITS) and Autonomous driving system (ADS) predict only a single attribute, such as intent or trajectory, while overlooking the interaction between behavioral and environmental factors. We propose a unified multi-attribute framework based on a Vision Transformer, which is enhanced with lightweight, parameter-efficient adapters. This innovative modular design enables independent learning of eight complementary attributes, including pedestrian behavior (action, attention, crossing, occlusion) and scene context (weather, time of day, road presence, density). Our proposed framework updates fewer than 2% of the model parameters. We ran our proposed framework using the JAAD dataset, and the framework achieves over 90% accuracy on most attributes and exhibits consistent behavior–context relationships and demonstrates robustness under varied environmental conditions. To evaluate real-time suitability, we implemented an adaptive strategy that activates only the most relevant adapters based on scene conditions. This approach reduces computational cost by 28.5% and increases throughput by 40.8% without loss of accuracy. This research distinctly highlights the practical potential of modular, PEFT-enhanced transformers for scalable multi-attribute perception as foundational vision models continue to advance. The best-performing adapters and inference code are available on Github Repo.
Highly Automated Vehicles (HAVs) and Advanced Driver-Assistance Systems (ADAS) are transforming modern transportation with enhanced mobility, safety, and efficiency. Despite their advantages, cybersecurity vulnerabilities in these systems can lead to abnormal behavior, posing significant risks to surrounding human-driven vehicles (HDVs) in mixed traffic environments. This article addresses the challenge of detecting abnormal lateral movements of HAVs/ADAS vehicles using only trajectory profiles of following HDVs. Specifically, we propose a novel modeling approach that captures both normal and abnormal lateral behaviors through vehicle kinematics, integrated decision-making processes, vehicle control using symbolic regression for lane change vehicles. Additionally, we introduce an abnormality detection framework that relies on observable HDV data, even in occlusion scenarios. The framework evaluates the sensitivity of various car-following models to detect abnormal behaviors, providing insights into the interaction between HAVs/ADAS and HDVs in mixed autonomy systems.
As autonomous vehicles continue to evolve and integrate into modern transportation systems, ensuring their safety in uncertain and dynamic environments remains a critical challenge. Fault Tree Analysis (FTA) is widely applied in autonomous vehicle risk assessments, offering structured insights into system failure. However, traditional FTA mainly address hardware and software malfunctions, while hazards arising from functional insufficiencies remain less systematically explored. The Safety of the Intended Functionality (SOTIF), outlined in ISO 21448, focuses on these non-fault-based hazards. There is a lack of surveys that thoroughly investigate FTA within the SOTIF context to comprehensively address both malfunction-induced and functionality-induced risks. It is valuable to bridge the traditional risk analysis with SOTIF-specific hazard evaluation through a dedicated review. This survey reviews the fundamentals and key challenges of autonomous vehicle systems and FTA, and discusses how FTA can be adapted to support SOTIF-based safety assessments. Qualitative and quantitative FTA methods are summarized, along with advanced extensions like Bayesian Networks and Dynamic FTA. The survey highlights the critical research trends in integrating FTA with SOTIF for autonomous vehicle safety assurance. Finally, current challenges, opportunities and future directions of use of FTA within the SOTIF framework are outlined to enhance risk analysis frameworks for next-generation autonomous vehicles.
Teams of unmanned ground vehicles (UGVs) and drones are often proposed for various applications in autonomous transportation, where drones quickly move from one point of interest to the next while UGVs act as moving base stations that can both recharge and ferry the drones around. In this paper, we look at how to plan collaborative actions among multiple teams of energy-sharing drones and UGVs for patrolling applications over an indefinite time horizon. We demonstrate how to form a second-order cone (SOC) program that finds optimal solutions, in polynomial time, when the order of drone and UGV actions are fixed. We propose an algorithm that uses various heuristics and our SOC program to find locally optimal solutions while considering the limited energy of both vehicle types. Additionally, we present a new nomenclature for classifying drone-UGV planning problems and propose a novel metric for evaluating multi-agent patrolling plans. We ran various numerical simulations using field data to evaluate our proposed approach. Our algorithm improves solution quality by up to 37.7% compared to a baseline method from the literature. Furthermore, we demonstrate the authenticity of our problem setup through a proof-of-concept experiment on a physical UGV and drone testbed.
Trajectory prediction is a key element of autonomous vehicle systems, enabling them to anticipate and react to the movements of other road users. Evaluating the robustness of prediction models against adversarial attacks is essential to ensure their reliability in real-world traffic. However, current approaches tend to focus on perturbing the past positions of surrounding agents, which can generate unrealistic scenarios and overlook critical vulnerabilities. This limitation may result in overly optimistic assessments of model performance in real-world conditions. In this work, we demonstrate that perturbing not just past but also future states of adversarial agents can uncover previously undetected weaknesses and thereby provide a more rigorous evaluation of model robustness. Our novel approach incorporates dynamic constraints and preserves tactical behaviors, enabling more effective and realistic adversarial attacks. We introduce new performance measures to assess the realism and impact of these adversarial trajectories. Testing our method on a state-of-the-art prediction model revealed significant increases in prediction errors and collision rates under adversarial conditions. Qualitative analysis further showed that our attacks can expose critical weaknesses, such as the inability of the model to detect potential collisions in what appear to be safe predictions. These results underscore the need for more comprehensive adversarial testing to better evaluate and improve the reliability of trajectory prediction models for autonomous vehicles.
Low-latency traffic prediction is vital for smart city traffic management. Federated Learning has emerged as a promising technique for Traffic Prediction (FLTP), offering several advantages such as privacy preservation, reduced communication overhead, improved prediction accuracy, and enhanced adaptability to changing traffic conditions. However, majority of the current FLTP frameworks lack a real-time model updating scheme, which hinders their ability to continuously incorporate new incoming traffic data and adapt effectively to the changing dynamics of traffic trends. Another concern with the existing FLTP frameworks is their reliance on the conventional FL model aggregation method, which involves assigning an identical model (i.e., the global model) to all traffic monitoring devices to predict their individual local traffic trends, thereby neglecting the non-IID characteristics of traffic data collected in different locations. Building upon these findings and harnessing insights from reinforcement learning, we propose NeighborFL, an individualized real-time federated learning scheme that introduces a haversine distance-based and error-driven, personalized local models grouping heuristic from the perspective of each individual traffic node. This approach allows NeighborFL to create location-aware and tailored prediction models for each client while fostering collaborative learning. Simulations demonstrate the effectiveness of NeighborFL, offering improved real-time prediction accuracy over three baseline models, with one experimental setting showing a 16.9
This paper introduces the architecture and real-world deployment of a collision avoidance system based on shared perception between vehicles and infrastructure in autonomous mobility. It employs microservices on both the infrastructure and vehicle platforms to integrate data from multiple sensors and distributed computing units. By leveraging Vehicle to Infrastructure communication, the system autonomously interfaces with vehicle’s systems in critical emergency scenarios, such as executing an emergency stop triggered by the presence of Vulnerable Road Users (VRUs), signaled by a roadside unit. Edge computing at key points enhances the system’s capabilities, enabling swift object detection through computer vision and real-time integration with the vehicle’s interfaces via Vehicle to Infrastructure message exchanges. The system is integrated in both fully autonomous and semi-autonomous vehicles, performing emergency stops when a hazard is detected. Depending on the vehicle, it either returns control to the driver or allows the autonomous control system to resume operation. The integration with Autoware, the autonomous control system for the fully autonomous vehicle used in this work, highlights the system’s ability to seamlessly integrate vehicular communications into autonomous mobility. It cooperates with Autoware using information from surrounding entities to execute emergency stop maneuvers without disrupting autonomous driving. Once the maneuver is completed and the hazard is cleared, the vehicle resumes its path without manual intervention. The experimental results with both fully autonomous and semi-autonomous vehicles, in the presence of Vulnerable Road User (VRU), demonstrate the system’s ability to deliver timely alerts and ensure successful vehicle responses. Additionally, a comprehensive timing analysis of the system’s performance is conducted across both vehicles and infrastructure.
With recent advances in computing and sensing technologies, autonomous driving has gained increasing interest and become a promising platform to support the next generation intelligent transportation systems. A critical requirement for autonomous driving systems is to be able to utilize AI and machine learning techniques to make reliable decisions on edge devices in a timely manner. Deploying reliable machine learning models on edge devices in a real-time environment is a challenging task. Real-time applications such as traffic surveillance or traffic sign detection require consistently low latency in order for the device to keep up with its environment. Edge devices are able to compute machine learning tasks without needing to offload computation to a cloud server, however they often have limited resources which present challenges for computationally intensive deep learning applications. Therefore, the optimization of neural network models for autonomous driving applications is pivotal for real-time performance on resource-constrained edge devices. In this paper, we present a comprehensive study on utilizing deep learning optimization techniques to enable efficient and effective decision-making for autonomous driving applications. Our contributions include the implementation of channel and fine-grained pruning on YOLOv8, direct optimization of detection layers, and the integration of INT8 quantization using NVIDIA TensorRT. These methods significantly improve computational efficiency while preserving the model accuracy. Experimental evaluations on the Jetson Orin Nano demonstrate significant improvements in inference speed and memory utilization with minimal accuracy degradation. This work highlights the feasibility of deploying state-of-the-art object detection models in resource-constrained autonomous driving systems.
The Internet of Drone Things (IoDT) advances autonomous drone operations by integrating live sensor inputs with environmental and situational awareness and intelligent decision-making capabilities. The full capabilities of IoDT remain limited by the difficulties of dynamic task assignment and path optimization, along with adaptive decision-making, when operating in complex environments such as disaster relief and smart agriculture. Traditional task-scheduling techniques have difficulty adapting to real-time changes caused by dynamic constraints such as weather variations, battery limitations, and drone malfunctions. We present an LLM-based task scheduling framework that uses Large Language Models (LLMs) to improve task prioritization performance and path planning accuracy while minimizing operational failures. We combine heuristic algorithms (A*, Dijkstra) with decision-making processes driven by LLMs to allow drones to adapt to environmental changes while optimizing efficiency and resource consumption. Integrating LLM technology into IoDT operations results in up to 95% task completion rates and improves the scenario completion time by up to 42%, while adding reasonable computational overhead. Our framework demonstrates improved task adaptability, battery efficiency, and stronger system resilience against non-LLM baselines during disaster relief and package delivery operations. Our research shows that LLM-based IoDT task management has transformative potential, leading to the development of more innovative and autonomous drone ecosystems.
In VANET, although data sharing among vehicles improves safety and traffic, malicious data providers might share false information to compromise its effectiveness. One typical solution to address this issue is to manage the trustworthiness of data providers by using feedback received from data users quantifying the veracity of the shared data. However, data users might behave maliciously and share untruthful feedback. This article introduces TrCoin, a blockchain-based trust management system, for VANET which tackles both fake data and untruthful feedback. TrCoin uses an honesty value calculation algorithm to distinguish honest from malicious data users and filters out the feedback shared by malicious data users. TrCoin then runs a trust value calculation algorithm to compute the trust values of data providers based on the truthful feedback shared by honest data users. Extensive simulations are conducted to demonstrate the effectiveness of TrCoin under various attack scenarios with false data injection and untruthful feedback.
Visual perception is crucial for autonomous vehicle navigation, enabling situational awareness through depth estimation and semantic segmentation. This study presents a novel transformer-based pipeline for depth perception and semantic aware path planning for enhanced visual understanding. Our approach combines a dense prediction model for relative depth estimation and semantic information with a causal transformer and Q-learning for navigation. We evaluate the pipeline’s performance using metrics such as Root Mean Square Error and Depth Accuracy alongside safety-based indicators. Qualitative assessments of agent behavior in diverse simulated environments and real-world scenarios demonstrate improvements in autonomous vehicles’ contextual understanding of surroundings. Our results demonstrate that the proposed pipeline enhances obstacle anticipation and vehicle responsiveness in simulated and real-world environments. By improving the reliability of autonomous navigation systems, this work contributes to SDG 9 by advancing resilient infrastructure for intelligent transportation while supporting SDG 11 through safer and more efficient urban mobility solutions.
In this paper, we address the issue of learning fair policies in decentralized cooperative multi-agent reinforcement learning (MARL), with a focus on traffic light control systems. We show that standard MARL algorithms that optimize the expected rewards often lead to unfair treatment across different intersections. To overcome this limitation, we design control policies that optimize a generalized Gini welfare function that explicitly encodes two aspects of fairness: efficiency and equity. Specifically, we propose three novel adaptations of MARL baselines that enable agents to learn decentralized fair policies, where each agent estimates its local value function while contributing to welfare optimization. We validate our approaches through extensive experiments across six traffic control environments with varying complexities and traffic layouts. The results demonstrate that our proposed methods consistently outperform existing MARL approaches both in terms of efficiency and equity.
We consider the problem where a warehouse has multiple drones and wants to deliver parcels to several customers. Each drone picks a parcel from the warehouse, delivers it and returns to the warehouse. The speeds and battery lives of the drones could be different and due to the limited battery life, each drone has a bounded range in which it can deliver parcels. The goal is to assign parcels to the drones so that the time taken to deliver all the parcels is minimized. This turns out to be an NP-hard problem, and hence, the focus of this paper is to design a fast approximation algorithm. We do this by adapting the Longest Processing Time First (LPT) heuristic, which is a classical heuristic for the fundamental problem of uniform machine scheduling to the Drones Warehouse Problem (DWP), which is a natural generalization of the uniform machine scheduling problem. We prove that the natural approach of solving this problem via the LPT heuristic has an approximation factor of ϕ, where ϕ ≈ 1.62 is the golden ratio. LPT heuristic was first considered by Gonzalez et al. (SIAM J. Comput. 6(1):155–166, 1977). Since then, extensive work has been done to improve the approximation factor of the LPT heuristic. However, all known implementations of the LPT heuristic take O(mn) time, where m is the number of machines and n is the number of jobs. In this work, we come up with the first near-linear time implementation for LPT both for DWP and Uniform Scheduling. Specifically, the running time is O((n + m)(log 2m + log n)). Somewhat surprisingly, the result is obtained by mapping the problem to dynamic maintenance of lower envelope lines, which has been well-studied in the computational geometry community.
With global population growth accelerating and food demand rising, the need for efficient farmland surveying to maximize crop yields through precision agriculture has never been more critical. Unmanned Aerial Vehicles (UAVs) have emerged as capable tools due to their advanced sensor capabilities, high mobility, and ability to collect data with minimal risk to both farmers and crops. While single UAV platforms offer significant potential, their effectiveness remains limited by factors such as restricted battery life, limited flight duration, high operational costs, among other aspects. This paper addresses these limitations by introducing a sensing- and constraint-aware, cooperative coverage framework designed for agricultural surveying using a heterogeneous team of UAVs. Our approach accounts for both the physical constraints of UAVs and the user-defined data collection needs, while minimizing the total mission time and energy consumption. The framework begins by defining the Region of Interest (ROI) as a convex polygon, which is then decomposed into subregions based on the relative sensing and flight capabilities of each UAV. Each partition is assigned to a specific UAV based on its relative capability, and coverage paths are generated using an efficient back-and-forth sweeping strategy. A novel receding horizon Mixed Integer Linear Programming (MILP) technique is then applied to produce collision-free trajectories tailored to each UAV’s constraints. We validated the framework within the CoppeliaSim physics-based simulator, where results demonstrate the effective trajectory coordination and full ROI coverage, confirming the system’s adaptability and performance under realistic conditions. The derived solutions will provide an opportunity for farmers to regularly and efficiently survey agricultural fields to maximize crop yields.
The massive deployment of soil moisture and temperature sensors inside agricultural plots will become essential in the next years to meet the challenges of the agroecological transition. In such a context, the use of buried communicating sensor nodes is particularly relevant because they are protected underground from human activities, animals and agricultural machinery. Their communication range is however limited to a few meters above the ground, that leads to difficulties in collecting their data remotely. One possible solution is to use one or several UAVs to visit them successively. Nevertheless, this approach requires being able to take into account the presence of obstacles at low flight height in the path planning algorithms, and to solve the Close Enough Traveling Salesman Problem (CE-TSP) in cluttered environments. To meet this need, the objective of this paper is to present the possibility of exploiting real data from an airborne Light Detection and Ranging sensor (LiDAR) to determine the location and height of obstacles in the workspace of the UAV. Based on this data, a cost matrix associated to the trajectories between each node can be calculated, with the possibility for the UAV to fly above the obstacle areas. Next, to optimize the route of the UAV in the CE-TSP framework, this paper presents a simple approach based on a Partheno Genetic Algorithm (PGA) completed with some heuristic rules. Our method is applied in a real use case consisting of 50 sensor nodes distributed in our experimental farm. The results highlight the performances of the method proposed. They demonstrate the capabilities of our method to address the CE-TSP with obstacles, and open the way to future prospects for collecting data of buried sensor nodes using UAVs.
