Autonomous driving has attracted great interest due to its potential capability in enhancing safety and improving traffic efficiency. Both model-based and learning-based methods are widely used in autonomous driving. Out of which, model-based methods rely on existing events in the dataset but are poor in learning extended situations. As a comparison, the deep Q-learning network (DQN) has a strong capability in learning within interactive driving. However, existing DQN faces challenges in convergence in terms of speed and accuracy, especially in interactive environments. Furthermore, the poor convergence causes high risks of collisions and slow driving speed. Therefore, this paper presents a modified DQN to achieve a lower number of collisions, higher average driving speed, and faster convergence during interactive driving. The modified DQN is developed by introducing a risk-attention mechanism, a balanced reward function, and a collision-supervised mechanism (RBDQN-CS). The proposed risk-attention mechanism enhances the DQN to pay attention to high-frequent interactions. The proposed balanced reward function specifies the weight of the control strategy to handle the interactions with surrounding human driven vehicles. The collision-supervised mechanism detects the collision risks and prevents the collision occurrence during lane-changing. Simulation results demonstrate that RBDQN-CS outperforms DQN and other popular baseline DRL algorithms.
Multi-Agent Reinforcement Learning (MARL) has demonstrated significant potential for cooperative decision-making in connected and autonomous vehicles (CAVs). However, existing approaches often fail to address the task-specific characteristics and varying requirements of vehicles in high-dynamic, unsignalized intersection scenarios. In these environments, vehicles are frequently exposed to conflict zones where risks, such as collisions, are difficult to perceive, particularly in the absence of traffic signals. Additionally, current methods lack effective mechanisms for balancing learning performance across multiple tasks. To overcome these challenges, we propose a novel multi-task MARL framework tailored for unsignalized intersections. The framework incorporates a hybrid-attention network that captures the influence of surrounding vehicles on different driving tasks, improving multi-agent decision-making. A multi-task diversity priority sampling mechanism is introduced to prioritize high-quality episodes from more complex tasks, enhancing performance in dynamic intersection settings. Furthermore, a risk-aware local decision corrector optimizes decision-making in high-risk conflict zones by enabling vehicles to predict and adapt to surrounding traffic behaviors. The proposed framework is evaluated through simulations, demonstrating superior performance compared to state-of-the-art baselines. A miniature intelligent vehicle testbed further validates its effectiveness and potential for real-world deployment.
In complex traffic environments, autonomous vehicles must simultaneously cope with multi-source potential risks arising from road boundaries, lane structures, and both dynamic and static traffic participants. These risks interact and overlap across spatial and temporal dimensions, increasing the safety challenges of trajectory planning. Existing methods face limitations in handling high-dimensional obstacle avoidance constraints and dynamic environmental factors, often resulting in degraded trajectory quality and reduced safety. To overcome these challenges, this paper proposes a safety-enhanced trajectory planning method, which improves trajectory quality through the recursive generation of the Spatio-Temporal Risk Map (STRM) and the Dynamic Safety Corridor (DSC). First, based on potential field theory, a risk modeling approach is applied to various traffic elements, such as road boundaries, lane markings, and both dynamic and static obstacles, to generate the STRM, thereby mapping the traffic environment information into a unified risk space. Subsequently, a method for constructing the DSC is proposed, considering boundary risk thresholds. This approach effectively decouples the relationship between complex environmental factors and driving tasks, thereby simplifying collision constraints. Finally, an Optimal Control Problem (OCP) is formulated, taking into account factors such as trajectory smoothness, comfort, and collision risk, in order to generate safe optimized trajectories. The effectiveness of the proposed method in various multi-lane scenarios was evaluated through simulation experiments. The results show that the proposed method significantly enhances the safety of trajectory planning, especially in highly dynamic and high-risk environments.
Visual localization is a critical method for achieving reliable train localization, especially in Global Navigation Satellite System (GNSS)-denied environments. However, its effectiveness is severely hampered in harsh operational conditions, such as extreme weather and low-light tunnels, which induce significant visual degradation. To address this fundamental challenge, we propose PI-VIMamba, a novel physics-informed visual-inertial mamba framework. The innovation is centered around a tripartite technical system that integrates a backbone network, a detection module, and a depth estimation module. Firstly, an adaptive channel convolution module is proposed to dramatically enhance feature representation under visual degradation, leading to superior object detection accuracy and more precise depth estimation. Secondly, a kinematics-constrained loss function is introduced leveraging Inertial Measurement Unit (IMU) data. This physical constraint effectively suppresses monocular visual estimation drift, thereby ensuring high-precision, long-distance localization capability even in visually degraded environments. Finally, an improved mamba block is designed for refined general feature extraction, which substantially reduces the network’s parameters and computational complexity while accelerating inference. Evaluated on real-world subway line data, PI-VIMamba demonstrably excels in harsh conditions, achieving state-of-the-art performance across key metrics, including detection accuracy, depth estimation precision, and computational efficiency, significantly outperforming existing mainstream models.
Metro trains often operate in highly complex environments, characterized by illumination variations, high-speed motion, and adverse weather conditions. These factors pose significant challenges for visual perception systems, especially those relying solely on conventional RGB cameras. To tackle these difficulties, we explore the integration of event cameras into the perception system, leveraging their advantages in low-light conditions, high-speed scenarios, and low power consumption. Specifically, we focus on Kilometer Marker Recognition (KMR), a critical task for autonomous metro localization under GNSS-denied conditions. In this context, we propose a robust baseline method based on a pre-trained RGB OCR foundation model, enhanced through multi-modal adaptation. Furthermore, we construct the first large-scale RGB-Event dataset, EvMetro5K, containing 5,599 pairs of synchronized RGB-Event samples, split into 4,479 training and 1,120 testing samples. Extensive experiments on EvMetro5K and other widely used benchmarks demonstrate the effectiveness of our approach for KMR. Both the dataset and source code will be released on https://github.com/Event-AHU/EvMetro5K_benchmark
With the rapid advancement of Vehicle-to-Everything (V2X) communication and automated driving technologies, connected and automated vehicle (CAV) platoons have emerged as a promising solution for enhancing road capacity, energy efficiency, and traffic safety. Unlike the ideal fully CAV environment, real-world traffic will remain in a mixed traffic phase for an extended period, where CAVs and human-driven vehicles (HDVs) coexist. Factors such as cut-in by surrounding vehicles, HDV leader disturbances, and communication delay significantly increase the complexity of platoon control. This paper provides a systematic review of CAV platoon control methods under mixed traffic from three interrelated perspectives: control architecture, HDV interaction mecha nisms, and control methods. From the architectural perspective, centralized management, distributed cooperation, and hybrid hierarchical architectures are comparatively examined in terms of decision efficiency, communication dependency, and scalability. Regarding HDV interaction mechanisms, HDV behavior modeling and prediction approaches based on model-driven, data-driven, and game-theoretic frameworks are synthesized, highlighting the trade-offs between modeling fidelity and uncertainty quantification capability. In terms of control methods, the technical characteristics and applicability limits of model-driven control, data-driven control, and hybrid model-data-driven control are systematically analyzed. Building upon this comparative assessment, the advantages, limitations, and application scenarios of these approaches are summarized to provide guidance for control strategy design and selection. Finally, future research directions for CAV platoon control under mixed traffic flow are outlined, aiming to provide theoretical support for establishing safe, stable, and efficient CAV platoons in complex mixed traffic environments.
Mixed platoons comprising connected automated vehicles (CAVs) and human-driven vehicles (HDVs) are subject to dual-layer disturbances arising from unpredictable traffic oscillations and stochastic communication delays. To address these disturbances, this study proposes Resilience-Physics Enhanced Residual Learning (R-PERL), a resilience-oriented control framework that integrates model predictive control with communication-quality-aware residual learning. R-PERL establishes a unified system model that couples knowledge-guided longitudinal dynamics with a comprehensive stochastic communication-delay model as a fixed foundation, ensuring inter-pretability and constraint satisfaction. A gating mechanism adaptively regulates residual learning to compensate for model-reality mismatch under degraded information quality, while resilience operations leverage aggregated HDV and macroscopic traffic information to prevent overreaction and accelerate disturbance attenuation. This architecture enables the controller to absorb oscillations, maintain core functionality during communication deterioration, and rapidly restore nominal conditions. Comparative simulations indicate that R-PERL outperforms baseline methods in string stability, ride comfort, tracking accuracy, and resilience-related metrics. During communication interruptions, performance degrades smoothly without abrupt collapse and resynchronizes rapidly after recovery. Across varying CAV penetration rates, consistent improvements in traffic operation and oscillation suppression are observed. Generalization validation over repeated random delay trials confirms bounded tracking behavior in the tested scenarios and supports safe operation with effective oscillation attenuation under the considered conditions.
Automated vehicles (AVs) are transforming urban transportation systems, as end-to-end autonomous driving models show great promise in enhancing traffic safety and operational efficiency. Despite these advances, their performance in highly interactive driving scenarios remains limited due to insufficient decision-making diversity and the absence of explicit safety guarantees. To address these challenges, we propose NavDrive, a safety-enhanced end-to-end autonomous driving framework that formulates planning as a multi-modal generative process.Specifically, NavDrive integrates navigation-based guidance into a diffusion policy. To focus on decision-critical information, a Decision-Aware Channel Fusion (DCF) module adaptively emphasizes regions involving key interactions between the ego vehicle and surrounding agents. Furthermore, a safety-aware generative planner refines trajectory samples toward feasible regions via the Target-Prior Diffusion Transformer (TDiT), which explicitly embeds physical constraints to ensure safe and human-aligned driving behaviors. Extensive experiments on the NAVSIM and nuScenes benchmarks demonstrate that NavDrive consistently outperforms existing baselines, delivering substantial gains in planning quality, safety, and robustness under complex and adverse conditions. The details will be available at https://github.com/zgchongbo/NavDrive
Virtual Coupling (VC) technology is expected to increase railway capacity and enable flexible coordinated train formations in complex scenarios. A representative VC operation occurs at track junctions, where trains merge and transition among coupling, coupled, and decoupled states. This paper investigates the problem of integrating the sideline train into mainline virtually coupled train set (VCTS) while maintaining coupling-state continuity, and proposes an MPC-based virtual pilot injection (VPI) strategy. A virtual pilot trajectory is planned, and distributed MPC controllers are designed under a leader-follower information framework to achieve smooth merging and recoupling. Robustness and stability are theoretically established to account for disturbances and unreliable communication. Extensive simulation results demonstrate that, compared with existing methods, the proposed approach significantly reduces the coupling time under both ideal operating conditions and scenarios with disturbances and delays, while maintaining effective safety protection even in extreme cases.
Unmanned aerial vehicles (UAVs) play a pivotal role in air-ground collaborative mobile edge computing (MEC) systems. They function as aerial cloudlets, deploying flexibly closer to ground users (GUs) to provide enhanced computational capacity in edge computing scenarios. While extensive studies have optimized resource allocation and UAV trajectories collaboratively to improve energy and offloading efficiency, few have simultaneously addressed the system's communication security and reliability. This paper proposes a joint optimization model to ensure both security and reliability in an energy-efficient UAV-assisted MEC system. Specifically, we introduce an artificial noise generation technique to enhance system security and derive a closed-form expression for the optimal ratio between the generated noise and the data transmission power to ensure secure communication. Additionally, we propose a probabilistic model to characterize the reliability of data transmission and derive the worst-case transmission rate. Furthermore, we present an energy-efficient model for optimizing resource allocation and UAV trajectory planning, with the goal of improving the overall energy efficiency of the UAV-assisted MEC system. Finally, we design an optimization algorithm with polynomial-time complexity based on the augmented Lagrangian multiplier method. Simulation results demonstrate that the proposed method outperforms existing approaches in terms of both global secure energy efficiency and average secure energy efficiency.
Unmanned aerial vehicle vision-language navigation (UAV-VLN) requires agents to translate visual observations and language instructions into reliable flight actions in complex environments. Although recent end-to-end UAV vision-language-action (UAV-VLA) policies reduce reliance on separately designed perception, planning, and control modules, their behavior-cloning objectives provide limited corrective supervision for interactive closed-loop execution. Reinforcement learning (RL) offers a promising solution, while its effectiveness is constrained by inefficient use of samples, long-tailed scene distributions, and policy distribution shift during optimization. To this end, we propose RecoverFly, a failure-aware RL post-training framework for end-to-end UAV-VLA policies. Specifically, RecoverFly adapts token-level RL for stable optimization of grammar-constrained autoregressive UAV actions, revisits unresolved failure cases to strengthen corrective learning and sample utilization, and combines a two-stage long-tail scene curriculum with reference-policy regularization to improve scene adaptation while preserving acquired capabilities. Experiments on the TravelUAV benchmark demonstrate that RecoverFly achieves the best performance on the seen, unseen-map, and unseen-object splits. Moreover, compared to the AerialVLA initialization, RecoverFly improves success rate by 3.12 to 8.37 percentage points under a total rollout budget of about 30% of the training-set size, validating its effectiveness, robustness, and generalization capabilities.
Deep Reinforcement Learning (DRL) has demonstrated significant potential in tackling the complex challenges of autonomous vehicles, particularly in decision-making and motion planning. However, many existing DRL methods optimize individual components in isolation, leading to a lack of integration across modules. Additionally, current reinforcement learning methods often focus on high-reward samples while neglecting the valuable information contained in low-reward samples. Furthermore, the efficiency of action search requires further improvement. To overcome these challenges, we propose a novel closed-loop DRL framework that utilizes feedback to simultaneously optimize both decision-making and motion planning. To address these, we propose a novel closed-loop DRL framework that leverages closed-loop feedback to optimize both decision-making and motion planning concurrently. The feedback framework refines the DRL process by utilizing the deviation between real and predicted trajectories, enabling continuous improvement of the system’s decision-making capabilities. The framework also incorporates a dual-value priority sampling strategy, which enhances the utilization of low-reward samples by prioritizing the sampling of high-value samples across categories. Additionally, the framework integrates a hybrid action selection mechanism that combines rule-based and learning-based mechanism, effectively reducing exploration inefficiencies in the early stages of DRL while maintaining long-term optimization through experience-driven learning. Experimental results demonstrate that the proposed framework outperforms state-of-the-art models, showing strong potential for real-world applications.
Autonomous aerial vehicles (AAVs) have gained considerable attention in data collection due to their mobility and flexibility. These capabilities are crucial in time-sensitive missions (e.g., disaster response, military reconnaissance). In such cases, tasks are often subject to tight deadlines and require timely access to information. To address these challenges, this paper investigates collaborative data collection and task offloading in multi-AAV systems, aiming to maximize mission area coverage while minimizing total energy consumption. To overcome the limited computing power of data collection AAVs, we propose a heterogeneous multi-tier AAV system. In this design, an assisted AAV with strong computing capabilities is introduced to handle data offloading and processing. This enhances energy efficiency and enables timely task execution. Consequently, we develop an integrated optimization model to jointly design trajectory planning and task offloading under communication, energy, and deadline constraints. We propose a deep reinforcement learning algorithm called data collection optimized proximal policy optimization (DCOPPO). This approach optimizes both AAV trajectories and offloading decisions. Simulation results demonstrate that DCOPPO significantly outperforms baseline DRL approaches in terms of energy efficiency and task completion performance.
Low-altitude wireless networks (LAWNs) have garnered significant attention in the forthcoming 6G networks. In LAWNs, satellites with wide coverage and unmanned aerial vehicles (UAVs) with flexible mobility can complement each other to form integrated satellite-UAV networks, thereby providing ubiquitous and high-speed connectivity for low-altitude operations. However, the higher line-of-sight probability in low-altitude airspace increases transmission security concerns. In this work, we present a collaborative beamforming-based physical layer security scheme for LAWNs. We introduce the fundamental aspects of integrated satellite-UAV networks, physical layer security, UAV swarms, and collaborative beamforming for LAWN applications. Following this, we highlight several opportunities for collaborative UAV swarm secure applications enabled by satellite networks, including achieving physical layer security in scenarios involving data dissemination, data relay, eavesdropper collusion, and imperfect eavesdropper information. Next, we detail two case studies, i.e., a secure relay system and a two-way aerial secure communication framework specifically designed for LAWN environments. Simulation results demonstrate that these physical layer security schemes are effective and beneficial for secure low-altitude wireless communications. A practicality analysis shows that the proposed method is applicable to LAWN scenarios. Finally, we discuss current challenges and future research directions for enhancing security in LAWNs.
Large Vision-Language Models (VLMs) have demonstrated exceptional performance in knowledge-based visual question answering, while small-scale VLMs, owing to their distinct training paradigms, retain complementary strengths that large VLMs lack. Consequently, collaboration between large and small VLMs has emerged as a promising research direction. However, existing collaborative frameworks predominantly rely on unidirectional prompting, which limits the depth of synergistic reasoning across model scales. To address this limitation, we propose SyMRR, a large-small model Synergy framework based on Multimodal Reflective Reasoning, which enables deep, reflective, and visually grounded collaboration between compact and large VLMs. Specifically, we introduce a multimodal reflective chain-of-thought mechanism that guides the large VLM to critically assess the small model's predictions, generating both textual rationales and visually grounded evidence. These reflective signals are then integrated back into the small model's features via a dual reflective interaction strategy, utilizing modality-specific cross-attention to refine the initial multimodal cues. The resulting interaction-enhanced representations guide the large VLM toward more reliable and interpretable answer generation. Notably, SyMRR adopts a parameter-efficient design by freezing both backbone models and optimizing only lightweight interaction modules. Extensive experiments on the OK-VQA and A-OKVQA benchmarks demonstrate that SyMRR achieves state-of-the-art performance among collaborative frameworks, highlighting the effectiveness of deep reflective interaction for complex, knowledge-intensive multimodal reasoning.
With the growing deployment of intelligent applications in the Internet of Vehicles (IoV), collaborative deep neural network (DNN) inference across edge nodes has become essential for satisfying real-time processing and service demands. As a key enabler for 6G networks, rate-splitting multiple access (RSMA) offers improved transmission performance to meet the diverse requirements of multi-user communications. In this work, we propose an RSMA-assisted edge co-inference framework for distributed DNN task offloading in the IoV, where DNN inference tasks are partitioned and offloaded across roadside unit (RSU) and vehicles in a collaborative manner. The objective is to minimize the total task completion time for all vehicles by jointly optimizing the DNN partitioning and offloading strategy, RSMA power allocation, and computing resource allocation. The problem is inherently non-convex due to the strong coupling among communication, computation, and model partitioning variables. To address this, we develop a unified optimization framework and decompose it into three coordinated sub-problems. A triple-iteration joint optimization algorithm is formulated, combining successive convex approximation (SCA) based RSMA power control, genetic algorithm based DNN task scheduling, and Lagrange multiplier method based computing resource allocation at RSU. Simulation results demonstrate that, compared with baseline schemes, the proposed framework can reduce inference latency by up to $20\%$.
Multimodal large language models (MLLMs) have been well acknowledged as the generalist across a broad spectrum of vision-language understanding tasks. Despite notable advancements, their potential for autonomous-driving perception remains largely underexplored. In response, we conduct an in-depth investigation of image-text-point interaction and propose a versatile paradigm of vision-language supervision (VLS) for 3D object detection, where multi-sensory proposals are primarily refined with meticulously-designed text-referred expression, and multimodal correspondences are further incorporated in a contrastive-learning manner. Moreover, VLS holds great advantages. (1) No complicated engineering. It could be seamlessly integrated into a camera-LiDAR 3D detector without troublesome hand-crafted engineering. (2) No extra computation. It provides auxiliary guidance only during training. (3) No additional data. It derives multimodal pairs from ground-truth label instead of a laborious annotation pipeline. Empirical study on publicly available KITTI and nuScenes benchmarks demonstrates the state-of-the-art detection performance against a wide span of counterparts, suggesting its effectiveness and advancement. We hope this work could pave a substantial path towards multimodal feature fusion and object detection for autonomous driving.
In complex and unknown environments, uncrewed ground vehicles (UGVs) with no perception capability or degraded sensing struggle to achieve efficient and safe autonomous navigation. To address this challenge, this letter proposes a low-cost aerial-ground collaborative autonomous navigation system designed for unknown environments, which supports efficient navigation of perception-denied ground platforms while minimizing hardware requirements. In the proposed system, an uncrewed aerial vehicle (UAV) equipped with a lightweight camera provides the UGV with relative position observations and semantic bird & sacute;-eye-view (BEV) maps. The semantic BEV is designed as a unified intermediate representation for UAV-UGV collaboration. While reducing communication overhead, semantic risk information is explicitly incorporated into the diffusion inference process, enabling the planner to satisfy geometric feasibility constraints during path generation while proactively avoiding semantically high-risk regions. Both simulation and real-world experiments validate the effectiveness of the proposed framework. In particular, within an unknown obstacle-filled space of 7 m & times; 4 m & times; 3 m, the UAV enables efficient and safe collaborative navigation for perception-denied UGVs, demonstrating strong application potential in complex and unknown environments.