Sufficient testing under corner cases is critical for the long-term operation of vehicle-infrastructure cooperation systems (VICS). However, existing corner case generation methods are primarily AI-driven, and VICS testing under corner cases is typically limited to simulation. In this paper, we introduce an L5 “Interactable” level to the VICS digital twin (VICS-DT) taxonomy, extending beyond the conventional L4 “Optimizable” level. We further propose an L5-level VICS testing framework, IMPACT (Interactive Mixed-digital-twin Paradigm for Advanced Cooperative vehicle-infrastructure Testing). By enabling direct interactions between human operators and VICS entities, IMPACT introduces realistic human-related uncertainties into the testing loop and supports the human-in-the-loop generation of corner cases as a complement to AI-driven methods. Furthermore, the mixedDT-enabled “Physical-Virtual Action Interaction” facilitates safe VICS testing under corner cases, incorporating real-world environments and entities rather than purely in simulation. Finally, we implement IMPACT on the I-VIT (Interactive Vehicle-Infrastructure Testbed), and experiments demonstrate its effectiveness. The experimental videos are available at our project website: https://dongjh20.github.io/IMPACT.
In emerging mixed traffic environments, connected and autonomous vehicles (CAVs) must interact with surrounding human-driven vehicles (HDVs). This study introduces multisource human-in-the-loop mixed cloud control testbed (MSH-MCCT), a novel CAV testbed that captures complex interactions between various CAVs and HDVs. Utilizing the mixed digital twin concept, which combines mixed reality with digital twins, MSH-MCCT integrates physical, virtual, and mixed platforms, along with multisource control inputs. Bridged by the mixed platform, MSH-MCCT allows human drivers and CAV algorithms to operate both physical and virtual vehicles within multiple fields of view. In particular, this testbed facilitates the coexistence and real-time interaction of physical and virtual CAVs and HDVs, significantly enhancing the experimental flexibility and scalability. Experiments on vehicle platooning in mixed traffic showcase the potential of MSH-MCCT to conduct CAV testing with multisource real human drivers in the loop through driving simulators of diverse fidelity.
Switching communication topologies may cause instability in vehicle platoons, as vehicle information can be lost during the dynamic switching process. This highlights the need to design a controller capable of maintaining the stability of vehicle platoons under dynamically changing topologies. However, it remains a significant challenge to capture the dynamic characteristics of switching topologies and obtain sufficient vehicle information for controller design while ensuring stability. In this study, an observer-based distributed model predictive control (DMPC) framework is developed for vehicle platoons under directed Markovian switching communication topologies. The directed switching communication topology is modeled using a continuous-time Markov chain to characterize its stochastic switching behavior. To estimate the leader vehicle information required for control, a fully distributed adaptive observer is designed, whose estimation performance is robust to randomly switching topologies. A sufficient condition is further derived to guarantee the mean-square stability of the observer error dynamics. Based on the estimated information, a terminal update law is constructed to ensure mean-square consensus, and a string stability constraint is formulated to explicitly enforce predecessor-follower string stability within the DMPC framework. Recursive feasibility and closed-loop stability properties of the resulting control scheme are established. Numerical simulation results demonstrate that the proposed method enhances tracking performance, accelerates convergence, and reduces control effort, while maintaining predecessor-follower string stability. With acceleration fluctuations and packet loss, the proposed method reduces the maximum position error by 45%, and improves stability by 39%, demonstrating significant improvements in tracking performance and system stability. Furthermore, the effectiveness of the proposed framework is validated using real-world driving data.
Cooperative 3D perception via Vehicle-to-Everything (V2X) communication is a promising paradigm for enhancing autonomous driving, offering extended sensing horizons and occlusion resolution.However, the practical deployment of existing methods is hindered at long distances by two critical bottlenecks:the quadratic computational scaling of dense BEV representations and the fragility of feature association mechanisms under significant observation and alignment errors.To overcome these limitations, we introduce Long-SCOPE, a fully sparse framework designed for robust long-distance cooperative 3D perception.Our method introduces two novel components: a Geometry-guided Query Generation module to accurately detect small, distant objects,and a learnable Context-Aware Association module that robustly matches cooperative queries even despite severe positional noise.Experiments on the V2X-Seq and Griffin datasets validate that Long-SCOPE achieves state-of-the-art performance, particularly in challenging long-range settings, while maintaining superior computational efficiency and a highly competitive transmission cost.
Connected autonomous vehicles (CAVs), which represent a significant advancement in autonomous driving technology, have the potential to greatly increase traffic safety and efficiency through cooperative decision-making. However, existing methods often overlook the individual needs and heterogeneity of cooperative participants, making it difficult to transfer them to environments where they coexist with human-driven vehicles (HDVs). To address this challenge, this paper proposes an adaptive potential game (APG) cooperative driving framework. First, the system utility function is established on the basis of a general form of individual utility and its monotonic relationship, allowing for the simultaneous optimization of both individual and system objectives. Second, the Shapley value is introduced to compute each vehicle’s marginal utility within the system, allowing its varying impact to be quantified. Finally, the HDV preference estimation is dynamically refined by continuously comparing the observed HDV behavior with the APG’s estimated actions, leading to improvements in overall system safety and efficiency. Ablation studies demonstrate that adaptively updating Shapley values and HDV preference estimation significantly improve cooperation success rates in mixed traffic. Comparative experiments further highlight the APG’s advantages in terms of safety and efficiency over other cooperative methods. Moreover, the approach’s applicability of the approach to real-world scenarios was validated through field tests.
Large language models (LLMs) have shown great promise for autonomous driving. However, discretizing numbers into tokens limits precise numerical reasoning, fails to reflect the positional significance of digits in the training objective, and makes it difficult to achieve both decoding efficiency and numerical precision. These limitations affect both the processing of sensor measurements and the generation of precise control commands, creating a fundamental barrier for deploying LLM-based autonomous driving systems. In this paper, we introduce DriveCode, a novel numerical encoding method that represents numbers as dedicated embeddings rather than discrete text tokens. DriveCode employs a number projector to map numbers into the language model's hidden space, enabling seamless integration with visual and textual features in a unified multimodal sequence. Evaluated on OmniDrive, DriveGPT4, and DriveGPT4-V2 datasets, DriveCode demonstrates superior performance in trajectory prediction and control signal generation, confirming its effectiveness for LLM-based autonomous driving systems.
Most autonomous driving safety benchmarks use time-to-collision (TTC) to assess risk and guide safe behaviour. However, TTC-based methods treat risk as a one-dimensional closing problem, despite the inherently two-dimensional nature of collision avoidance, and therefore cannot faithfully capture risk or its evolution over time. Here, we report evasive acceleration (EA), a hyperparameter-free and physically interpretable two-dimensional paradigm for risk quantification. By evaluating all possible directions of collision avoidance, EA defines risk as the minimum magnitude of a constant relative acceleration vector required to alter the relative motion and make the interaction collision-free. Using interaction data from five open datasets and more than 600 real crashes, we derive percentile-based warning thresholds and show that EA provides the earliest statistically significant warning across all thresholds. Moreover, EA provides the best discrimination of eventual collision outcomes and improves information retention by 54.2-241.4
Visible-infrared (VIS-IR) image pairs offer complementary information and hold significant potential for UAV-based object detection. However, most existing multi-modal detection approaches overlook the prevalent issue of weak spatial misalignment in real-world scenarios. Although several methods attempt to address this challenge, they often rely on complex architectures for explicit alignment. In this work, we propose DMDet, a novel detection framework based on deformable attention to achieve adaptive and implicit alignment between modalities. To guide effective feature fusion, we introduce a Competitive Modal Feature Selection (CMFS) module that dynamically identifies the most informative modality for each object instance. Furthermore, we design a Deformable Cross-Feature Attention (DeformCFA) module that adaptively samples and aggregates complementary VIS-IR features, effectively mitigating misalignment and enhancing fusion quality. Extensive experiments on the DroneVehicle dataset demonstrate that DMDet significantly outperforms state-of-the-art multi-modal object detectors, especially under weak misalignment conditions.
Managing mixed vehicle platoons, which integrate intelligent and connected vehicles and human-driven vehicles, presents significant challenges due to the uncertainties inherent in human driving behaviors. Although data-driven control techniques utilizing trajectory data have shown a potential to address these challenges, their performance is often undermined by inevitable noise and external disturbances. To address this limitation, we propose a tube-based robust data-driven predictive control (TRDDPC) framework to enhance the robustness of mixed vehicle platoons. The framework begins by constructing a matrix zonotope set from data, which provides an over-approximation of system dynamics under the influence of noise and disturbances. By decoupling the over-approximated system into nominal and error subsystems, a data-driven minimal robust positively invariant set is used to encapsulate the impact of disturbances and noise. The TRDDPC framework then formulates an optimization problem to compute robust control inputs, which are implemented through a tube-based control mechanism. Simulation results show that for a mixed vehicle platoon with three vehicles, TRDDPC achieves reductions of 30.6 % in velocity error and 26.7 % in spacing error compared to data-enabled predictive control in comprehensive scenarios, with further reductions of 30.6 % and 32.3 % in emergency scenarios. Moreover, TRDDPC decreases computation time by 90.6 % , significantly enhancing the robustness and efficiency of mixed vehicle platoon control.
Ensuring safety in long-tail scenarios is a critical challenge for autonomous driving, where rare yet high-risk events demand not only precise prediction but also formal safety guarantees. This paper proposes a Large-Model-Driven Risk Reasoning Framework (LMRRF) that integrates probabilistic reasoning, spatiotemporal modeling, and explainable analysis into a unified three-layer design. First, anomalous trajectories are detected through deep generative modeling with statistical reliability. Second, spatiotemporal interactions are captured by a LiTraFormer-GNN module, from which we analytically derive a provable upper bound on collision probability. Third, large language models generate natural-language risk reports whose semantic alignment with quantitative indicators is validated via correlation tests. Evaluations on the HighD and AD4CHE datasets, as well as real-vehicle experiments, show that LMRRF achieves centimeter-level trajectory accuracy under real-time constraints and reduces collision-probability bounds by up to 87% compared with direct LLM baselines. By combining provable risk bounds, spatiotemporal interaction reasoning, and statistically validated interpretability, LMRRF provides a theoretically grounded and practically deployable framework for risk-aware autonomous driving in long-tail scenarios.
Engineers and researchers suffer from current LiDAR perception: each new sensor typically requires a dedicated dataset, model, and training pipeline, and each team repeats the costly process, as data are usually not shared, severely hindering deployment and increasing R&D costs. This limitation largely stems from the prevailing single-dataset training paradigm, under which models easily overfit to sensor-specific sampling patterns and generalize poorly to unseen LiDARs. This work tackles this problem from a new perspective: any LiDAR can be viewed as a sparse sampler of the underlying 3D world , so each scan can be decomposed into scene-intrinsic geometry (what the scene is) and sensor-specific sampling effects (how the sensor samples it). Based on this insight, we propose LiSeer, a unified pipeline ''learning from randomness'' for zero-shot cross-LiDAR generalization. LiSeer first reconstructs the underlying scene using a realistic diffusion-based densification model, and then applies a physics-based controllable sampling strategy to mimic diverse LiDAR point clouds. This enables virtually unlimited generation of cross-sensor data and alleviates the data-hungry bottleneck. Building on this foundation, we further encourage the model to prioritize scene-intrinsic geometry over sensor-specific sampling effects through explicit scene-sensor disentanglement and consistency learning. Extensive experiments across three datasets and two mainstream segmentation backbones show that LiSeer delivers large, consistent zero-shot gains, improving baseline cross-sensor mIoU by 25.1 to 43.6 percentage points. In our experiments, scaling the generated data further improves accuracy. With only 10%-20% of the target-domain data, it approaches fully-supervised models. Deployed on real vehicles with previously unseen 32-, 80-, and 128-beam LiDARs, LiSeer produces accurate and stable zero-shot segmentation, providing strong evidence of its practical value.
With the development of deep learning technologies, the accuracy of trajectory prediction has significantly improved. Leveraging predicted trajectories for motion planning can reduce the risk of collisions between vehicles more effectively. However, prior studies have shown that directly using predicted trajectories in planning often leads to suboptimal performance and may even cause collisions. To address this, we propose a novel framework that integrates trajectory prediction with artificial potential fields (APFs), enabling the effective use of prediction results in planning. In this paper, we introduce a trajectory planning framework that constructs a spatiotemporal risk (STR) field based on deep learning-based trajectory prediction and inter-vehicle interactions. The planning module then employs an optimization-based approach that considers multiple objectives, including safety, comfort, rule compliance, efficiency, and human likeness. Evaluated on a real-world dataset, our method achieves an 8.72% improvement in minimum Anticipated Collision Time ($\text{ACT}\_{\min }$) and a 36.0% gain in efficiency compared to the baseline Driving Safety Field (DSF) method. Compared with other competing approaches, our method improves $\text{ACT}\_{\min }$ by 10.67% while maintaining competitive performance in terms of comfort, efficiency, and human likeness. The results from the NuPlan closed-loop experiments indicate that our method exhibits superior planning capabilities in complex scenarios, surpassing the performance of all baseline methods. Furthermore, closed-loop experiments demonstrate that our approach enables continuous and collision-free planning in highly interactive traffic scenarios.
Multi-modal trajectory generation is essential for safe autonomous driving, yet existing diffusion-based planners suffer from high inference latency due to iterative neural function evaluations. This paper presents MISTY (Mixer-based Inference for Single-step Trajectory-drifting Yield), a high-throughput generative motion planner that achieves state-of-the-art closed-loop performance with pure single-step inference. MISTY integrates a vectorized Sub-Graph encoder to capture environment context, a Variational Autoencoder to structure expert trajectories into a compact 32-dimensional latent manifold, and an ultra-lightweight MLP-Mixer decoder to eliminate quadratic attention complexity. Importantly, we introduce a latent-space drifting loss that shifts the complex distribution evolution entirely to the training phase. By formulating explicit attractive and repulsive forces, this mechanism empowers the model to synthesize novel, proactive maneuvers, such as active overtaking, that are virtually absent from the raw expert demonstrations. Extensive evaluations on the nuPlan benchmark demonstrate that MISTY achieves state-of-the-art results on the challenging Test14-hard split, with comprehensive scores of 80.32 and 82.21 in non-reactive and reactive settings, respectively. Operating at over 99 FPS with an end-to-end latency of 10.1 ms, MISTY offers an order-of-magnitude speedup over iterative diffusion planners while while achieving significantly robust generation.
This article addresses a generalization problem of multiagent pathfinding (MAPF), called collaborative task sequencing-MAPF (CTS-MAPF), where agents must plan collision-free paths and visit a series of intermediate task locations in an optimized order before reaching their final destinations. To address this problem, we propose a new approach, collaborative task sequencing-conflict-based search (CTS-CBS), which conducts a two-level search. At the high level, it generates a search forest, where each tree corresponds to a joint task sequence derived from the joint-traveling salesman problem (jTSP) solution. At the low level, CTS-CBS performs constrained single-agent path planning to generate paths for each agent while adhering to high-level constraints. Furthermore, we integrate explicit estimation search (EES) into CTS-CBS to enhance the search efficiency. We also provide theoretical guarantees of its solution completeness and optimality (or suboptimality with a bounded parameter). To evaluate the performance of CTS-CBS, we utilize three datasets with different map sizes, map difficulty, and agent/task numbers, and conduct comprehensive experiments. The results show that our algorithms demonstrate significant improvements in success rate and runtime, while maintaining competitive solution quality. Finally, practical robot simulation and tests demonstrate the algorithm's applicability in real-world scenarios.
End-to-end (E2E) autonomous driving systems often suffer from limited interpretability, weak cognitive reasoning, and unreliable trajectory generation in complex interactive environments. This work presents Gaze-Flow, an interpretable and human-like end-to-end planning framework that explicitly connects scene understanding, goal reasoning, and motion generation. First, a Frequency-Progressive Multimodal Fusion (FPMF) backbone is developed to exploit the modeling preferences of different fusion modules, preserving spatial details, enhancing semantic relations, and capturing global dependencies to produce sharp and human-understandable BEV representations. Second, a 3D gaze-guided goal selection module is introduced, which converts the human driving principle of “near-field risk vigilance and far-field trend guidance” into a computable prior for interpretable and scene-adaptive decision guidance. Third, a rectified-flow trajectory generator with curvature-based feasibility checking is integrated to generate smooth, diverse, and physically reasonable trajectories. Experiments on the NAVSIM benchmark demonstrate that Gaze-Flow achieves a PDMS of 91.5 on NAVSIM v1 and an EPDMS of 86.7 on NAVSIM v2, showing competitive performance in safety, comfort, rule compliance, and human-likeness. Case studies further show mechanism-level interpretability, enabling transparent planning diagnostics and more trustworthy planning.
Autonomous excavation requires an intelligent control system that can convert spatial work intent into coordinated bucket motion under contact-rich soil interaction. This paper presents a target-conditioned intelligent control framework for autonomous excavation in a physics-based deformable-soil simulation workflow. An image-aligned target mask serves as a visual spatial command for the desired digging region, while a mask-conditioned Action Chunking Transformer maps multi-view RGB observations, proprioception, and the target mask to temporally extended joystick commands. To reduce target-ignoring behavior, demonstrations are organized with paired-condition supervision, where the same or closely matched scene is demonstrated with different target masks and corresponding action chunks. The framework is evaluated through both a diagnostic manipulation task and an excavation simulation benchmark with single-scoop and sequential pile-clearing protocols. In manipulation, target success is 4% for no-condition ACT, 63% for non-paired mask-conditioned ACT, and 96% for paired-condition mask-conditioned ACT. In sequential pile clearing, paired-condition mask-conditioned ACT removes 76.8% of the pile versus 27.4% and 15.7% for the two baselines, with 91.0% human-normalized efficiency. The results show that visual target conditioning, paired demonstration structure, and action-chunk control form a practical cyber-physical simulation pipeline for excavator automation.
Truck platooning can reduce energy consumption and improve traffic efficiency. However, under emergency braking or other unexpected situations, the shortened inter-vehicle distances increase the risk of multi-vehicle chain collisions, which may result in severe damage. This study focuses on truck platoons with heterogeneous masses and aims to minimise overall collision severity by optimising platoon formation through a combination of chain collision dynamics modelling and simulation experiments. At the theoretical level, we develop a chain collision dynamics model for the reaction-sustained braking stage and formulate the platoon formation problem as a mixed-integer nonlinear programming model. At the simulation level, we conduct experiments under reaction-time delay and overspeed scenarios, and statistical analyses of the optimal platoon configurations reveal a bell-shaped configuration, with heavier trucks in the middle and lighter ones at both ends, which mitigates chain collision severity. Finally, we conducted comparative experiments across different scenarios and platoon sizes, comparing the bell-shaped configuration with the ascending mass configuration, descending mass configuration, and random mass configuration. The results show that the bell-shaped formation reduces energy dissipation by approximately 18%-20% in the reaction-delay scenario and by 7%-20% in the overspeed scenario. Overall, the bell-shaped configuration demonstrates broad applicability and strong robustness in mitigating chain collision severity.