Purpose Locomotion tracking is a critical capability for humanoid robots to navigate environments and perform loco-manipulation tasks. Achieving this requires fulfilling various kinematic and dynamic sub-objectives, such as accurate tracking of the robot’s base, joints and feet, environment-collision avoidance, and dynamic balance and stability. The purpose of this paper is to propose a controller to generate motions for humanoid robots considering all sub-objectives of locomotion tracking. Design/methodology/approach This paper introduces a hierarchical model predictive control (MPC) framework for the locomotion tracking control problem of humanoid robots. All kinematics sub-objectives are firstly solved at the high-level MPC using full kinematics with second-order kinematics of base. Both kinematics and dynamics sub-objectives are optimized in the low-level kinodynamic MPC considering centroidal dynamics and surface contact dynamics. Findings This paper validates the effectiveness of this method through extensive simulation and hardware experiments. In comparison to traditional whole-body MPC, the proposed method improves the locomotion tracking accuracy while reducing the violations of the system’s physical limit constraints and environment-collision avoidance constraints. Originality/value Both reinforcement learning (RL) and whole-body MPC have become popular approaches for motion control of legged robot. However, achieving all the sub-objectives of locomotion within a single policy remains a challenge for RL methods. Due to computation limitations and strict real-time requirements, it is difficult for the whole-body MPC to generate optimal motions over a short-time horizon while considering multiple tracking goals and nonlinear dynamics of humanoid robots.

With the convergence of smart wearables and digital art, intelligent materials that integrate perception, interaction, and aesthetic expression have emerged as a frontier in the interdisciplinary domain of textiles and interaction design. Intelligent textile-leather systems, owing to their flexibility, tactile quality, and cultural expressiveness, hold significant potential in immersive fashion and artistic installations. However, existing studies predominantly focus on single functionalities, such as pressure sensing or static lighting effects, and lack a closed-loop design encompassing "perception-interaction-aesthetic feedback." Although some efforts have embedded electronic components into materials, they struggle to simultaneously achieve high functional density, mechanical durability, and aesthetic coherence. To address these limitations, this paper proposes a layered heterogeneous integration architecture for intelligent textile-leather systems, based on conformal embedding of microfluidic ionogel sensors and flexible circuits, along with an event-driven, low-latency bidirectional interaction protocol. Experimental results demonstrate that the system maintains 98.2% signal retention under repeated bending with a curvature radius ≤5 mm, achieves an end-to-end interaction latency of 43.6 ms, and reduces energy consumption by 21.3%. This work establishes a co-optimization design paradigm that harmonizes functionality and aesthetics, filling a systemic gap in the application of intelligent materials within artistic expression contexts and offering a scalable technical pathway for dynamic fashion and interactive heritage exhibitions.

Existing depth estimation methods are fundamentally limited to predicting depth on discrete image grids. Such representations restrict their scalability to arbitrary output resolutions and hinder the geometric detail recovery. This paper introduces InfiniDepth, which represents depth as neural implicit fields. Through a simple yet effective local implicit decoder, we can query depth at continuous 2D coordinates, enabling arbitrary-resolution and fine-grained depth estimation. To better assess our method's capabilities, we curate a high-quality 4K synthetic benchmark from five different games, spanning diverse scenes with rich geometric and appearance details. Extensive experiments demonstrate that InfiniDepth achieves state-of-the-art performance on both synthetic and real-world benchmarks across relative and metric depth estimation tasks, particularly excelling in fine-detail regions. It also benefits the task of novel view synthesis under large viewpoint shifts, producing high-quality results with fewer holes and artifacts. Code and data will be made publicly available.

Visual odometry (VO) plays a critical role in unmanned systems, as its performance directly determines localization accuracy and stability. However, existing visual odometry selection methods often rely on expert knowledge or single-performance metrics, limiting adaptability under diverse user preferences, diverse environments, and device constraints. This paper proposes a user-guided framework for visual odometry algorithm selection and composition, named UG-VOS, which models the selection process as a cognitive decision-making system driven by user preferences and environmental context. First, We design a scene-adaptive selection module that embeds lightweight adapters into a Transformer encoder to adaptively model environmental features and jointly optimize front-end and back-end algorithm selection. Second, we propose a multiobjective evaluation module that integrates user preferences with environmental context and visual odometry performance to guide algorithm selection, achieving a balanced trade-off among accuracy, efficiency, and system resource usage. Experiments on diverse datasets demonstrate that UG-VOS consistently achieves superior algorithm compositions compared to existing methods, offering a robust trade-off between precision and resource efficiency.

Accurate and robust localization is essential for the safe and efficient operation of autonomous systems. Visual-inertial navigation systems (VINS), which integrate camera and inertial measurement unit (IMU) data, are widely used for this purpose. The loop closure detection and update module in VINS contributes to the correction of global accumulated errors, whereas the traditional Multi-State Constraint Kalman Filter (MSCKF) method and its variants lack in-depth investigation on loop closure mechanisms. Besides, standard loop closure updates typically operate in a drifted global frame. It violates the small-error linearization assumption of Extended Kalman Filter (EKF) and thereby degrading correction accuracy. This paper is aimed at proposing a loop closure strategy that redefines the global reference frame prior to loop closure up dates, effectively resetting the EKF linearization point. Theoretical analysis shows that this reduces second-order linearization error, thereby enhancing estimation accuracy. Moreover, the filter's state vector is augmented with the gravity vector, and the filter performs high-precision loop closure update. Finally, experimental validation on public datasets and self-collected sequences demonstrates that both contributions improve localization accuracy.

Slender medical continuum robots with flexibility and highly redundant degrees of freedom are widely used in various minimally invasive surgery. However, when interacting with anatomical structures, the continuum robot adopts diverse shapes, posing challenges for operation and control. To achieve real-time intraoperative shape sensing and provide online guidance for manipulation, most existing methods rely on optical fibers embedded within the robot, which often require specialized robot designs and come with high costs. Here, we present a novel approach utilizing thin and flexible carbon nanotube piezoresistive fibers as a bandage, helically integrated on the surface of existing slender medical continuum robots for shape sensing. The spatial configuration of the robot is effectively inferred by downsampling the resistance changes along the robot’s body and applying a learning-based method. The results demonstrate that the proposed helically arranged carbon nanotube piezoresistive fibers, combined with a data-driven approach, are capable of reconstructing the robot’s spatial shape. In vitro and ex vivo experiments on animal tissues further highlight its promising potential for enhancing the shape-sensing ability of existing medical continuum robots.

Open-vocabulary panoptic reconstruction is crucial for advanced robotics and simulation. However, existing 3D reconstruction methods, such as NeRF or Gaussian Splatting variants, often struggle to achieve the real-time inference frequency required by robotic control loops. Existing methods incur prohibitive latency when processing the high-dimensional features required for robust open-vocabulary segmentation. We propose Fast-SegSim, a novel, simple, and end-to-end framework built upon 2D Gaussian Splatting, designed to realize real-time, high-fidelity, and 3D-consistent open-vocabulary segmentation reconstruction. Our core contribution is a highly optimized rendering pipeline that specifically addresses the computational bottleneck of high-channel segmentation feature accumulation. We introduce two key optimizations: Precise Tile Intersection to reduce rasterization redundancy, and a novel Top-K Hard Selection strategy. This strategy leverages the geometric sparsity inherent in the 2D Gaussian representation to greatly simplify feature accumulation and alleviate bandwidth limitations, achieving render rates exceeding 40 FPS. Fast-SegSim provides critical value in robotic applications: it serves both as a high-frequency sensor input for simulation platforms like Gazebo, and its 3D-consistent outputs provide essential multi-view 'ground truth' labels for fine-tuning downstream perception tasks. We demonstrate this utility by using the generated labels to fine-tune the perception module in object goal navigation, successfully doubling the navigation success rate. Our superior rendering speed and practical utility underscore Fast-SegSim's potential to bridge the sim-to-real gap.

Depth estimation is a fundamental task in 3D computer vision, crucial for applications such as 3D reconstruction, free-viewpoint rendering, robotics, autonomous driving, and AR/VR technologies. Traditional methods relying on hardware sensors like LiDAR are often limited by their high costs, low resolution, and sensitivity to the environment, limiting their applicability to real-world scenarios. Recent advances in vision-based methods offer a promising alternative, yet they face challenges in generalization and stability due to either the low capacity of model architectures or reliance on domain-specific and small-scale datasets. The emergence of scaling laws and foundation models in other domains has inspired the development of “depth foundation models”: deep neural networks trained on large datasets with strong zeroshot generalization capabilities. This paper surveys the evolution of deep learning architectures and paradigms for depth estimation across monocular, stereo, multiview, and monocular video settings. We explore the potential of these models to address existing challenges and we also provide a comprehensive overview of large-scale datasets that can facilitate their development. By identifying key architectures and training strategies, we aim to highlight the path towards robust depth foundation models, offering insights for future research and applications.

Controlling symmetric objects is an indispensable but challenging task in robotic manipulation. Mainstream perception-action frameworks rely on accurate 6D pose estimation to guide the controller. However, the majority of existing 6D pose estimation methods for symmetric objects are designed to output a single pose, which can flicker between multiple equivalent solutions across consecutive frames, leading to instability in the control loop. While some approaches can output multiple hypotheses to represent the ambiguity, above methods generally cannot achieve model-free manner and strong generalization simultaneously. In this paper, we formulate the problem from a multi-solution task in pose space to an end-to-end visual servo task that admits a unique optimal solution. We propose a visual servo framework Sym-Servo. Sym-Servo uses a joint learning mechanism where a deterministic policy is trained with a diffusion-based generator to encourage the shared vision encoder to learn a symmetry-aware representation, and the policy is then refined via reinforcement and self-imitation learning to produce an efficient and stable final policy. We validate Sym-Servo with simulations and real-world experiments, demonstrating its efficiency and generalization in controlling symmetric objects in a model-free manner.

Reinforcement Learning from Human Feedback (RLHF) and its variants have emerged as the dominant approaches for aligning Large Language Models with human intent. While empirically effective, the theoretical generalization properties of these methods in high-dimensional settings remain to be explored. To this end, we build the generalization theory on RLHF of LLMs under the linear reward model, through the framework of algorithmic stability. In contrast to the existing works built upon the consistency of maximum likelihood estimations on reward model, our analysis is presented under an end-to-end learning framework, which is consistent with practice. Concretely, we prove that under a key feature coverage condition, the empirical optima of policy model have a generalization bound of order 𝒪(n^-1/2). Moreover, the results can be extrapolated to parameters obtained by gradient-based learning algorithms, i.e., Gradient Ascent (GA) and Stochastic Gradient Ascent (SGA). Thus, we argue that our results provide new theoretical evidence for the empirically observed generalization of LLMs after RLHF.

Urban scene reconstruction under noisy camera poses remains a critical challenge for autonomous driving. While recent neural dense Bundle Adjustment (BA) methods have shown promising results in specific settings, their performance often degrades in real-world urban scenarios due to noisy correspondences and imbalanced optimization between camera poses and scene parameters, which leads the scene representation to overfit to erroneous geometric constraints, causing the system to converge to suboptimal local minima. We propose pi-BA, a neural BA framework designed to improve robustness in large-scale outdoor reconstruction. pi-BA introduces an uncertainty-aware probabilistic model that adaptively down-weights unreliable correspondences, enabling more stable joint optimization of camera poses and scene geometry. To decouple pose estimation from geometry artifacts, we employ a cyclic optimization strategy that periodically reinitializes the radiance field while preserving refined camera poses. Extensive experiments on diverse real-world datasets demonstrate that pi-BA consistently outperforms existing neural BA methods in terms of pose and reconstruction accuracy under noisy initialization. These results highlight the effectiveness of integrating probabilistic modeling with iterative refinement for scalable and robust neural reconstruction in complex outdoor environments.

Vision-Language Navigation in Continuous Environments (VLN-CE) requires an agent to follow natural language instructions through unseen environments. Existing imitation learning (IL) pipelines struggle in this closed-loop setting: behavior cloning suffers from distribution shift, and DAgger's expert actions become ambiguous upon trajectory deviation. While Reinforcement Learning (RL) offers a natural paradigm to address this, directly applying RL to micro action spaces is sample-inefficient due to reward sparsity. To overcome this bottleneck, we reformulate VLN-CE as a Hierarchical Markov Decision Process (MDP), explicitly decoupling high-level planning from low-level control. By abstracting the environment into a topological graph, our high-level policy operates on a macro action space of frontier nodes, with a training-free low-level controller acting as its state transition, which significantly compresses the decision horizon and makes closed-loop RL tractable. To support RL optimization on the macro MDP, we propose an action-aware value head to effectively evaluate state values under the dynamic frontier action space, powering a graph-based PPO. Extensive experiments demonstrate the effectiveness of our architecture. Finally, our model achieves state-of-the-art performance on the R2R-CE and RxR-CE benchmarks.

With the rapid growth of optical computing and integrated photonics, low-power optoelectronic logic devices are essential for on-chip information processing. Conventional implementations rely on multiple discrete components, limiting system scalability. We demonstrate a wavelength-dependent bipolar optoelectronic logic device based on a ReS2/Gr/h-BN/Gr van der Waals heterostructure. The fundamental mechanism is VOC-dictated: under 365 nm illumination, a substantial VOC magnitude defines a high-capacity absorption response and a photovoltaic (PV) process, generating a negative photocurrent. In contrast, at 635 nm, a diminished VOC restricts the absorption response strength, yielding a positive photocurrent and enabling reversible PV polarity switching. This bipolarity is key to achieving "single-device multifunctional logic." By adjusting wavelength and power, a single physical unit reconfigures six fundamental logic gates (OR, AND, NOR, NAND, NOT, and XNOR), reducing the transistor count required for complex circuits by 87.5% compared to CMOS implementations. The device exhibits high responsivity (730 mA/W at 365 nm; 490 mA/W at 635 nm), a detectivity of 1011 Jones, ultralow dark current of 15 pA, and sub-microsecond working speed. This VOC-driven reconfigurable logic offers a compact, energy-efficient pathway for integrated photonics and secure optical communication.

Tendon-Driven Continuum Robots (TDCRs) pose significant control challenges due to their highly nonlinear, path-dependent dynamics and non-Markovian characteristics. Traditional Jacobian-based controllers often struggle with hysteresis-induced oscillations, while conventional learning-based approaches suffer from poor generalization to out-of-distribution trajectories. This paper proposes a reference-augmented offline learning framework for precise 6-DOF tracking control of TDCRs. By leveraging a differentiable RNN-based dynamics surrogate as a gradient bridge, we optimize a control policy through an augmented reference distribution. This multi-scale augmentation scheme incorporates stochastic bias, harmonic perturbations, and random walks, forcing the policy to internalize diverse tracking error recovery mechanisms without additional hardware interaction. Experimental results on a three-section TDCR platform demonstrate that the proposed policy achieves a 50.9% reduction in average position error compared to non-augmented baselines and significantly outperforms Jacobian-based methods in both precision and stability across various speeds.

We study the task of language-conditioned pick and place in clutter, where a robot should grasp a target object in open clutter and move it to a specified place. Some approaches learn end-to-end policies with features from vision foundation models, requiring large datasets. Others combine foundation models in a zero-shot setting, suffering from cascading errors. In addition, they primarily leverage vision and language foundation models, focusing less on action priors. In this paper, we aim to develop an effective policy by integrating foundation priors from vision, language, and action. We propose A^2, an action prior alignment method that aligns unconditioned action priors with 3D vision-language priors by learning one attention layer. The alignment formulation enables our policy to train with less data and preserve zero-shot generalization capabilities. We show that a shared policy for both pick and place actions enhances the performance for each task, and introduce a policy adaptation scheme to accommodate the multi-modal nature of actions. Extensive experiments in simulation and the real-world show that our policy achieves higher task success rates with fewer steps for both pick and place tasks in clutter, effectively generalizing to unseen objects and language instructions. Videos and codes are available at https://xukechun.github.io/papers/A2.

Sim-to-real transfer for contact-rich manipulation remains challenging due to the inherent discrepancy in contact dynamics. While existing methods often rely on costly real-world data or utilize blind compliance through fixed controllers, we propose a framework that leverages expert-designed controller logic for transfer. Inspired by the success of privileged supervision in kinematic tasks, we employ a human-designed finite state machine based position/force controller in simulation to provide privileged guidance. The resulting policy is trained to predict the end-effector pose, contact state, and crucially the desired contact force direction. Unlike force magnitudes, which are highly sensitive to simulation inaccuracies, force directions encode high-level task geometry and remain robust across the sim-to-real gap. At deployment, these predictions configure a force-aware admittance controller. By combining the policy's directional intent with a constant, low-cost manually tuned force magnitude, the system generates adaptive, task-aligned compliance. This tuning is lightweight, typically requiring only a single scalar per contact state. We provide theoretical analysis for stability and robustness to disturbances. Experiments on four real-world tasks, i.e., microwave opening, peg-in-hole, whiteboard wiping, and door opening, demonstrate that our approach significantly outperforms strong baselines in both success rate and robustness. Videos are available at: https://yifei-y.github.io/project-pages/DirectionMatters/.

Object-goal visual navigation requires robots to reason over semantic structure and act effectively under partial observability. Recent approaches based on object-level topological maps enable long-horizon navigation without dense geometric reconstruction, but their execution remains limited by the gap between global topological guidance and local perception-driven control. In particular, local decisions are made solely from the current egocentric observation, without access to information beyond the robot's field of view. As a result, the robot may persist along its current heading even when initially oriented away from the goal, moving toward directions that do not decrease the global topological distance. In this work, we propose IntentReact, an intent-conditioned object-centric navigation framework that introduces a compact interface between global topological planning and reactive object-centric control. Our approach encodes global topological guidance as a low-dimensional directional signal, termed intent, which conditions a learned waypoint prediction policy to bias navigation toward topologically consistent progression. This design enables the robot to promptly reorient when local observations are misleading, guiding motion toward directions that decrease global topological distance while preserving the reactivity and robustness of object-centric control. We evaluate the proposed framework through extensive experiments, demonstrating improved navigation success and execution quality compared to prior object-centric navigation methods.

Simulation plays a crucial role in assessing autonomous driving systems, where the generation of realistic multi-agent behaviors is a key aspect. In multi-agent simulation, the primary challenges include behavioral multimodality and closed-loop distributional shifts. In this study, we formulate a unified mixture model (UniMM) framework revisit mixture models for generating multimodal agent behaviors, which can cover the mainstream methods including regression-based continuous mixture models and discrete NTP GPT-like discrete models. Furthermore, we introduce a closed-loop sample generation approach tailored for mixture models to mitigate distributional shifts. Within the UniMM framework, we recognize critical configurations from both the model and data perspectives. We conduct a systematic examination of various model configurations, and comprehensively characterize their effects including positive component matching, continuous regression, prediction horizon, and the number of components. Moreover, our investigation into the data configuration highlights the pivotal role of closed-loop samples in achieving realistic simulations. To extend the benefits of closed-loop samples across a broader range of mixture models, we further introduce a temporal disentanglement-and-alignment mechanism to address the shortcut learning and off-policy learning issues. Leveraging insights from our exploration, the distinct variants proposed within the UniMM framework, including discrete, anchor-free, and anchor-based models, all achieve state-of-the-art performance on the WOSAC benchmark.

In dynamic urban scenarios involving autonomous vehicles and vehicle-infrastructure interactions, LiDAR-inertial odometry (LIO) methods are widely adopted to provide real-time vehicle pose estimation with robust and accurate positioning, particularly in urban environments where GPS signals are frequently disrupted. However, existing methods typically struggle with balancing real-time computational efficiency and localization accuracy, limiting their practical applicability. To address this challenge, we propose IO-LIO, an information-oriented voxel-based LIO system specifically designed for diverse real-world transportation scenarios. IO-LIO enhances real-time performance through concurrent processing and targeted computational optimizations, while efficiently extracting and utilizing high-value voxel information. Specifically, a lookup table-based method is introduced for rapid point cloud undistortion, coupled with an incremental distribution computation approach for information-preserving voxel downsampling. The mapping module adopts an adaptive voxel merging strategy with a multilevel Least Recently Used (M-LRU) mechanism, effectively reducing memory usage. Moreover, we propose a weighted GICP residual construction method, where residuals are weighted by voxel information values, quantitatively improving trajectory accuracy compared to state-of-the-art methods. Additionally, IO-LIO systematically addresses the critical but often overlooked challenge of memory management in LIO systems through a multi-threaded, template-based object pool. Extensive experiments were conducted in realistic ITS-relevant scenarios, including autonomous vehicles operating in dense urban traffic, vehicles navigating campus and park environments, and pedestrian-based backpack mapping. These experiments were complemented by a dedicated ablation study that quantified the incremental benefit of each module. Collectively, the results demonstrate that IO-LIO significantly surpasses state-of-the-art methods in localization accuracy, real-time performance, and reliability, highlighting its strong potential for practical deployment in intelligent transportation applications.

Tendon-Driven Continuum Robots (TDCRs) pose significant modeling and control challenges due to complex nonlinearities, such as frictional hysteresis and transmission compliance. This paper proposes a differentiable learning framework that integrates high-fidelity dynamics modeling with robust neural control. We develop a GRU-based dynamics model featuring bidirectional multi-channel connectivity and residual prediction to effectively suppress compounding errors during long-horizon auto-regressive prediction. By treating this model as a gradient bridge, an end-to-end neural control policy is optimized through backpropagation, allowing it to implicitly internalize compensation for intricate nonlinearities. Experimental validation on a physical three-section TDCR demonstrates that our framework achieves accurate tracking and superior robustness against unseen payloads, outperforming Jacobian-based methods by eliminating self-excited oscillations.