In this work, we investigate how to leverage learning processes to design a perception-driven control framework for robot motion. In this regard, inspired by the fact that potential-based control represents an effective approach for modeling robotic tasks, we study how neural networks can be effectively exploited to approximate unknown perception-based potential functions, for which an analytical closed form may not even be available, thus extending the field of applicability of potential-based control. Numerical results along with an experimental validation are provided to empirically demonstrate the validity of the proposed control architecture.
Autonomous navigation for unmanned ground vehicles (UGVs) in complex environments requires robust perception for reliable traversability estimation and path planning. Traditional geometric methods often fail in unstructured terrains, necessitating robust scene understanding. Current 3D semantic segmentation methods are mostly directed towards structured environments and are not generalisable for off-road domains. To address this, we propose modifications to LiDAR semantic segmentation by incorporating spatial context and adding an auxiliary task of learning point-wise ground-relative height to capture robust features. Experiments on both real-to-real and synthetic-to-real transfer demonstrate that the proposed approach outperforms state-of-the-art methods in segmentation accuracy and adaptability, offering a scalable solution for UGV perception in diverse outdoor environments.
This paper introduces a technical solution for efficient rebar tying by developing a multi-arm rebar tying robot system. We introduced a multi-arm collaboration to achieve parallel task execution, which is 2.6 times more efficient than a single-arm system. A real-time dynamic adjustment algorithm based on market auctions ensures a coordinated and efficient direct operation of robotic arms. An optimized hybrid path planning method is used to improve the positioning accuracy of the rebar intersection, reducing the positioning error to 0.59 mm. Through modeling and simulation analysis, we show that the system can increase efficiency to 62% under different working conditions and minimize labor requirements.
Autonomous terrain leveling and material handling operations play a pivotal role in earthworks, directly affecting project timelines, cost efficiency, and environmental impact. We present the Optimal Orchestration Transport Planner Framework (OOTPF), a unified orchestration architecture for multi robot systems that casts surface leveling as an online Multi Agent Pickup and Delivery (MAPD) problem. OOTPF comprises two principal layers: an Optimal Transport layer that formulates material movement via discrete Earth Mover's problems, and a Task Allocation layer that employs heuristic bidding and consensus based auctions to assign pickup-delivery jobs to agents. To guarantee safe, collision free execution, we embed a token driven, priority based planning discipline atop a shared space time Reservation Table, invoking Safe Interval Path Planning (SIPP) for single robot requests and Conflict Based Search (CBS) when multiple robots contend. This hierarchical scheme dynamically balances throughput with conflict resolution. Numerical experiments on representative leveling scenarios demonstrate that OOTPF achieves high utilization, zero collisions, and flexible to import different path planner algorithms.
Vision-based coordination of Unmanned Aerial Vehicles (UAVs) presents challenges in target tracking, formation control, and real-time adaptability under dynamic conditions. This work proposes a unified Nonlinear Model Predictive Controller (NMPC) framework that fully integrates perception-driven maneuvering, energy-efficient trajectory generation, formation maintenance, and robust information acquisition for vision-based target pursuit in UAV swarms. The framework shapes UAV trajectories to maintain proximity to a moving target while optimizing perception-related objectives, such as centering the target in each UAV’s field of view (FOV). Multi-agent collaboration is exploited to enhance target state observability by dynamically distributing UAVs around the target, while soft constraints ensure operational feasibility in rapidly changing environments. The proposed methodology is validated through extensive simulations across diverse target behaviors and environmental scenarios, demonstrating robustness, formation stability, and computational efficiency. Real-world experiments with a single UAV confirm the framework’s viability for operating in real-time, performing computations on-board. The results highlight the framework’s potential for scalable multi-agent UAV applications in surveillance, search and rescue, and environmental monitoring, combining perception-driven control with robust multi-agent coordination.
