We present the LunarLeaper mission concept, which aims to robotically investigate volcanic pits on the lunar surface. Volcanic pits, or skylights, are collapse features that may provide access to subsurface lava tubes, which could serve as shelters for future human explorers and offer insight into the volcanic history of the Moon by exposing ancient lava flows. The existence and extent of large caves are still debated today and require in situ analysis. The Marius Hills site in particular offers a potential entry point to a cave system in a volcanic region on the lunar nearside. Our mission aims to deploy a payload-equipped 15kg-class legged robot that can approach a pit, such as the Marius Hills pit, while taking measurements during the traverse. During the mission, measurements from a ground-penetrating radar (GPR) and a gravimeter will allow us to survey the subsurface and map any underlying lava tube, if present. The mission will investigate key questions regarding lunar volcanism, such as the existence and geometry of subsurface caves and the magnitude and timing of lava flows, while assessing the site’s suitability for future human utilization and habitation. Furthermore, the mission will demonstrate key enabling technologies such as legged robots, serving as building blocks for the next generation of planetary missions.

The spine plays a crucial role in the dynamic locomotion of quadrupedal animals, improving the stability, speed, and efficiency of their gait, especially for fast-paced and highly agile movements. Therefore, the spine is also a promising and natural way to extend the capabilities of quadruped robots. This paper empirically investigates the benefits of an actuated spine for learning agile quadruped locomotion. We evaluate whether the use of the spine brings benefits in terms of high-speed running, climbing stairs, climbing high-angle slopes, hurdling, and crawling scenarios. We conducted an empirical study in MuJoCo simulation using the Silver Badger robot from MAB Robotics with an actuated 1-DOF spine in the sagittal plane. The obtained results show that the use of the spine provides the robot with increased agility and allows it to overcome higher stairs, steeper slopes, higher obstacles, and smaller passages.

Model Predictive Control (MPC) is among the most widely adopted and reliable methods for robot control, relying critically on an accurate dynamics model. However, existing dynamics models used in the gradient-based MPC are limited by computational complexity and state representation. To address this limitation, we propose the Hyper Prediction Model (HyperPM) - a novel approach in which we project the unmodeled dynamics onto a time-dependent dynamics model. This time-dependency is captured through time-varying model parameters, whose evolution over the MPC prediction horizon is learned using a neural network. Such formulation preserves the computational efficiency and robustness of the base model while equipping it with the capacity to anticipate previously unmodeled phenomena. We evaluated the proposed approach on several challenging systems, including real-world F1TENTH autonomous racing, and demonstrated that it significantly reduces long-horizon prediction errors. Moreover, when integrated within the MPC framework (HyperMPC), our method consistently outperforms existing state-of-the-art techniques.

Fully autonomous vehicles promise enhanced safety and efficiency. However, ensuring reliable operation in challenging corner cases requires control algorithms capable of performing at the vehicle limits. We address this requirement by considering the task of autonomous racing and propose solving it by learning a racing policy using Reinforcement Learning (RL). Our approach leverages domain randomization, actuator dynamics modeling, and policy architecture design to enable reliable and safe zero-shot deployment on a real platform. Evaluated on the F1TENTH race car, our RL policy not only surpasses a state-of-the-art Model Predictive Control (MPC), but, to the best of our knowledge, also represents the first instance of an RL policy outperforming expert human drivers in RC racing. This work identifies the key factors driving this performance improvement, providing critical insights for the design of robust RL-based control strategies for autonomous vehicles.

Quadruped robots are currently a widespread platform for robotics research, thanks to powerful Reinforcement Learning controllers and the availability of cheap and robust commercial platforms. However, to broaden the adoption of the technology in the real world, we require robust navigation stacks relying only on low-cost sensors such as depth cameras. This paper presents a first step towards a robust localization, mapping, and navigation system for low-cost quadruped robots. In pursuit of this objective, we combine contact-aided kinematic, visual-inertial odometry, and depth-stabilized vision, enhancing the system’s stability and accuracy. Our results in simulation and two different real-world quadruped platforms show that our system can generate an accurate 2D map of the environment, robustly localize itself, and navigate autonomously. Furthermore, we present in-depth ablation studies of the system’s key components and their impact on localization accuracy. Videos, code, and additional experiments can be found on the project website.1

Data scarcity remains one of the most limiting factors in driving progress in robotics. However, the amount of available robotics data in the wild is growing exponentially, creating new opportunities for large-scale data utilization. Reliable temporal task completion prediction could help automatically annotate and curate this data at scale. The Generative Value Learning (GVL) approach was recently proposed, leveraging the knowledge embedded in vision-language models (VLMs) to predict task progress from visual observations. Building upon GVL, we propose OpenGVL, a comprehensive benchmark for estimating task progress across diverse challenging manipulation tasks involving both robotic and human embodiments. We evaluate the capabilities of publicly available open-source foundation models, showing that open-source model families significantly underperform closed-source counterparts, achieving only approximately $70\%$ of their performance on temporal progress prediction tasks. Furthermore, we demonstrate how OpenGVL can serve as a practical tool for automated data curation and filtering, enabling efficient quality assessment of large-scale robotics datasets. We release the benchmark along with the complete codebase at \href{github.com/budzianowski/opengvl}{OpenGVL}.

Trajectory planning under kinodynamic constraints is fundamental for advanced robotics applications that require dexterous, reactive, and rapid skills in complex environments. These constraints, which may represent task, safety, or actuator limitations, are essential for ensuring the proper functioning of robotic platforms and preventing unexpected behaviors. Recent advances in kinodynamic planning demonstrate that learning-to-plan techniques can generate complex and reactive motions under intricate constraints. However, these techniques necessitate the analytical modeling of both the robot and the entire task, a limiting assumption when systems are extremely complex or when constructing accurate task models is prohibitive. This paper addresses this limitation by combining learning-to-plan methods with reinforcement learning, resulting in a novel integration of black-box learning of motion primitives and optimization. We evaluate our approach against state-of-the-art safe reinforcement learning methods, showing that our technique, particularly when exploiting task structure, outperforms baseline methods in challenging scenarios such as planning to hit in robot air hockey. This work demonstrates the potential of our integrated approach to enhance the performance and safety of robots operating under complex kinodynamic constraints.

Deep Reinforcement Learning techniques are achieving state-of-the-art results in robust legged locomotion. While there exists a wide variety of legged platforms such as quadruped, humanoids, and hexapods, the field is still missing a single learning framework that can control all these different embodiments easily and effectively and possibly transfer, zero or few-shot, to unseen robot embodiments. We introduce URMA, the Unified Robot Morphology Architecture, to close this gap. Our framework brings the end-to-end Multi-Task Reinforcement Learning approach to the realm of legged robots, enabling the learned policy to control any type of robot morphology. The key idea of our method is to allow the network to learn an abstract locomotion controller that can be seamlessly shared between embodiments thanks to our morphology-agnostic encoders and decoders. This flexible architecture can be seen as a potential first step in building a foundation model for legged robot locomotion. Our experiments show that URMA can learn a locomotion policy on multiple embodiments that can be easily transferred to unseen robot platforms in simulation and the real world.

In recent years, the domain of robotic manipulation has broadened its focus from rigid objects to more complex tasks involving deformable objects. Compared to rigid bodies, the model-based manipulation of deformable objects requires online adaptation of the model, as it may change during manipulation. This challenge is also evident in the case of Defromable Linear Objects (DLOs), such as wires, hoses, or pipes. In this paper, we introduce a novel model-based method for manipulating DLO that eliminates the need for calibration between the robots and the RGBD camera. We achieve this by utilizing a local linear DLO model, represented by a Jacobian, which maps the movement of the robot's grippers to the observed DLO displacement. We propose updating this model online using a Recursive Least Squares (RLS) adaptive filter and three distinct Jacobian update strategies. We assess the efficiency of the proposed approaches in nine different real-world manipulation scenarios using three types of wires. The experiments conducted demonstrate that all the proposed strategies provide accurate DLO shape control. However, the update strategy that employs a single action per step, combined with an intermediate DLO state prediction, emerges as the most efficient.

Collaborative robots (cobots) are a promising technology for frontline workers in industry. They can support tasks that cannot be fully automated but are repetitive, fatiguing, boring, or dangerous for humans. Although cobots are explicitly designed to work with humans, they remain primarily non-intuitive and difficult to collaborate with. Thus, there is a need for new interaction approaches to facilitate efficient human-robot collaboration. Recently, we could see emerging examples of using augmented reality (AR) to assist a worker in collaborative task execution with a cobot. However, for such an approach to provide truly efficient support for the seamless bimanual task execution, we need to first investigate interaction methods offered by an AR interface. To that end, we performed a study with sixteen participants to compare eye-gaze and hand-ray pointing methods for part selection in collaborative, manual assembly tasks. The results of our study show that both techniques provide similar perceived usability, with the eye-gaze selection leading to significantly shorter completion times.

Advances in engineering enable wheeled-legged hybrid locomotion, an achievement not feasible in biological systems.

Velocity estimation is of great importance in autonomous racing. Still, existing solutions are characterized by limited accuracy, especially in the case of aggressive driving or poor generalization to unseen road conditions. To address these issues, we propose to utilize Unscented Kalman Filter (UKF) with a learned dynamics model that is optimized directly for the state estimation task. Moreover, we propose to aid this model with the online-estimated friction coefficient, which increases the estimation accuracy and enables zero-shot adaptation to the new road conditions. To evaluate the UKF-based velocity estimator with the proposed dynamics model, we introduced a publicly available dataset of aggressive manoeuvres performed by an F1TENTH car, with sideslip angles reaching 40°. Using this dataset, we show that learning the dynamics model through UKF leads to improved estimation performance and that the proposed solution outperforms state-of-the-art learning-based state estimators by 17% in the nominal scenario. Moreover, we present unseen zero-shot adaptation abilities of the proposed method to the new road surface thanks to the use of the proposed learning-based tire dynamics model with online friction estimation.

Motion planning is a mature area of research in robotics with many well-established methods based on optimization or sampling the state space, suitable for solving kinematic motion planning. However, when dynamic motions under constraints are needed and computation time is limited, fast kinodynamic planning on the constraint manifold is indispensable. In recent years, learning-based solutions have become alternatives to classical approaches, but they still lack comprehensive handling of complex constraints, such as planning on a lower-dimensional manifold of the task space while considering the robot's dynamics. This paper introduces a novel learning-to-plan framework that exploits the concept of constraint manifold, including dynamics, and neural planning methods. Our approach generates plans satisfying an arbitrary set of constraints and computes them in a short constant time, namely the inference time of a neural network. This allows the robot to plan and replan reactively, making our approach suitable for dynamic environments. We validate our approach on two simulated tasks and in a demanding real-world scenario, where we use a Kuka LBR Iiwa 14 robotic arm to perform the hitting movement in robotic Air Hockey.

This article introduces passive wheels as the new localization system for automated guided vehicles (AGVs). The article focuses on investigating the accuracy of the proposed system and comparing with other widely used solutions: rotary encoders coupled with drive wheels, AHRS and LiDAR scanner. The fusion of dead reckoning and inertial data is acquired by the implementation of the Kalman filter. On the other hand, the fusion of LiDAR data depends on application of the AMCL or the graph-based algorithm. The study was conducted on five different scenarios, designed to investigate the influence of specific types of movements on the performance of tested localization methods. Results indicate, that passive wheels dead reckoning outperforms drive wheels dead reckoning in most scenarios, minimizing errors due to reduced both longitudal and lateral slippages, which appear during AGV accelerating, decelerating and turning. AHRS integration improves accuracy, especially in scenarios involving significant amount of angular motion. LiDAR-based methods in short term show mediocre results, due to relatively high, but steady values of error. The study highlights the importance of the map's quality for LiDAR-based techniques and points up the conditions, under which the LiDAR-based techniques do not operate very well. In conclusion, the research provides insights into the strengths and weaknesses of various AGV localization techniques in various movement scenarios, emphasizing the impact of sensor choice and path-shape conditions on accuracy.

This paper proposes a neural network model for terrain classification using force/torque signals registered with sensors mounted on robot feet. The proposed model is based on the latest architecture for Linear-Time Sequence Modeling using Selective State Spaces, called Mamba. We obtained lightweight models with very low inference times, which are two times faster than comparable transformer-based solutions. We evaluated HAPmamba alongside other state-of-the-art approaches, and while the classification measures are comparable, HAPmamba is the fastest among all evaluated models.

Manipulating Deformable Linear Objects (DLOs) is a challenging task for a robotic system due to their unpredictable configuration, high-dimensional state space and complex nonlinear dynamics. This paper presents a framework addressing the manipulation of DLOs, specifically targeting the model-based shape control task with the simultaneous online gradient-based estimation of model parameters. In the proposed framework, a neural network is trained to mimic the DLO dynamics using the data generated with an analytical DLO model for a broad spectrum of its parameters. The neural network-based DLO model is conditioned on these parameters and employed in an online phase to perform the shape control task by estimating the optimal manipulative action through a gradient-based procedure. In parallel, gradient-based optimization is used to adapt the DLO model parameters to make the neural network-based model better capture the dynamics of the real-world DLO being manipulated and match the observed deformations. To assess its effectiveness, the framework is tested across a variety of DLOs, surfaces, and target shapes in a series of experiments. The results of these experiments demonstrate the validity and efficiency of the proposed methodology compared to existing methods. Project website at https://sites.google.com/view/dlo-manipulation .

Articulated vehicles with multiple trailers also called N-Trailers have been widely used for transportation tasks in industrial applications. Solutions for the control problem of N-Trailers have been formulated mostly for vehicles with solely on-axle hitching or off-axle hitching but in field applications such as agriculture, most of the structures used are Generalized N-Trailers (GNT) which have a combination of on- and off-axle hitching. Moreover, most of the solutions in literature were developed and tested for laboratory-scale platforms under ideal indoor conditions. However, in outdoors conditions, the motion performance is commonly degraded by model uncertainties, slipping of wheels, and trailers localization loss product of noisy or inaccurate sensor data. In this context, this paper reports the use of Active Disturbance Rejection Control (ADRC) with a Dual-Stage Disturbance Observer (DS-DO) to improve the backward trajectory-tracking performance of GNT in no ideal conditions, where the DS-DO aims to attenuate the effects of error propagation on the ADRC compensation loop and improve the overall closed-loop performance. The proposed ADRC+DS-DO has been validated in simulation and real experiments showing overall improvements on the controller effort reduction and reduction of up to 57% on the tracking error against a traditional ADRC approach already existent in the literature.

Wiring harnesses, i.e. a collection of electrical cables organized into branches, are vastly present in the automotive industry. Moreover, the number of wires and overall weight of automotive wiring harnesses are steadily increasing over time. Deformable wiring harness bags were introduced by manufacturers to simplify assembly operations. However, this task is still entirely performed manually by human labor. Despite the efforts, the degree of automation in wiring harness assembly is still close to zero. Due to the lack of task-specific datasets, modern state-of-the-art computer vision approaches are not commonly employed in the wiring harness industrial processes. In this work, we propose an approach to generate a dataset of a specific object of interest, i.e. deformable wiring harness bags, with minimal effort employing the copy and paste technique. The obtained dataset is validated on the semantic segmentation task in a real-world test setup, consisting of laboratory and automotive factory environments. An overall IoU of 53.8% and Dice score of 65.6% is obtained, demonstrating the capability of the proposed method.

Moon exploration has gained significant momentum in recent decades, with growing interest from space agencies and private investors. A diverse range of activities is associated with moon exploration, encompassing spacecraft design, payload transportation, launcher capabilities, resource identification and mining, and establishing a sustained presence on our only natural satellite. Such endeavors would require the shipment of both scientific and life-sustaining equipment. However, communication between Earth’s mission control and the Moon’s bases can still present challenges. In this paper, we introduce Cognition – a rover-lander distributed system that approaches this problem by distributing the data processing between the rover and the lander. The primary goal of the Cognition system is to optimize lunar surface exploration by minimizing data transmission to the Earth’s surface, prioritizing the transfer of valuable data, and augmenting the level of autonomy in the process.