The control of fully actuated hexa-rotors is highly dependent on their physical parameters, especially those describing the rotors' positions and orientations. Uncertainties in these parameters significantly affect the performance of classical control approaches, particularly in small drones, where manufacturing tolerances amplify these uncertainties. To address this challenge, we propose a novel adaptive and robust control strategy that compensates for parameter uncertainties and external disturbances without requiring precise prior knowledge of the rotor poses. Unlike existing methods, our approach explicitly incorporates motor dynamics into the control design, resulting in a more realistic and implementable framework. Using Lyapunov-based stability analysis, we demonstrate the global asymptotic stability of the proposed control system under parameter uncertainties and disturbances. Extensive simulations validate the efficacy of our method, showcasing superior tracking performance and robustness compared to conventional controllers. This work represents a significant step toward enabling fully actuated multi-rotor UAVs to perform in real-world scenarios with uncertain and dynamic environments.
Imitation learning (IL) policies in robotics deliver strong performance in controlled settings but remain brittle in real-world deployments: rare events such as hardware faults, defective parts, unexpected human actions, or any state that lies outside the training distribution can lead to failed executions. Vision-based Anomaly Detection (AD) methods emerged as an appropriate solution to detect these anomalous failure states but do not distinguish failures from benign deviations. We introduce FIDeL (Failure Identification in Demonstration Learning), a policy-independent failure detection module. Leveraging recent AD methods, FIDeL builds a compact representation of nominal demonstrations and aligns incoming observations via optimal transport matching to produce anomaly scores and heatmaps. Spatio-temporal thresholds are derived with an extension of conformal prediction, and a Vision-Language Model (VLM) performs semantic filtering to discriminate benign anomalies from genuine failures. We also introduce BotFails, a multimodal dataset of real-world tasks for failure detection in robotics. FIDeL consistently outperforms state-of-the-art baselines, yielding +5.30
This paper describes the Inria team's integrated robotics system used in the 1st euROBIN coopetition, during which service robots performed voice-activated household tasks in a kitchen setting. The team developed a modified TIAGo++ platform that leverages a whole-body control stack for autonomous and teleoperated modes, and an LLM-based pipeline for instruction understanding and task planning. The key contributions (opens-sourced) are the integration of these components and the design of custom teleoperation devices, addressing practical challenges in the deployment of service robots.
The ability of Flow Matching (FM) to model complex conditional distributions has established it as the state-of-the-art for prediction tasks (e.g., robotics, weather forecasting). However, deployment in safety-critical settings is hindered by a critical extrapolation hazard: driven by smoothness biases, flow models yield plausible outputs even for off-manifold conditions, resulting in silent failures indistinguishable from valid predictions. In this work, we introduce Diverging Flows, a novel approach that enables a single model to simultaneously perform conditional generation and native extrapolation detection by structurally enforcing inefficient transport for off-manifold inputs. We evaluate our method on synthetic manifolds, cross-domain style transfer, and weather temperature forecasting, demonstrating that it achieves effective detection of extrapolations without compromising predictive fidelity or inference latency. These results establish Diverging Flows as a robust solution for trustworthy flow models, paving the way for reliable deployment in domains such as medicine, robotics, and climate science.
Teleoperating precise bimanual manipulations in cluttered environments is challenging for operators, who often struggle with limited spatial perception and difficulty estimating distances between target objects, the robot's body, obstacles, and the surrounding environment. To address these challenges, local robot perception and control should assist the operator during teleoperation. In this work, we introduce a safe teleoperation system that enhances operator control by preventing collisions in cluttered environments through the combination of immersive VR control and voice-activated collision avoidance. Using HTC Vive controllers, operators directly control a bimanual mobile manipulator, while spoken commands such as "avoid the yellow tool" trigger visual grounding and segmentation to build 3D obstacle meshes. These meshes are integrated into a whole-body controller to actively prevent collisions during teleoperation. Experiments in static, cluttered scenes demonstrate that our system significantly improves operational safety without compromising task efficiency.
Quality diversity (QD) algorithms, such as the multidimensional archive of phenotypic elites (MAP-Elites), are a class of optimization techniques that attempt to find many high-performing points that all behave differently according to a user-defined behavioral metric. In this article we propose the Bayesian optimization of elites (BOP-Elites) algorithm. Designed for problems with expensive fitness functions and coupled behavior descriptors, it is able to return a QD solution-set with excellent performance already after a relatively small number of samples. BOP-Elites models both fitness and behavioral descriptors with Gaussian Process surrogate models and uses Bayesian optimization strategies for choosing points to evaluate in order to solve the quality-diversity problem. In addition, BOP-Elites produces high-quality surrogate models which can be used after convergence to predict solutions with any behavior in a continuous range. An empirical comparison shows that BOP-Elites significantly outperforms other state-of-the-art algorithms without the need for problem-specific parameter tuning.
Air ducts are integral to modern buildings but are challenging to access for inspection. Small quadrotor drones offer a potential solution, as they can navigate both horizontal and vertical sections and smoothly fly over debris. However, hovering inside air ducts is problematic due to the airflow generated by the rotors, which recirculates inside the duct and destabilizes the drone. In this article, we map the aerodynamic forces that affect a hovering drone in a duct using a robotic setup and a force/torque sensor. Based on the collected aerodynamic data, we identify a recommended position for stable flight, which is not the center of a circular duct. We then develop a neural network-based positioning system that leverages low-cost time-of-flight sensors. By combining these aerodynamic insights and the data-driven positioning system, we show how to improve the stability of a small quadrotor drone (here, 180 mm) inside small air ducts (down to 350 mm diameter) and fly autonomously over 2 m.
Imitation learning is a promising approach for enabling generalist capabilities in humanoid robots, but its scaling is fundamentally constrained by the scarcity of high-quality expert demonstrations. This limitation can be mitigated by leveraging suboptimal, open-ended play data, often easier to collect and offering greater diversity. This work builds upon recent advances in generative modeling, specifically Flow Matching, an alternative to Diffusion models. We introduce a method for estimating the minimum or maximum of the learned distribution by leveraging the unique properties of Flow Matching, namely, deterministic transport and support for arbitrary source distributions. We apply this method to develop several goal-conditioned imitation and reinforcement learning algorithms based on Flow Matching, where policies are conditioned on both current and goal observations. We explore and compare different architectural configurations by combining core components, such as critic, planner, actor, or world model, in various ways. We evaluated our agents on the OGBench benchmark and analyzed how different demonstration behaviors during data collection affect performance in a 2D non-prehensile pushing task. Furthermore, we validated our approach on real hardware by deploying it on the Talos humanoid robot to perform complex manipulation tasks based on high-dimensional image observations, featuring a sequence of pick-and-place and articulated object manipulation in a realistic kitchen environment. Experimental videos and code are available at: https://hucebot.github.io/extremum_flow_matching_website/
Logistics and service operations involving parcel preparation, delivery, and unpacking from a supply point to a user's home could be carried out completely by robots in the near future, taking advantage of the capabilities of the different robot morphologies for the logistics, outdoor, and domestic environments. The use of robots for parcel delivery can contribute to the goals of sustainability and reduced emissions by exploiting their different locomotion modalities (wheeled, legged, and aerial). This article reports the development and results obtained from the first robotics hackathon celebrated as part of the European Robotics and Artificial Intelligence Network involving eight robotic platforms in three domains: 1) an industrial robotic arm for parcel preparation at the supply point, 2) a Centauro robot, a dual-arm aerial manipulator, and a wheeled-legged quadruped for parcel transportation, and 3) two humanoid robots and two commercial mobile manipulators for parcel delivery and unpacking in domestic scenarios. The article describes the joint operation and the evaluation scenario, the features and capabilities of the robots, particularly those involved in the realization of the tasks, and the lessons learned.
branch of computer science known as genetic programming has been given a boost with the application of large language models that are trained on the combined intuition of the world’s programmers.
Humanoid robots could benefit from using their upper bodies for support contacts, enhancing their workspace, stability, and ability to perform contact-rich and pushing tasks. In this paper, we propose a unified approach that combines an optimization-based multi-contact whole-body controller with Flow Matching, a recently introduced method capable of generating multi-modal trajectory distributions for imitation learning. In simulation, we show that Flow Matching is more appropriate for robotics than Diffusion and traditional behavior cloning. On a real full-size humanoid robot (Talos), we demonstrate that our approach can learn a whole-body non-prehensile box-pushing task and that the robot can close dishwasher drawers by adding contacts with its free hand when needed for balance. We also introduce a shared autonomy mode for assisted teleoperation, providing automatic contact placement for tasks not covered in the demonstrations. Full experimental videos are available at: https://hucebot. github.io/flow_multisupport_website/
Optimizing a set of functions simultaneously by leveraging their similarity is called multi-task optimization.Current black-box multitask algorithms only solve a finite set of tasks, even when the tasks originate from a continuous space.In this paper, we introduce Parametric-Task MAP-Elites (PT-ME), a new black-box algorithm for continuous multi-task optimization problems.This algorithm (1) solves a new task at each iteration, effectively covering the continuous space, and (2) exploits a new variation operator based on local linear regression.The resulting dataset of solutions makes it possible to create a function that maps any task parameter to its optimal solution.We show that PT-ME outperforms all baselines, including the deep reinforcement learning algorithm PPO on two parametric-task toy problems and a robotic problem in simulation.
This paper presents Words2Contact, a language-guided multi-contact placement pipeline leveraging large language models and vision language models. Our method is a key component for language-assisted teleoperation and human-robot cooperation, where human operators can instruct the robots where to place their support contacts before whole-body reaching or manipulation using natural language. Words2Contact transforms the verbal instructions of a human operator into contact placement predictions; it also deals with iterative corrections, until the human is satisfied with the contact location identified in the robot’s field of view. We benchmark state-of-the-art LLMs and VLMs for size and performance in contact prediction. We demonstrate the effectiveness of the iterative correction process, showing that users, even naive, quickly learn how to instruct the system to obtain accurate locations. Finally, we validate Words2Contact in real-world experiments with the Talos humanoid robot, instructed by human operators to place support contacts on different locations and surfaces to avoid falling when reaching for distant objects.
In industrial workstations, the morphology of the worker is a key factor for the feasibility and the ergonomics of an activity. Existing digital human modeling tools can simulate different morphologies at work, but hardly scale to a large population of workers because of limited consideration of morphology-specific behaviors and computational cost. This letter presents a framework to efficiently evaluate the suitability of a workstation over a large population of workers in a physics-based simulation. Activities are simulated through a two-step optimization process, involving a quadratic-programming-based whole-body controller and a multi-task optimizer for behavioral adaptation. On a screwdriving scenario, we demonstrate how our framework can help ergonomists improve workstation designs thanks to the resulting suitability maps where generated behaviors are optimized for each morphology w.r.t. ergonomics and performance.
Many humanoid and multi-legged robots are controlled in positions rather than in torques, which prevents direct control of contact forces, and hampers their ability to create multiple contacts to enhance their balance, such as placing a hand on a wall or a handrail. This paper introduces the SEIKO (Sequential Equilibrium Inverse Kinematic Optimization) pipeline, and proposes a unified formulation that exploits an explicit model of flexibility to indirectly control contact forces on traditional position-controlled robots. SEIKO formulates whole-body retargeting from Cartesian commands and admittance control using two quadratic programs solved in real time. Our pipeline is validated with experiments on the real, full-scale humanoid robot Talos in various multi-contact scenarios, including pushing tasks, far-reaching tasks, stair climbing, and stepping on sloped surfaces. Code and videos are available at: https://hucebot.github.io/seiko_controller_website/
We address the problem of grasping unknown objects identified from top-down images with a parallel gripper. When no object 3D model is available, the state-of-the-art grasp generators identify the best candidate locations for planar grasps using the RGBD image. However, while they generate the Cartesian location and orientation of the gripper, the height of the grasp center is often determined by heuristics based on the highest point in the depth map, which leads to unsuccessful grasps when the objects are not thick, or have transparencies or curved shapes. In this paper, we propose to learn a regressor that predicts the best grasp height based from the image. We train this regressor with a dataset that is automatically acquired thanks to the DIGIT optical tactile sensors, which can evaluate grasp success and stability. Using our predictor, the grasping success is improved by 6% for all objects, by 16% on average on difficult objects, and by 40% for objects that are notably very difficult to grasp (e.g., transparent, curved, thin).
This short paper outlines two recent works on multi-contact teleoperation and the development of the SEIKO (Sequential Equilibrium Inverse Kinematic Optimization) framework. SEIKO adapts commands from the operator in real-time and ensures that the reference configuration sent to the underlying controller is feasible. Additionally, an admittance scheme is used to implement physical interaction, which is then combined with the operator's command and retargeted. SEIKO has been applied in simulations on various robots, including humanoid and quadruped robots designed for loco-manipulation. Furthermore, SEIKO has been tested on real hardware for bimanual heavy object carrying tasks.
The use of MAP-Elites in high-dimensional behavioral spaces requires a scalable method for dividing the space into regions of equal volume. So far, the recommended approach to generate these regions has been the Centroidal Voronoi Tesselation (CVT), but this algorithm has a significant computational cost (typically a few minutes for more than 50 dimensions). In this paper, we investigate alternative approaches to generate regions of equal volumes for MAP-Elites. In particular, we experiment with generating region centroids with low-discrepancy sequences (Sobol, Halton), pseudorandom numbers, and a simple blue noise generator. Our results show that, for spaces with 100 dimensions, most methods perform similarly, including pseudo-random numbers. For spaces with dimensions between 5 and 50, a CVT generates significantly better centroids. In lower dimensions (1--5), a scrambled Sobol sequence generates well-spread centroids in a few milliseconds.