Multi-UAV relative localization using onboard 3D LiDAR is challenged by intermittent target detections caused by sparse point returns and field-of-view limitations. In the outdoor experiments considered, valid detections are available in only 13–41% of scan frames, resulting in detection outages of up to 11 s, during which standard kinematic predictors accumulate substantial drift. To address this problem, a semi-cooperative dual-inertial measurement unit (IMU) framework is proposed, in which the target UAV transmits inertial measurements and attitude estimates to the observing UAV. An Extended Kalman Filter (EKF) fuses inertial data from both platforms to propagate the relative state through detection outages, while a lightweight spatiotemporal consistency mechanism rejects clutter-induced false detections prior to filter updates. Six EKF configurations are evaluated, including kinematic baselines based on constant velocity (CV) and constant turn rate and velocity (CTRV), as well as four dual-IMU formulations with different bias parameterizations. Results over 13 outdoor flight segments show that dual-IMU propagation consistently outperforms kinematic prediction, and that a minimal 9-DOF formulation provides the best trade-off between estimation accuracy, robustness, and computational cost. In particular, the proposed 9-DOF model reduces relative position root mean square error by 64% compared with the CV baseline while maintaining real-time performance during extended LiDAR detection outages.
As humanoid robots become increasingly integrated into daily environments, developing systems capable of socially intelligent and intuitive Human–Robot Interaction (HRI) is essential. Mutual gaze, the shared exchange of eye contact, is a critical nonverbal cue for attention and intent recognition. However, gaze estimation in real-world, multi-person environments remains challenging due to varying illumination, occlusions, and computational constraints.This work presents a dual-Field-of-View (FOV) gaze estimation framework for the Vizzy humanoid robot, combining wide-angle and zoom Red-Green-Blue (RGB) cameras to balance coverage and precision. A lightweight gaze estimation model based on L2CS-Net [1] was optimized through Knowledge Distillation (KD), transferring knowledge from a teacher to a compact student network. The system achieves real-time inference, ≈18 Frames per Second (FPS), with less than 3° mean Angular Error (AE) degradation compared to the teacher. Experiments conducted in Gazebo simulation and real-world scenarios confirm that the dual-FOV setup significantly improves mutual gaze recognition in dynamic, multi-person environments, enabling robots to detect human attention cues more efficiently.
This work presents a complete autonomous racing stack for a 1/10-scale F1TENTH vehicle, addressing perception, planning, and control on embedded hardware. The perception module fuses LiDAR and IMU data via an Extended Kalman Filter (EKF) and Cartographer-based SLAM for drift-compensated localization. A planner was implemented that extracts smooth centerlines from occupancy grids, coupled with Kinematic and Dynamic Model Predictive Control (MPC) in curvilinear coordinates. Experiments in simulation and in a real scenario demonstrated real-time feasibility, stable trajectory tracking, and bounded errors. This framework offers a reproducible approach for deploying optimization-based control near the limits of handling on resource-constrained systems 1.
Automated weld inspection is a critical task in industrial manufacturing, as undetected defects can compromise structural integrity, safety, and product quality. This paper investigates computer vision and deep learning models for the detection and classification of weld surface defects. A custom dataset of weld images was acquired using a dedicated image acquisition system, and several detection and classification model configurations were investigated. A hybrid detection–classification strategy is proposed, in which YOLOv8n performs defect localization and ConvNeXt-Small is employed for defect classification, with particular emphasis on underrepresented defect classes. Experimental results demonstrate that the proposed hybrid approach outperforms single-model strategies, highlighting the importance of dataset design choices and the effectiveness of deep learning-based methods for automated weld inspection.
Synthetic data has become a crucial enabler for training deep learning models in robotic perception tasks. This work presents a configurable synthetic data generation pipeline, built on BlenderProc, that leverages CAD model annotations to produce part-based labels directly at the geometry level. The proposed approach enables the automatic generation of Oriented Bounding Box (OBB) annotations and supports the definition of multiple regions of interest through a vertex-level annotation tool. The pipeline is validated in an warehouse environment use case involving a storage container for automotive parts for autonomous forklift operations. Experimental results show that YOLO11m-Oriented Bounding Box model trained exclusively on synthetic data achieve strong performance on real-world images, reaching up to 0.768 mAP@0.5:0.95. Furthermore, increasing synthetic data diversity consistently improves detection performance, and hybrid training with a small amount of real data provides an effective trade-off between annotation effort and accuracy.
Mobile manipulators combine the mobility of a mobile base with the manipulation capabilities of robotic arms, which results in a high-dimensional control space when trying to achieve autonomous navigation. This paper presents a reinforcement learning approach aiming to achieve simultaneous base and manipulator autonomous local planning capabilities while ensuring collision-free navigation through a teacher-student learning framework, where a teacher agent, trained with privileged information, serves as a reference for a student that learns from sensor data. The objective of the robot is to reach a target position and orientation with its base, as well as a desired arm configuration, while perceiving the three-dimensional environment through a downsampled point cloud representation, reducing the high-dimensionality of point clouds generated by raw sensor data and enhancing generalization to new obstacle shapes not seen during training. The results demonstrate that the robot is capable of navigating in cluttered environments, successfully transferring the knowledge gained during training in scenarios from simple-shaped objects to more complex ones.
This work addresses the generation of collision-free, human-like movements in an anthropomorphic robotic assistant during collaborative tasks, guided by ergonomic constraints for human operators. Human-like motion enhances natural interaction and reduces cognitive workload, while ergonomic constraints improve physical comfort and further reduce mental workload. The study focuses on a window assembly task involving the handover of aluminum profiles. A novel ergonomic model is introduced to improve human posture by adjusting objects’ delivery position and orientation, minimizing the operator’s physical and cognitive load. The computed handover position induces comfortable postures, as evaluated by the Rapid Upper Limb Assessment (RULA), and the profiles’ orientation simplifies subsequent tasks, avoiding reorientation. Simulated and real-world experiments were conducted to validate the approach. Results indicate a notable improvement in operator comfort, reflected by a overall reduction in RULA scores under ergonomic constraints. Crucially, integrating ergonomic constraints did not reduce the human-like quality of the robot’s movements. Operators’ feedback, collected through a quantitative questionnaire, further confirms these findings. This research fills a critical gap in human-robot collaboration by ensuring both natural interaction and operator well-being through a balance of motion planning and ergonomic consideration.
The state-of-the-art for in-hand manipulation works in limited hand poses, usually using the palm as the support surface of the object, with no capacity to deal with diversity in object shape. These constraints limit the deployment of humanoid robotics in real-world applications. In this work, we present a method that combines and iterates on existing approaches to address these performance restrictions. Our method separates the grasping and manipulation tasks by creating a dedicated pose initialization policy and a manipulation policy, allowing the latter to focus solely on its specific task. In our manipulation policy, we extended an existing approach that uses learned object shape encodings by integrating orientation information and employing a gravity curriculum to aid in learning. Testing on the pose initialization policy showed good overall performance and generalization capabilities, whereas the manipulation policy, despite multiple iterations involving reward shaping, simulator configurations, and reset condition changes, did not result in any successful agents. Our results suggest that achieving in-hand manipulation requires either the incorporation of demonstration-based learning or the use of a hand with additional degrees of movement.
This paper presents a connectivity-aware extension to a Simulated Annealing-based multi-UAV Coverage Path Planning (CPP) framework. Motivated by mission-critical applications such as search and rescue, where timely information reporting is as important as target detection, the proposed approach integrates inter-UAV network connectivity directly into the planning objective through a soft penalisation mechanism. The resulting framework jointly evaluates coverage performance and communication availability, and introduces metrics that capture both detection efficiency and reporting delay. Simulation results reveal a controllable trade-off between coverage effectiveness and communication robustness, showing that moderate connectivity enforcement can substantially improve information relay without severely degrading detection performance. These findings provide practical insight into the deployment of cooperative UAV teams in environments with limited or no communication infrastructure.
Path optimization in swarm robotics is typically validated in simulation, with limited implementation on physical multi-robot platforms. This paper presents a methodology integrating physical robotic swarms with real-time mapping and online multi-objective path optimization using NSGA-II, rarely implemented beyond simulation. A key contribution is incorporating danger (representing a pheromone spatial distribution) as an optimization objective alongside distance, enabling safe navigation in hazardous environments. Validation with up to three Atta-Bot robots demonstrated that coverage increased consistently with robot count and execution time, aligning with multi-robot exploration literature. NSGA-II successfully identified Pareto-optimal paths balancing distance minimization and danger avoidance. Results showed high correlation between distance and danger (R2 = 0.96), though both objectives derive from independent spatial factors. This framework enables danger-aware cooperative navigation for applications in disaster response, environmental monitoring, and precision agriculture.
In human–robot interaction (HRI), first impressions critically shape user comfort and the success of collaborative relationships. In the context of a mobile robot, a key issue is to plan the right trajectory to meet a person and start the interaction. Despite advances in trajectory prediction algorithms leveraging visual data and transformer architectures, it remains unclear how the complexity of these prediction models affects human perception during robot approaches, especially under real-time responsiveness constraints. This study compares two trajectory prediction models: (i) a simple linear regression model and (ii) a complex transformer-based model; both implemented on the Vizzy social robot. Using monocular RGB cameras and a human detection model, predicted trajectories were used to define the approach goal point to the robot. Twenty-eight participants engaged with one of the two models in a controlled environment and completed validated questionnaires assessing anthropomorphism, trust, and user experience. The results suggest that the complex model moderately enhanced participants’ perception of the robot, particularly in terms of trust and efficiency.
A key step for successful navigation in outdoor environments is traversability analysis - the ability to compute the feasibility and/or effort associated with the traversal of a certain portion of terrain. Traversability analysis is often hindered by multiple factors, such as the unstructured nature of the environment and sensor occlusion, rendering the estimates inaccurate. We propose P-STEER, a method to analyze the long-term traversability of a ground robot considering its proprioceptive data. Using the robot’s instantaneous current draw while traveling through previously observed terrain, the system is able to reliably learn and accurately predict future traversability estimates, enabling autonomous robots to operate in large-scale environments with increased effectiveness. Our method is evaluated experimentally using a quadruped robot, in real-world natural settings, including a dense forest. We compare it to a state-of-the-art approach and show that it outperforms it in both known and unseen rough terrain conditions. The open-source software is available at https://github.com/Forestry-Robotics-UC/P-STEER.
Deploying a team of small tailsitter UAVs in mid-air from a large mothership aircraft can significantly extend operational range in remote missions (for example, maritime search and rescue), enabling efficient and fast area coverage through cooperation and hybrid locomotion capabilities of hovering and forward flight. However, such deployment creates a safety-critical need for rapid recovery after an unstable release, when the vehicle may exhibit large attitude errors and nonzero initial velocities. This paper addresses recovery control of a quadrotor tailsitter without any control surfaces under these conditions. We adopt a two-stage recovery procedure that first stabilizes the vehicle inclination and only then regulates heading and altitude, using a quaternion-based decomposition of attitude error into inclination and heading components. Two offboard control strategies are implemented and compared: (i) a cascaded attitude and altitude PID design augmented with aerodynamic feedforward terms, and (ii) a nonlinear model predictive controller (NMPC) based on the nonlinear vehicle model with explicit thrust and torque constraints. Software-in-the-loop simulations evaluate upset recovery, high-speed recovery, and robustness under stochastic wind disturbances. In upset recovery, NMPC completes the second recovery stage in 0.66 s (PID: 5.0 s) and reduces maximum altitude drop (6.41 m vs 33.14 m). In Monte Carlo wind tests, PID achieves higher success rates as wind intensity increases (for example, 70% vs 50% at the highest tested wind mean), while NMPC generally yields smaller height drop when successful. Overall, NMPC improves recovery speed and altitude loss when the model is accurate, whereas PID is more robust to unmodeled wind and is computationally lighter.
This paper presents a vision-guided bin picking system for stiff black fabric parts used in overmolded polymer injection parts for the automotive industry. The proposed approach combines deep learning-based object detection with classical vision methods to address low contrast and cluttered scenes. A single RGBD camera mounted on the robot end effector is used for picking point detection and verification, supported by a mirror-based inspection stage that avoids additional sensors. Experimental results under varying backgrounds and part densities demonstrate reliable placement and cycle times compatible with industrial requirements. The system's performance was assessed through a "single-fill test" and trials with varying part densities in the bin.
Handling delicate products with high variability in shape and texture remains a key limitation of rigid industrial end-effectors. This paper evaluates three Fin Ray Effect (FRE)-inspired soft gripper geometries: Straight, Beak, and Constant Curve, manufactured by 3D printing and assessed through coupled numerical and experimental analysis. Finite element simulations were conducted in Ansys using a hyperelastic material representation of TPU, and a cylindrical object was subjected to a prescribed displacement of 50 mm under low-friction contact to quantify deformation patterns, stress/strain fields, and reaction forces. Experimental tests were performed using a dedicated fixture on a universal testing machine, replicating the gripper arrangement and measuring force–displacement response for cross-validation. Results show clear geometry-dependent trade-offs: the Straight design delivers the highest reaction force and is therefore suitable for tasks requiring firmer retention; the Beak design provides the lowest force and stress, supporting gentler interaction with delicate or fragile objects; and the Constant Curve design offers an intermediate solution with more uniform contact pressure distribution. This work serves as a foundation for optimising gripper design to improve handling capabilities across diverse applications.
This paper evaluates the use of the OPC UA publisher/subscriber (PubSub) protocol in a real-time motion control case study, specifically, a Conveyor Tracking application using a GoFa10-CRB15000 collaborative robot and a linear conveyor belt. The robot’s goal is to maintain the end-effector in contact with a touch screen placed in the conveyor, maintaining a predetermined relative pose, independently of the conveyor motion. To achieve this, the conveyor’s encoder signals are sent to the robot, in real-time, using OPC UA PubSub. The communication is established through low cost Raspberry Pi 5 single board computers that serve as gateways between the devices. Tests are performed using standard Ethernet and WiFi 7, using dedicated and shared networks (i.e. with and without additional traffic in the network). The touch screen is used to sense the point of contact and measure the positioning error.Results show that using OPC UA PubSub, over standard Ethernet or WiFi 7, is a feasible solution for real-time motion tracking and control. The measured tracking errors obtained with the OPC UA PubSub communication were comparable to the reference case, where the robot uses encoder signals through direct electrical connections.
As drone technology scales from single-agent operations to coordinated teams, traditional two-dimensional interfaces often fail to provide the spatial context and intuitive control necessary for effective supervision. To address these challenges, this paper introduces a novel unified Mixed Reality (MR) framework designed to enhance operator situational awareness and streamline the dynamic control of multiple Unmanned Aerial Vehicles (UAVs). The system utilizes a Meta Quest 3 headset to provide an immersive planning interface, enabling operators to define waypoint-based missions and generate automated volumetric area mapping paths using intuitive spatial inputsThe proposed architecture integrates a custom ROS–Unity bridge (Middleman) deployed on the drone’s companion computer, which facilitates reliable, bidirectional telemetry streaming and supports dynamic mission management, including real-time preemption and automated safety failsafes. The framework was validated through Software-In-The-Loop (SITL) simulations and hardware testing, demonstrating a mean command latency of approximately 408 ms and linear network scalability. Furthermore, a user study (N=15) confirmed that the spatial registration of mission data significantly reduces cognitive load, with participants reporting high confidence in the interaction mechanics.
This paper proposes a method for implementing an Adaptive Cruise Control system based on positioning information from a Global Navigation Satellite System. The aim of such a system is to be implemented in a car-sharing solution, where each vehicle is connected to a main server that shares crucial information among them. Therefore, we present a solution for this type of ADAS system that, contrarily to most solutions currently in the market, does not rely on distance sensors. To achieve this, we developed a positioning system based on GPS coordinates that fuses multiple sensors for greater robustness, an algorithm that identifies the leading vehicle, and an adaptive cruise control algorithm that leverages the connectivity provided by this solution. The results were obtained both in simulation for testing in road vehicles and in a real scenario, where tests were conducted using two mobile platforms connected via the Internet, each replicating a vehicle’s motion.
Soft anthropomorphic robotic hands offer intrinsic compliance, adaptability, and passive safety, making them well suited for manipulation tasks in unstructured human environments. Their integration into robotic systems is essential to enhance the capabilities of modern anthropomorphic robots and promote acceptance in human–robot collaboration. This paper presents a kinematic and collision-aware simulation model of the qb SoftHand 2 to support human-like manipulation planning, despite the lack of direct joint sensing. A forward kinematic model based on a modified Denavit–Hartenberg formulation and a hierarchical sphere-based hand representation are introduced to enable efficient fingertip pose computation and collision avoidance. The model is implemented in CoppeliaSim and validated through simulated manipulation tasks involving objects with diverse physical properties, demonstrating realistic, collision-free grasping behavior.
Multi-Grid Markov Decision Processes (mg-MDPs) decompose decision environments into hierarchical reward layers, addressing the complexity of multi-faceted planning problems. Construction and validation of mg-MDPs requires expertise in both domain knowledge and optimization theory, limiting practical adoption. This paper presents an interactive system for real-time mg-MDP construction and visualization, combining natural language interfaces with dynamic policy display. The framework supports iterative layer design, weight adjustment, and immediate observation of resulting policies through an integrated solver. Case studies in autonomous navigation, urban planning, and resource allocation demonstrate how the approach connects theoretical mg-MDP formulations to practical applications. The system incorporates AI-assisted layer generation, real-time convergence visualization, and interactive parameter adjustment. Experimental evaluation indicates that the interactive framework exposes layer interactions, weight sensitivity, and policy robustness characteristics that are difficult to extract through batch analytical methods, with design cycle reductions of approximately 10–20× compared to manual specification.
