This paper presents an innovative approach to semantic path planning for mobile robots by integrating semantic data from building digital twins. Semantic and metric information extracted from the digital twin is used to assign weights to a connectivity graph, allowing for path computation using the A* algorithm. Our method excels at generating robot-specific maps that combine both geometric and semantic data, diverging from traditional static maps. This semantic integration equips robots with diverse navigation skills, enabling them to navigate complex environments within large smart facilities. A key innovation of this work is our path planner, which utilizes semantic data from Building Information Modeling (BIM) databases. This marks a significant advancement in mobile robotic navigation, accommodating robots with varying navigation abilities. The significance of this work lies in the seamless integration of semantic data, enhancing the adaptability and efficiency of mobile robots, regardless of their navigation skills. This coordinated navigation system not only improves safety but also optimizes shared space management for both humans and robots.
We propose a deep learning based measurement model for Monte Carlo self-localization to improve the robustness to robot calibration errors in the RoboCup Humanoid League context. A deep neural network estimates the distance between a line mask computed by the vision pipeline from a camera image and given pose candidates. This distance is then used to derive weights for the particles of the Monte Carlo localization. The network is trained using images generated by applying a perspective transformation on the map of the field. We use a style transfer model to preprocess the input data for finetuning the model to build robustness towards noise and detection errors in real-world data. Our model is more robust to erroneous head pose measurements than the baseline. Further, we show significant improvements in global localization.
In 2024, B-Human won the Standard Platform League Champions Cup and the Shared Autonomy Challenge at the RoboCup world championship as well as the RoboCup German Open. This paper provides an overview of some of our new approaches that we believe have contributed most to improving our already very successful code base. This ranges from adaptions to recent rule changes over multiple new vision and motion approaches to the implementation of this year’s technical challenge.
We present a lightweight keypoint-based Recurrent Neural Network (RNN) approach to the 2023 RoboCup Standard Platform League Visual Referee Challenge. The goal of the challenge is to classify 13 different static and dynamic referee gestures during a robotic soccer match. The developed solution must be lightweight and perform well from anywhere on the field. Multiple referees may be present on the field, with the distinguishing trait of the visual challenge referee being their red gloves. We use the lightweight Convolutional Neural Network pose detector BlazePose to extract the pose key-points. To isolate the referee in the frame, HSV colour segmentation is used to find red glove regions and mask unneeded sections of the image. A lightweight RNN is used to classify the sequence of keypoints. We compare the results of our work to previous solutions developed for the 2022 Visual Referee Challenge. Analysis on real-time testing shows our method achieves strong performance for all gestures on many different locations on the soccer field.
In this paper, we present our approach that earned first place at the RoboCup@Work 2024 competition. We briefly outline the current competition challenges and give an update on our system architecture and methods used for planning, manipulation, perception and navigation. The emphasis is on newly implemented features and relevant changes compared to the design of prior years. Finally, we evaluate the results of the 2024 RoboCup@Work including the additional technical coworker challenge. A performance comparison with other teams is provided and further conclusions are drawn by revisiting the challenges and goals outlined last year.
Object tracking is one of the main challenges in soccer-playing robots. Due to its fast movement, detecting and tracking the soccer ball is challenging for goalkeepers in both humanoid and wheeled robots. To speed up object tracking, we propose the use of Event-based Cameras (ECs). ECs are bio-inspired sensors that asynchronously report changes in brightness for each pixel. Because of their high dynamic range, pixel bandwidth, temporal resolution, low power consumption, and computational simplicity, they are beneficial for vision-based projects in challenging lighting conditions and can detect fast movements with their microsecond response time. We created a dataset using two different event-based cameras from iniVation and Prophesee that recorded events in the lab and during Middle Size League matches at RoboCup 2023. Additionally, we created a Graphical User Interface (GUI) working with the Robot Operating System (ROS) to simplify camera and camera-projector setup calibration for RoboCup participants. The proposed ROS GUI is able to control the camera bias parameters online and publish a stream of events in addition to event “frames” on ROS topics. These advancements will help all RoboCup teams shift from frame-based to event-based technologies, enhancing ball detection regardless of color or lighting. The dataset is available publicly: github.com/MISTLab/event_based_data .
Through the course of RoboCup 2024 our team scored 45 goals over 6 seated matches and, ultimately, unseated the longstanding reigning champions in our division with a victory of 6 goals to 1. In this paper we will focus on hardware, vision, and high-level decision making while briefly touching upon others. Our humanoid robot named Artemis is around 1.4 m tall, weighing over 35 kg. Artemis is designed to be lightweight featuring quasi-direct drive actuators with motor housings which serve as drive inputs to the next motor at the hips to save weight. For the tournament the foot was redesigned to allow for increased kicking accuracy, contact rate, and durability when stepped on. For vision we introduce several practical filters and an additional component termed proximity to address potential failure cases when an opponent or obstacle is very near. Finally, we discuss the strategies and overall planning architecture used during the tournament for high level decision making.
A reliable stand-up movement is particularly important for humanoid robots. This becomes even more important in competitions such as the Standard Platform League (SPL), as a lying robot is an obstacle for other robots on the field and can neither effectively defend nor score a goal. Therefore, we have developed a system enabling the NAO robots to stand up reliably even with worn-out joints and on challenging floors. Here, we combine conditions influenced by the environment with an online-learned score for different parts of the stand-up movement. This enables us to stand up more reliably on a wide range of floor conditions.
The RoboCup, an international robotics competition, serves as a platform to improve robot capabilities. In the RoboCup Soccer leagues, continuous enhancements target critical aspects like robot perception, locomotion, shooting, and decision-making. This paper introduces a novel decision-making strategy based on a mental simulations architecture integrating prediction algorithms to anticipate diverse potential outcomes of ongoing actions. Implemented within the context of a goalkeeper scenario in the RoboCup Soccer 3D simulation league, this strategy allows the robot to recognize potential future threats, thus preventing the opposing team from scoring opportunities. A comparative analysis demonstrates the efficacy of employing complex algorithms over traditional decision-making approaches such as state machines. Notably, as the complexity of the robot’s operating environment increases.
Simulators in robotics research are widely used and are the cornerstones of many advances in the field. As time goes on, there are now more and more options available to roboticists than ever before. However, navigating through the choices in search of the right simulator is often non-trivial. There is a lack of quantitative studies to justify the usage of different simulators for different applications, especially regarding the quality of simulated computer vision solutions. A common vision solution with inaccuracies in simulators is object detection. We conduct thorough quantitative experiments and provide in-depth analysis of comparability of object detection performance in the commonly used simulator Gazebo and the newly introduced NVIDIA Isaac Sim to real life solutions using the Toyota Human Support Robot (HSR) as the platform. Our results indicate that Gazebo grossly overestimates in terms of prediction accuracy while NVIDIA Isaac Sim matches much more closely with reality. Additionally, we found that performance in Gazebo can be unstably variant at times. Furthermore, both simulators share highly correlated low-end to high-end performance distribution with reality. We can recommend NVIDIA Isaac Sim for robotic research and developments where computer vision is integral.
In this paper we will introduce an event-based agent simulation for the RoboCup Logistics League (RCLL). RCLL provides a testbed for the automation of flexible production to allow the development and evaluation of novel concepts and algorithms. The development is stimulated by a competition. Competing teams need to develop a team of autonomous robots able to fabricate on-demand product orders embedded into a flexible factory floor. Simulation plays a major role in developing and testing the needed planning and scheduling algorithms. As previously developed full-fledged simulations are too resource-demanding and hard to use we developed a light-weighted abstract simulation. In this paper we will discuss how the abstraction of the real setup to the simulation has been performed. Moreover, to allow a quick adoption of the simulation by teams within the league a standardized perception-action interface was developed. We will present the concepts and implementation of the simulation in detail. In order to evaluate the usability and potential for adoption a user study was performed with members of the RCLL.
The RoboCup Middle Size League (MSL) is designed to advance robotics research through autonomous soccer played by robots. In this league, two teams of five robots compete on a field measuring 22 by 14 m. Research in the MSL focuses on distributed multi-agent systems with robots that rely solely on on-board sensors. At RoboCup 2024, held in Eindhoven, The Netherlands, the Tech United Eindhoven team from Eindhoven University of Technology claimed victory in the MSL competition, which felt particularly special as the tournament took place in their hometown. This article reflects on the tournament and the key developments that contributed to Tech United’s success. We highlight three main areas of progress. Firstly, we showcase a new trajectory planner based on a setpoint generator, which leads to more dynamic and more robust path planning. Secondly, we elaborate on the developments of Tech United towards implementing artificial intelligence for ball and obstacle detection. Finally, we present the progress made in using quadruped robots for the MSL.
This work explores the improvements and methods that led to SinfonIA Uniandes’ success in winning the RoboCup@Home Social Standard Platform League (SSPL) 2024, using the Pepper robot from SoftBank Robotics. Considering the challenges due to human-robot interaction (HRI) and the hardware constraints, our focus was on improving Pepper’s ability to offer practical assistance in home tasks. Key improvements include a robust person-following system that combined YOLO-based visual detection with a PID controller, achieving 80
Improving the latency of neural networks on low-end hardware such as the NAO robot used in the RoboCup Standard Platform league has been a relevant research topic since deep learning models can be effectively used for real-time computer vision. This paper focuses on optimizing the implementation of quantized neural networks (QNNs) for ball detection on the NAO V6 robot. The goal is to improve inference speed and reduce memory usage. A direct quantization scheme is employed, where weights and activations are represented as 8-bit integers. SIMD instructions available on the NAO’s processor architecture are utilized for efficient calculations. Quantization-aware training is used to train the models, and their inference is realized via automatically generated assembly code. The quantized models achieve a speedup of approximately 2× compared to float models. However, there is a slight decrease in model performance, as indicated by lower AUROC scores and mean ball IoU values for the quantized models. Paired t-tests confirm the statistical significance of the differences between float and quantized models. The optimized implementation of QNNs for ball detection on the NAO robot provides a practical speedup while maintaining reasonable model performance, however choosing simpler architectures leads to a better trade-off between latency and performance than quantization. Future work could involve comparing the results with the XNOR-Net for ball detection on the NAO and exploring the proposed approach in other RoboCup applications. All code used in the experiments is publicly available at https://github.com/fthielke/NaoQNN .
The paper addresses the challenges and implications of (lacking) synchronization between agents in real-time multi-agent simulation systems. Based on two specific manifestations of mis-synchronization in 2D Soccer Simulation, termed holes and clashes, the paper makes two contributions: First, it provides a thorough historical review of the problem’s existence and urgency of occurrence, covering the time from the very early beginnings of the RoboCup initiative in the late 1990s till today. Second, it presents the results of an exhaustive empirical study that reveals quantitatively what impact synchronization problems may have on the playing performance of a simulated soccer team.
The RoboCup Humanoid League has set itself the ambitious objective of developing humanoid robots capable of winning a soccer match against the most recent FIFA World Cup champions by the middle of the 21st century. This paper provides an overview of the development and performance of CIT Brains, the winning team of the RoboCup 2024 Humanoid League KidSize class, within the context of the league’s evolving trends and challenges. The SUSTAINA-OP2 robot represents a significant advancement from its predecessor, the SUSTAINA-OP robot, which was first introduced at RoboCup 2022. In response to the ongoing developments of the league, the SUSTAINA-OP2 incorporates newly released actuators and a new computer board. The robot was constructed using components that can be readily procured through online shops in conjunction with custom parts sourced from external manufacturing services. All design data is made available as open hardware on . These developments are based on the league’s progress and will facilitate future competition requirements.
This paper presents software that aims to generate extensive soccer gameplay data through computer simulations, addressing the scarcity of human-generated data for analysis. It discusses the challenges of analyzing human gameplay data, the need for computer simulations, and the development of software tools such as rcgamestats and rcg2data. The system uses the RoboCup Soccer Simulator to efficiently generate game data. In addition, it presents a case study on distribution analysis of ball interceptions to demonstrate the potential for various analytical purposes. The provided data includes 78,000 matches, which is equivalent to approximately 8,667 human soccer matches. Future efforts include expanding data collection and improving database system integration.
This study presents an approach to formulating the optimal control policy for self-driving cars. Our method integrates sensor fusion with deep reinforcement learning, specifically utilizing the Soft Actor-Critic (SAC) algorithm. By leveraging identity mapping and residual structures, we improve agent training through a two-branch fusion technique for vehicle image and tracking sensor data. We introduce a Non-linear Auto-Regressive model with Exogenous inputs (NARX) within the sensor fusion architecture. This model incorporates actions from previous time steps to capture temporal dependencies in the sensor data, providing a deeper understanding of the environment for the Reinforcement Learning (RL) agent and enhancing decision-making. Our research demonstrates this information fusion approach’s effectiveness and highlights our method’s advantages through comprehensive comparisons. It illustrates the practical application of reinforcement learning in improving intelligent vehicle decision-making and contributes to the evolving field of autonomous driving. This advancement can positively affect how self-driving cars operate, potentially fostering more trust in the progress of autonomous driving technology.
Since the use of multi-agent systems has significantly increased over the past few decades, it is crucial to study the methods used in this area and take implementation-related practical issues into consideration. For instance, in the formulation of the consensus problem, the communication between the agents is depicted using a graph, where each edge represents a communication link between the agents. In practical terms, there will be significant operational challenges and costs associated with implementing each of these communication channels in various forms. Therefore, in this study, the graph reduction problem has been presented as an optimization problem, accompanied by six distinct cost functions. The objective is to reduce the size of a graph without compromising its fundamental characteristics. Hence, the cost functions provided are formulated to minimize the number of edges in the resulting graph while ensuring that the consensus error remains bounded. Ultimately, simulations were performed to demonstrate the effectiveness of the proposed approach. The outcomes derived from the specified set of cost functions were then compared to several conventional topologies in multi-agent systems.
