Humans are crucial for industrial layout planning and technological design, as seen in assembly lines and order-picking layouts. These fields aim to incorporate robots for human convenience and collaboration, striving towards Industry 5.0. As a result, simulation, virtual and digital twin environments are used to facilitate employee training, ergonomic evaluations, and task modelling. Human motion generation for robotics and industrial simulation environments can accelerate process analysis by simplifying the generation of human motions. Given the data-intensive nature of generative models, human motion datasets containing industrial scenarios, terminologies and appropriate textual annotation are necessary. Thus, the key contribution of this work is a novel dataset collection framework for futuristic, industrial, large-scale datasets that can be adapted to the dataset requirements of human motion generation models. In addition, we aim to bring together the human motion generation, dataset and industrial simulation community by providing a platform for discussion and exchange.
Mobile robots are reaching unprecedented speeds, with platforms like Unitree B2, and Fraunhofer O3dyn achieving maximum speeds between 5 and 10 m/s. However, effectively utilizing such speeds remains a challenge due to the limitations of RGB cameras, which suffer from motion blur and fail to provide real-time responsiveness. Event cameras, with their asynchronous operation, and low-latency sensing, offer a promising alternative for high-speed robotic perception. In this work, we introduce MTevent, a dataset designed for 6D pose estimation and moving object detection in highly dynamic environments with large detection distances. Our setup consists of a stereo-event camera and an RGB camera, capturing 75 scenes, each on average 16 seconds, and featuring 16 unique objects under challenging conditions such as extreme viewing angles, varying lighting, and occlusions. MTevent is the first dataset to combine high-speed motion, long-range perception, and real-world object interactions, making it a valuable resource for advancing event-based vision in robotics. To establish a baseline, we evaluate the task of 6D pose estimation using NVIDIA's FoundationPose on RGB images, achieving an Average Recall of 0.22 with ground-truth masks, highlighting the limitations of RGB-based approaches in such dynamic settings. With MTevent, we provide a novel resource to improve perception models and foster further research in high-speed robotic vision. The dataset is available for download https://huggingface.co/datasets/anas-gouda/MTevent
Smart warehouses face rapid layout reconfigurations and frequent process adaptations, especially when using a Cyber-Physical Production Systems (CPPS) setup. These dynamic conditions introduce environmental uncertainty, making real-time navigation and obstacle avoidance a significant challenge for robot fleets. Limited sensing range and frequent occlusions further hinder local robot perception. Classic methods rely on centralized planning or vision-based systems, which struggle in low-visibility and cluttered environments. To address these gaps, we propose a graph-based collaborative perception network for the RoboFUSE (Framework for Unified Sensing and Exploration) system. Each robot operates onboard RoboFUSE with a dual-purpose waveform for sensing and communication (S&C) that emulates 6G Integrated Sensing and Communication (ISAC). This setup supports real-time data sharing across the fleet. On top of this platform, we develop RoboFUSE-Graph Neural Network (GNN), an uncertainty-aware GNN that fuses multi-robot radar data into a global spatial-semantic map. The model captures spatial relations and temporal dependencies using sliding window graphs. Experiments reveal an F1 score of 0.91 with a five-sliding window. The proposed approach enhances situational awareness, enables safe, scalable navigation, and represents a stride towards 6G-enabled smart warehouses.
This paper explores the process of trust development in human-drone interaction, focusing on how trust evolves over time when humans are unfamiliar with AI-assisted technology. The study investigates the development of trust, specifically whether it changes due to personal along a series of experiments. A study (N = 19) was conducted in a warehouse setting using aerial drones. Participants interacted with a drone that guided them through a that was based on real-world warehouse processes. Data was collected through questionnaires and distance measurements. Changes in comfortable human-drone distance were assessed before and after the interaction, changes in the mental model were measured before, right after, and two weeks later via an online questionnaire, and changes in trust were recorded during the interaction. Variables measured were 1) Subjective trust level: Participants rated their trust in the drone, 2) Changes in individual mental model: Participants answered a questionnaire to assess their understanding of the drone’s capabilities, and 3) Human-drone distance: The comfortable distance between the participant and the drone was measured. while the participants' mental models of the drone improved significantly after the interaction, their subjective trust levels remained relatively stable. The research suggests that initial trust levels are influenced by pre-existing psychological anchors and are difficult to change solely through new experiences delete word. It is concluded that, while human-drone experiences can enhance understanding of AI systems, it is more difficult to change pre-existing levels of trust. The findings suggest that a more comprehensive approach, that considers cognitive, affective, and behavioral measures, is important to examine human-drone interaction.
The rapidly evolving new concepts of the 6G technology raise the challenge of assessing how well 6G-enabled system-of-systems architectures perform in real-life situations beyond theories and simulations. To facilitate this, a new Visual Metaphor layer is presented in this paper to add the visualization feature for a system-of-systems Digital Twin (DT). Visual metaphor plays a crucial role in assisting network engineers to comprehend complex 6G concepts through intuitive visual representations. Powered by an immersive Augmented Reality (AR) based on laser projection system, the metaphor framework seamlessly projects the simulated environments into reality and vice versa. Thus, it bridges the gap between theoretical development and practical implementation. Furthermore, it features a plug-and-play solution to accommodate the validation of integrated systems and applications easily. To validate the performance and show the flexibility of the visual metaphor layer, we perform three case studies from the application contexts of teleoperation, platooning, and intralogistics. The results show that the visual metaphor can visualize internal network states, such as communication links, as well as virtual objects and other environmental features. The metaphor can accomplish these diverse and modular tasks while preserving an overall system response time of 55 ms in large-scale scenarios with up to 40 robots.
This contribution analyzes the self-perception and political biases of OpenAI’s Large Language Model ChatGPT. Considering the first small-scale reports and studies that have emerged, claiming that ChatGPT is politically biased towards progressive and libertarian points of view, this contribution is aimed at providing further clarity on this subject. Although the concept of political bias and affiliation is hard to define, lacking an agreed-upon measure for its quantification, this contribution attempts to examine this issue by having ChatGPT respond to questions on commonly used measures of political bias. In addition, further measures for personality traits that have previously been linked to political affiliations were examined. More specifically, ChatGPT was asked to answer the questions posed by the political compass test as well as similar questionnaires that are specific to the respective politics of the G7 member states. These eight tests were repeated ten times each and indicate that ChatGPT seems to hold a bias towards progressive views. The political compass test revealed a bias towards progressive and libertarian views, supporting the claims of prior research. The political questionnaires for the G7 member states indicated a bias towards progressive views but no significant bias between authoritarian and libertarian views, contradicting the findings of prior reports. In addition, ChatGPT’s Big Five personality traits were tested using the OCEAN test, and its personality type was queried using the Myers-Briggs Type Indicator (MBTI) test. Finally, the maliciousness of ChatGPT was evaluated using the Dark Factor test. These three tests were also repeated ten times each, revealing that ChatGPT perceives itself as highly open and agreeable, has the Myers-Briggs personality type ENFJ, and is among the test-takers with the least pronounced dark traits.
Rapidly changing conditions, such as continuous layout shifts and process adaptions in smart warehouses pose challenges for robot mapping and navigation. Single-robot perception is inadequate in spaces full of moving obstacles. Advanced communication and perception capabilities are required to accomplish multi-robot perception and collective environmental awareness. 6G emerges as a promising solution with UltraReliable Low-Latency Communication (uRLLC) and Integrated Sensing and Communication (ISAC), as 5G cannot fully support time-critical tasks. While 6G-driven Vehicle-to-Everything (V2X) cooperative communication made significant strides in the automotive sector, its potential in intralogistics is underexplored. Leveraging V2X concepts of Cooperative Intelligent Transport Systems (C-ITS) standards, addressing gaps in 6G ISAC, and robotics standardization are vital steps toward a 6G-driven multirobot framework for warehouses. This paper presents the 6GRoboFUSE (Framework for Unified Sensing and Exploration), designed for collaborative robotic perception using 6G ISAC. It paves the way for standardized 6G frameworks, enabling safe, efficient, and cooperative operations in intralogistics. For empirical validation, we test the ISAC management and controller (ISMAC) protocol in a robot platform. The results reveal that the robot’s actions impact resource allocation, affecting the sensing and communication time slot allocations of ISMAC.
Many planning and decision activities in logistics and supply chain management are based on forecasts of multiple time dependent factors. Therefore, the quality of planning depends on the quality of the forecasts. We compare different state-of-the-art forecasting methods in terms of forecasting performance. Differently from most existing research in logistics, we do not perform this in a case-dependent way but consider a broad set of simulated time series to give more general recommendations. We therefore simulate various linear and nonlinear time series that reflect different situations. Our simulation results showed that the machine learning methods, especially Random Forests, performed particularly well in complex scenarios, with the differentiated time series training significantly improving the robustness of the model. In addition, the time series approaches proved to be competitive in low noise scenarios.
This work introduces a novel approach to the currently available automatic volumetric measurement systems used in the industry. The proposed system uses a ceiling-mounted laser in tandem with an RGB camera to generate a point cloud of the object that is to be measured, utilizing an adaptation of a structured light scanning approach. The resulting point clouds are post-processed, removing detected structural anomalies to improve the yielded results. The resulting system is then tested and validated on a set of seven objects of various sizes commonly encountered in industrial environments. For these experiments, our approach yields outlier-corrected volumes which are accurate at up to 1% volume divergence. The results differ vastly when going beyond the system’s optimal object size range. The limits of the system are tested for smaller and larger objects, showing a notably higher inaccuracy when tested on relatively small or large objects. Further experiments, exploring optimal laser line distance, lighting conditions, and (in the case of containers) filling degrees are also conducted.
In this contribution, we introduce a novel ensemble method for the re-identification of industrial entities, using images of chipwood pallets and galvanized metal plates as dataset examples. Our algorithms replace commonly used, complex siamese neural networks with an ensemble of simplified, rudimentary models, providing wider applicability, especially in hardware-restricted scenarios. Each ensemble sub-model uses different types of extracted features of the given data as its input, allowing for the creation of effective ensembles in a fraction of the training duration needed for more complex state-of-the-art models. We reach state-of-the-art performance at our task, with a Rank-1 accuracy of over 77% and a Rank-10 accuracy of over 99%, and introduce five distinct feature extraction approaches, and study their combination using different ensemble methods.
Background Serious games and game-based learning are widely used in education. Gaming in logistics education relies on serious gaming with no or less consideration on enjoyment.Intervention This study examines in a first step whether and if so, how (serious) games are used as part of logistics majors' curricula at German universities. Based on the findings, an entertainment game about warehousing is developed and validated.Methods Warehousing is an application-based discipline in which the operation of different picking techniques has an impact on the order picking. To help students understand the impact of such techniques, the 2D game WareMover was developed. The open-access game combines education and entertainment with different gamification objects and can be played in singleplayer or multiplayer mode. In this competitive 2D game, the objective is to pick orders by navigating around the warehouse towards the right shelf and to click on the correct storage compartment. The player has to achieve the highest score, which is determined by the fastest and least erroneous delivery. Four different picking techniques can be used in two game modes.Results The results of a 14-player game session are presented. All players played both game variants with all picking techniques and completed a questionnaire. The players rated the game as user-friendly, entertaining, didactically useful, suitable for teaching and education, and engaging. In addition, the majority of players would play the game again and recommend it to others.Conclusion The open-source game WareMover can contribute to studies in the field of warehousing by teaching students about picking techniques and the differences between these techniques. The novel game approach of interpreting a serious game as an entertainment game can increase the enjoyment and motivation of the players.
This contribution presents the TOMIE framework (Tracking Of Multiple Industrial Entities), a framework for the continuous tracking of industrial entities (e.g., pallets, crates, barrels) over a network of, in this example, six RGB cameras. This framework, makes use of multiple sensors, data pipelines and data annotation procedures, and is described in detail in this contribution. With the vision of a fully automated tracking system for industrial entities in mind, it enables researchers to efficiently capture high quality data in an industrial setting. Using this framework, an image dataset, the TOMIE dataset, is created, which at the same time is used to gauge the framework's validity. This dataset contains annotation files for 112,860 frames and 640,936 entity instances that are captured from a set of six cameras that perceive a large indoor space. This dataset out-scales comparable datasets by a factor of four and is made up of scenarios, drawn from industrial applications from the sector of warehousing. Three tracking algorithms, namely ByteTrack, Bot-Sort and SiamMOT are applied to this dataset, serving as a proof-of-concept and providing tracking results that are comparable to the state of the art.
Safe and efficient real-time robotics control is highly delay-sensitive. Enabling such critical applications via mobile communication networks, therefore, hinges on reliably provisioning radio resources at low latency. Here, employing the Open Radio Access Network (O-RAN) concept, networks can be built with adaptive intelligent features, such as Artificial Intelligence (AI)-based scheduling policies for optimized resource management. By harnessing the innovative concept of distributed Applications (dApps) deployed inside the Open RAN Distributed Unit (O-DU), predictive resource allocation can reliably provide low latencies for robot control at increased spectral efficiency. This work demonstrates the Key Performance Indicators (KPIs) achieved with a proposed real-time proactive scheduling dApp employing AI methods. Results are derived from a real-world testbed that integrates predictive communication with a digital twin of the two-wheeled inverted pendulum robot evoBOT, designed for intralogistics. The closed-loop locomotion control, also providing upright stability control, is performed on the mobile edge via an evolved O-RAN system hosting the proposed dApp. Relative to optimized reactive network slicing, our approach yields a 34% mean reduction for uplink delays. Moreover, radio resource usage is reduced by up to 47% compared to highly optimized reactive scheduling, exhibiting similar control performance.
Enhancing transparency in production processes, especially in shared manufacturing, relies heavily on sharing data. Information asymmetries and coordination problems between parties with conflicting interests pose a challenge in this multi-stakeholder interaction. Blockchain technology with smart contracting can be a solution due to its immutable data and decentralised data storage features. Designing and executing blockchain in industrial applications is a highly intricate task that requires extensive testing, expertise, and proficiency. This paper is the first to propose a holistic simulation model for evaluating the impact of smart contracting on shared manufacturing, including a novel approach to simulated smart contracting in time-lapse for Ethereum-based networks. The introduced model guides the design and implementation process of blockchain applications in shared manufacturing to address this challenge. A systematic literature review establishes ten design process requirements and ten smart contract functions. The implementation is developed based on the design benchmarks of three Ethereum-based frameworks to investigate the simulation model's respective feasibility and scalability. The simulation model validation demonstrates our approach's suitability for simulating smart contracting in shared manufacturing within a hybrid production. It enables fast and scalable simulations, offering an innovative approach to extensively testing blockchain applications before their introduction to ongoing industrial operations.
The human eye consists of two types of photoreceptors, rods and cones. Rods are responsible for monochrome vision, and cones for color vision. The number of rods is much higher than the cones, which means that most human vision processing is done in monochrome. An event camera reports the change in pixel intensity and is analogous to rods. Event and color cameras in computer vision are like rods and cones in human vision. Humans can notice objects moving in the peripheral vision (far right and left), but we cannot classify them (think of someone passing by on your far left or far right, this can trigger your attention without knowing who they are). Thus, rods act as a region proposal network (RPN) in human vision. Therefore, an event camera can act as a region proposal network in deep learning Two-stage object detectors in deep learning, such as Mask R-CNN, consist of a backbone for feature extraction and a RPN. Currently, RPN uses the brute force method by trying out all the possible bounding boxes to detect an object. This requires much computation time to generate region proposals making two-stage detectors inconvenient for fast applications. This work replaces the RPN in Mask-RCNN of detectron2 with an event camera for generating proposals for moving objects. Thus, saving time and being computationally less expensive. The proposed approach is faster than the two-stage detectors with comparable accuracy
Future 6G communication systems need to be carefully evaluated under near real-life boundary conditions to prove their performance beyond theoretical considerations and simulations. In this work, we propose a new, lean approach to integrate and evaluate future 6G network architectures and AI-enabled approaches in scaled physical environments that emulate the full-scale communication, mobility, and environmental conditions powered by a Digital Network Twin (DNT). Our approach is enabled by the realistic modeling of the radio environmental impacts within the DNT, such as path loss, shadowing, and interference. One key contribution lies in the specific communication emulation functionality embedded in the real-time capable DNT, which allows imposing specific communication technology properties of the real-life scenario on the scaled physical environment. In this paper, we introduce a prototyping architecture and present a comprehensive case study to demonstrate the effectiveness of the approach: the AI-enabled mesh routing protocol PARRoT is evaluated in three scaled scenarios (teleoperation, platooning, and intralogistic transport). The results show that future 6G networks can be evaluated in realistic, yet safe and cost-efficient environments before moving onto the real world.
Wireless Sensor Network (WSN) applications reshape the trend of warehouse monitoring systems allowing them to track and locate massive numbers of logistic entities in real-time. To support the tasks, classic Radio Frequency (RF)-based localization approaches (e.g. triangulation and trilateration) confront challenges due to multi-path fading and signal loss in noisy warehouse environment. In this paper, we investigate machine learning methods using a new grid-based WSN platform called Sensor Floor that can overcome the issues. Sensor Floor consists of 345 nodes installed across the floor of our logistic research hall with dual-band RF and Inertial Measurement Unit (IMU) sensors. Our goal is to localize all logistic entities, for this study we use a mobile robot. We record distributed sensing measurements of Received Signal Strength Indicator (RSSI) and IMU values as the dataset and position tracking from Vicon system as the ground truth. The asynchronous collected data is pre-processed and trained using Random Forest and Convolutional Neural Network (CNN). The CNN model with regularization outperforms the Random Forest in terms of localization accuracy with ≈ 15 cm. Moreover, the CNN architecture can be configured flexibly depending on the scenario in the warehouse. The hardware, software and the CNN architecture of the Sensor Floor are open-source under https://github.com/FLW-TUDO/sensorfloor.
Automated guided vehicles (AGVs) are an essential area of research for the industry to enable dynamic transport operations. Furthermore, AGV-based multi-robot systems (MRS) are being utilized in various applications, e.g. in production or in logistics. Most research today focuses on ensuring that the system is operational, which is not always achieved. In daily use, faults and failures in an AGV-based MRS are most likely inevitable. So industrial systems must support some safety methods, e.g. an emergency stop function. Although emergency stop functions are designed to prevent larger issues, their usage leads to a failure of the control system, since the affected systems are typically shut down immediately. Depending on the AGV type, an uncontrolled behaviour can occur. In case of control failure, this behaviour can lead to collisions and associated high costs. In this paper, we present and compare three approaches for avoiding collisions in an intralogistics scenario in the case of control failure. In the said scenario, the trajectory planing is being adapted to minimize or avoid collisions. The first approach calculates the next collisionfree time slot when an emergency stop occurs and continues the planned trajectories until this time. The second approach calculates several alternative trajectories with the existing trajectory planning without considering emergency stop collisions. The third approach aims to completely avoid emergency stop collisions by extending trajectory planning to include the detection of possible emergency stop collisions. We evaluate and compare the approaches by employing multiple metrics to assess their performance, including runtime, number of collisions that occur, and the system's throughput. We thoroughly discuss and analyse the results, offering insights into the strengths and weaknesses of the approaches.