In this paper we study how transforming regular reinforcement learning environments into goal-conditioned environments can let agents learn to solve tasks autonomously and reward-free. We show that an agent can learn to solve tasks by selecting its own goals in an environment-agnostic way, at training times comparable to externally guided reinforcement learning. Our method is independent of the underlying off-policy learning algorithm. Since our method is environment-agnostic, the agent does not value any goals higher than others, leading to instability in performance for individual goals. However, in our experiments, we show that the average goal success rate improves and stabilizes. An agent trained with this method can be instructed to seek any observations made in the environment, enabling generic training of agents prior to specific use cases.

The Storage Location Assignment Problem (SLAP) is of central importance in warehouse operations. An important research challenge lies in generalizing the SLAP such that it is not tied to certain order-picking methodologies, constraints, or warehouse layouts. We propose the OBP-based SLAP, where the quality of a location assignment is obtained by optimizing an Order Batching Problem (OBP). For the optimization of the OBP-based SLAP, we propose a nested Metropolis algorithm. The algorithm includes an OBP-optimizer to obtain the cost of an assignment, as well as a filter which approximates OBP costs using a model based on the Quadratic Assignment Problem (QAP). In experiments, we tune two key parameters in the QAP model, and test whether its predictive quality warrants its use within the SLAP optimizer. Results show that the QAP model’s per-sample accuracy is only marginally better than a random baseline, but that it delivers predictions much faster than the OBP optimizer, implying that it can be used as an effective filter. We then run the SLAP optimizer with and without using the QAP model on industrial data. We observe a cost improvement of around 23

Order Picking in warehouses is often optimized using a method known as Order Batching, which means that one vehicle can be assigned to pick a batch of several orders at a time. There exists a rich body of research on Order Batching Problem (OBP) optimization, but one area which demands more attention is computational efficiency, especially for optimization scenarios where warehouses have unconventional layouts and vehicle capacity configurations. Due to the NP-hard nature of the OBP, computational cost for optimally solving large instances is often prohibitive. In this paper, we compare the performance of two approximate optimizers designed for maximum computational efficiency. The first optimizer, Single Batch Iterated (SBI), is based on a Seed Algorithm, and the second, Metropolis Batch Sampling (MBS), is based on a Metropolis algorithm. Trade-offs in memory and CPU-usage and generalizability of both algorithms is analyzed and discussed. Existing benchmark datasets are used to evaluate the optimizers on various scenarios. On smaller instances, we find that both optimizers come within a few percentage points of optimality at minimal CPU-time. For larger instances, we find that solution improvement continues throughout the allotted time but at a rate which is difficult to justify in many operational scenarios. SBI generally outperforms MBS and this is mainly attributed to the large search space and the latter’s failure to efficiently cover it. The relevance of the results within Industry 4.0 era warehouse operations is discussed.

Order Picking in warehouses is often optimized through a method known as Order Batching, wherein several orders can be assigned to be picked by the same vehicle. Although there exists a rich body of research on Order Batching optimization, one area which demands more attention is that of computational efficiency, especially for warehouses with unconventional layouts and vehicle capacity configurations. Due to the NPhard nature of Order Batching, computational cost for optimally solving large instances is often prohibitive. In this paper we focus on approximate optimization and study the rate of improvement over a baseline solution until a timeout, using the Single Batch Iterated (SBI) algorithm. Modifications to the algorithm, trading computational efficiency against increased memory usage, are tested and discussed. Existing and newly generated benchmark datasets are used to evaluate the algorithm on various scenarios. On smaller instances we corroborate previous findings that results within a few percentage points of optimality are obtainable at minimal CPU-time. For larger instances we find that solution improvement continues throughout the allotted time but at a rate which is difficult to justify in many operational scenarios. The relevance of the results within Industry 4.0 era warehouse operations is discussed.

Order-batching is an important methodology in warehouse material handling. This paper addresses three identified shortcomings in the current literature on order-batching optimization. The first concerns the overly large dependence on conventional warehouse layouts. The second is a lack of proposed optimization methods capable of producing approximate solutions in minimal computational time. The third is a scarcity of benchmark datasets, which are necessary for data-driven performance evaluation. This paper introduces an optimization algorithm, SBI, capable of generating reasonably strong solutions to order-batching problems for any warehouse layout at great speed. On an existing benchmark dataset for a conventional layout, Foodmart, results show that the algorithm on average used 6.9% computational time and 105.8% travel cost relative to the state of the art. New benchmark instances and proposed solutions for various layouts and problem settings were shared on a public repository.

To date, research on warehouse order-batching has been limited by reliance on rigid assumptions regarding rack layouts. Although efficient optimization algorithms have been provided for conventional warehouse layouts with Manhattan style blocks of racks, they are limited in that they fail to generalize to unconventional layouts. This paper builds on a generalized procedure for digitization of warehouses where racks and other obstacles are defined using two-dimensional polygons. We extend on this digitization procedure to introduce a layout-agnostic minisum formulation for the Order Batching Problem (OBP), together with a sub-problem for the OBP for a single vehicle, the single batch OBP. An algorithm which optimizes the single batch OBP iteratively until an approximate solution to the OBP can be obtained, is discussed. The formulations will serve as the fundament for further work on layout-agnostic OBP optimization and generation of benchmark datasets. Experimental results for the digitization process involving various settings are presented.

Small autonomous surface vehicles (ASV) will need both teleoperation support and redundant positioning technology to comply with expected future regulations. When at sea, they are limited by a satellite communication link with low throughput. We have designed and implemented a graphical user interface (GUI) for teleoperation using a communication link with low throughput, and one positioning system, independent of the Global Positioning System (GPS), supported by the teleoperation tool. We conducted a user study (N=16), using real-world data from a field trial, to validate our approach, and to compare two variants of the graphical user interface (GUI). The users experienced that the tool gives a good overview, and despite the connection with the low throughput, they managed through the GUI to significantly improve the positioning accuracy.

To make autonomous, affordable ships feasible in the real world, they must be capable of safely navigating without fully relying on GPS, high-resolution 3D maps, or high-performance navigation sensors. We suggest a method for estimating the position using affordable navigation sensors (compass and speed log or inertial navigation sensor), sensors used for perception of the environment (cameras, echo sounder, magnetometer), and publicly available maps (sea charts and magnetic intensity anomalies maps). A real-world field trial has shown that the proposed fusion mechanism provides accurate and robust navigation, applicable for affordable autonomous ships.

The Ontology for Robotics and Automation (ORA) group was formed to develop a standard ontology specifying the most general concepts and relations in robotics and automation (R&A). The result of its work has been published as the IEEE standard 1872-2015. Its main component is CORA, a core ontology that extends the top-ontology Suggested Upper Merged Ontology (SUMO) with concepts such as robot, robot group, robotic systems and robot interaction. CORA serves as basis for creating more specific domains ontologies for subfields of robotics. It can be used as a basis for the construction of a robot/human-robot communication system, where the robot can represent itself, other agents and the environment. Different groups have been working in specializing the main standard with additional knowledge structures referring to other areas of R&A. In this chapter, we navigate among the concepts and processes used during the development of CORA, showing how it is being used and how it is currently being developed for human-robot interaction.

We compared three different Graphical User Interfaces (GUI) that we have designed and implemented to enable human supervision of an unmanned ship. Our findings indicate that a 3D GUI presented either on a screen or in a Virtual Reality (VR) setting provides several objective and subjective benefits compared to a Baseline GUI representing traditional tools.

e summarize our successful efforts to support in-tuitive programming of industrial robotic assemblytasks with a knowledge based approach to the rep-resentation of skills. These skills can be specified,re-used, refined and transferred between robotswith the help of a multimodal interface combinedwith kinesthetic teaching. We argue that while itis certainly possible and suitable to have robotsacquire skills or skill primitives through variouslearning methods, it is still crucial to provide ex-plicit knowledge and semantics available to them.

Many ships today rely on Global Navigation Satellite Systems (GNSS), for their navigation, where GPS (Global Positioning System) is the most well-known. Unfortunately, the GNSS systems make the ships dependent on external systems, which can be malfunctioning, be jammed or be spoofed. There is today some proposed techniques where, e.g., bottom depth measurements are compared with known maps using Bayesian calculations, which results in a position estimation. Both maps and navigational sensor equipment are used in these techniques, most often relying on high-resolution maps, with the accuracy of the navigational sensors being less important. Instead of relying on high-resolution maps and low accuracy navigation sensors, this paper presents an implementation of the opposite, namely using low-resolution maps, but compensating this by using high-accuracy navigational sensors and fusing data from both bottom depth measurements and magnetic field measurements. A Particle Filter uses the data to estimate a position, and as a second step, a Kalman Filter enhances the accuracy even further. The algorithm has been tuned and evaluated using both a medium and a high-accuracy Inertial System. Comparisons of the various tuning methods are presented along with their performance results. The results from the simulated tests, described in this paper, show that for the high-end Inertial System, the mean position error is 10.2m, and the maximum position error is 33.0m during a 20 h test, which in most cases would be accurate enough to use for navigation.

An unmanned ship can be designed without considering humancomfort, and can thus be constructed lighter, smaller and less expensive.It can carry out missions in rough terrain or be in areaswhere it would be dangerous for a human to operate. By not havingto support a crew, lengthy missions can be accepted, enabling, e.g.reconnaissance missions, or reducing emissions by lowering thespeed.Breakthroughs with autonomous systems enable more advancedunmanned surface vessels (USVs), but to be able to handle complexmissions in a dynamic environment, a human operator is stillassumed an effective decision maker. Thus, we propose a methodfor remote operation of a USV, where the operator uses VirtualReality (VR) to comprehend the surrounding environment. Greatimportance has been given to the ability to perform safe navigation,by designing a Graphical User Interface (GUI) that guides the operatorthrough the navigation process, by presenting the importantinformation at the right place in the right orientation. (Less)

We assume that an intuitive means for the specification, re-use, modification and transfer of synchronized motions-both regarding the two arms of a dual-arm robotic system, as well as regarding the coordination of a user and a robot-is key in interactive and collaborative settings as they are currently targeted for industrial applications. We show, how our knowledge based approach to end-user programming of synchronized motions and other generalizable, robot-agnostic skills can support such specification of coordinated actions between two robot arms and explain how that could be extended to include coordination with a human user. We describe the underlying ontologies and possibilities to populate those with an interface for intuitive programming, and show the generality of our approach through a task transfer between different kinematics (different robots), where the user is supported through underlying reasoning about the fulfillment of certain parameters or constraints for the involved skills.

Programmable Logic Controllers (PLCs) play an important role for integration of hardware and software in industrial robot cells. In this paper we propose a semantic grounding of the Sequential Function Charts (SFC) notation for specification of PLC programs. Our semantic grounding is expressed in the OWL semantic language and forms part of our semantic robot framework, called KIF (Knowledge Integration Framework). KIF is a set of ontologies and associated tools to ensure interoperability between heterogeneous equipment making up a robot cell. We also present a tool set for manipulating SFC instances stored in RDF triple stores reachable through the RDF4J framework. The semantic grounding and tool set are evaluated in a teaching-by-demonstration experiment in a small parts assembly setup featuring a collaborative industrial robot, ABB YuMi, where the tool set is used to create and execute SFC descriptions on-the-fly based on data from human demonstrations.

Industrial robot systems being deployed today do not contain domain knowledge to aid robot operators in setup and operational use. To gather such knowledge in a robot context requires mechanisms for entering and capturing semantic data, that will gradually build a working vocabulary while interacting with environment and operators, for bootstrapping system knowledge and ensuring data collection over time. This paper presents a prototype user interface, assisting kinesthetic teaching of a collaborative industrial robot, that allows for capturing semantic information while working with the robot in day-to-day use. A graphical user interface with natural language processing builds a working vocabulary of the environment while modifying and/or creating robot programs. A simple demonstration illustrates the approach.

Conventional industrial robots are heavily dependent on hard automation that requires pre-specified fixtures and time-consuming (re)programming performed by experienced operators. In this work, teaching by human-only demonstration is used for reducing required time and expertise to setup a robotized assembly station. This is achieved by the proposed framework enhancing the robotic system with advanced perception and cognitive abilities, accessed through a user-friendly Human Robot Interaction interface. The approach is evaluated on a small parts’ assembly use case deployed onto a collaborative industrial robot testbed. Experiments indicate that the proposed approach allows inexperienced users to efficiently teach robots new assembly tasks.

We present our successful efforts to improve intuitive programming of synchronized dual-arm operations in industrialrobotic assembly tasks. To this end we extend an earlier proposed skill representation (based on our work with ontologies for industrial robots) and our respective programminginterface to integrate options for the specification, adaptationand refinement of synchronization points, synchronized motions and master-slave relations during program parts. Ourapproach supports the user by handling motion constraintsand geometrical transformations necessary for refinement andtransfer of program sequences between arms implicitly. Inturn, the knowledge necessary to describe and represent therespective operations for later re-use and refinement is madeexplicit and can hence be captured in our skill representation.We report on two experiments confirming the applicabilityand efficiency of our approach. (Less)