The military benefit of unmanned reconnaissance for infantry as the most exposed military branch is obvious. Furthermore, unmanned systems can also support by transporting heavy equipment, including sensor payloads usually not fielded by infantry units. While larger assets are typically controlled from afar, smaller assets can be controlled directly by nearby troops and satisfy immediate reconnaissance needs. In this work, the design, implementation and evaluation of a user interface (UI) for integrating unmanned platforms into the German army’s infantry platoons is presented. More specific, two unmanned aerial vehicles and two unmanned ground vehicles were to be integrated into a platoon. This work highlights the user interface aspects, training effort and organizational changes. German paratroopers and mountain infantry assisted with the requirements analysis and UI evaluation. In addition, the German Army Concepts and Capabilities Development Center supported the evaluation. The effort to bring unmanned systems into infantry units is motivated, related work concerning the control of unmanned systems is presented, the results of the requirements elicitation for this undertaking is reported, the design and implementation as well as the instruction strategy are outlined and the results of a test campaign reported. The paper concludes by summing up the current state and outlining future work regarding UI development for soldier-multi-robot-teams.

The workload of a single operator of a multi robot systems increases with the number of robots in use. Supervisory control is a general idea to solve this issue. In this paper we present an experiment in which we compare single robot control and group robot control. Using single robot control the user must task each robot separately. Using group robot control, he can task the whole group and a planning component generates a plan and observes the execution. We compare performance, errors and workload of the user under single robot control and group robot control conditions.

Bolstering up a military unit, e.g., an infantry platoon on a recce mission, by robots often is a double-edged sword. On the one hand, the robots are able to support the soldiers in multiple ways, they can transport bulky equipment, they can enter risky spots, and they may have sensor suits that help to detect dangers of all kinds. On the other hand, robots need to be equipped with energy sources that are cumbersome by themselves. In addition, the robots must be commanded. In order to optimize the support for their unit, it is often necessary to have a team of heterogeneous robots, e.g., UAVs as well as UGVs all with different sensors and specific abilities. Such a team, however, is even harder to command than a team of homogeneous robots. Our research aims at simplifying commanding teams of heterogeneous robots. In order to achieve this aim, we use the standards BML (Battle Management Language) and ROS (Robot Operating System) to communicate with the robot team. BML is used since our approach to commanding robots is from the language point of view very similar to commanding simulated units. Thus, we use language constructions modeled on those we developed as part of the NATO research groups on BML, NATO MSG-048 and NATO MSG-085. Currently, we are testing to use one single mobile GUI, also modeled on our NATO research groups’ results. That GUI is implemented on a tablet and enables the controller to command a team of two UAVs and four UGVs. All the robots can be equipped with different sensor suits. This article presents our solutions about the following topics essential for the described challenge. First, the robots have to introduce themselves by communicating their current abilities and their status to the commander. Second, the commander gives the commands to the robot team in a mission kind fashion. Thus, third, there has to be a kind of intelligence in the robot team that calculates sub tasks out of a given command and distributes these sub tasks to the team members taking the members’ specific abilities into account. Fourth, the robot team has to fuse sensor data in order to send a unified picture back to the commander in order to contribute to the operational picture.

Commanding and controlling a multi-robot system (MRS) is a challenging task with complexity increasing progressively as the number and the heterogeneity of robots in the system increases. A natural language-based interactive command and control system that allows a human operator the ability to focus on mission-critical tasks while directing the activities of the MRS is an essential tool to deal with the complexity inherent in a MRS operation. A controlled natural language is an efficient and intuitive way to interact with systems. We have developed and evaluated such a language for giving robots high level orders as well as for adjusting robot behavior with respect to those high level tasks. As additional support, a scheduling system based on genetic algorithms helps the user by suggesting an efficient schedule for all planned tasks and by keeping track of the resources required. In addition, we have also designed a mechanism to aggregate and fuse the information gathered by the robots so that the aggregated information can be sent to the commander formulated using our controlled language, thus enhancing situation awareness without overwhelming the commander with all the myriad details the multi-robot system gathers.

Dr. J. Mark Pullen and Douglas Corner C4I Center George Mason University Fairfax, VA 22030, USA +1 703 993 3682 mpullen@c4i.gmu.edu dcorner@c4i.gmu.edu Thomas Remmersmann and Dr. Irmtrud Trautwein Fraunhofer Institute for Communication, Information Processing and Ergonomics Fraunhofer Str. 20 53343 Wachtberg Germany +49 (0)228 9435-522 thomas.remmersmann@fkie.fraunhofer.de irmtrud.trautwein@fkie.fraunhofer.de

This paper presents the experience gained during demonstrations carried out between Denmark, France, Germany, the Netherlands and Spain under the umbrella of the NMSG-085 / CIG Land Operation group. The demonstration, also presented in this paper, focuses on command post exercise training. It highlights the benefits of the use of standards to save LOCON resources and time during exercise preparation. The demonstration combines surrogate and operational C2 systems (SIR provided by France, SITAWARE provided by Denmark, C2LG-GUI provided by Germany, and TALOS provided by Spain) with simulations (SWORD provided by France, VR-Forces provided by the Netherlands). The demonstrations addressed first the use of an effective initialization process based on the MSDL standard. MSDL was enriched (extended) for these demonstrations; the rational and content of the enrichment are also presented in the paper. Second, the demonstration deals with the execution of an operational scenario derived from the VIKING 2011 training exercise. Hence, it highlights the use of C-BML for two new military areas: logistics and artillery. Corresponding C-BML schema improvements are introduced in the paper. In addition, the demonstration intends to show that C-BML is also relevant for C2 to C2 exchange.

Using a multi-robot system in military operations poses many control problems. The ability to formulate clear and unambiguous commands for the robots thus is extremely desirable. As numerous autonomous functions have been developed both for single robots and for multirobot systems, we want the operator to express just what needs to be done, whereas the robots have to figure out how to do it. Therefore, we use the Battle Management Language (BML) to command multi-robot systems. As BML has been developed by the military research community, it is – as required – designed to express short, unambiguous orders readable by both humans and machines. This article presents an overview over the BML commands that can be given to the robots as well as an overview over the BML reports generated by robots and presented to the operator.

The paper presents the results of cooperative work done by three different research institutions. The cooperation concluded in a large field experiment with six heterogeneous unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs). It is shown, that it is possible to do reconnaissance and surveillance, namely detection of pedestrians and vehicles, with an autonomous multi-robot system under real world conditions. The participants were all using different robotic middlewares and interfaces to the graphical user interface. Hence, it has been agreed to use common interfaces and standards for communication. The Robot Operating System (ROS) is used as a communication layer, supported by commercially available 3G mobile radio communication for large operating distances. A standardized markup language has been used for the description of tasks and feedback from the robots to the control station. Moreover, the applied UGV control strategies and the implemented approach for detection, tracking and classification are presented. The concluding field test showed that it is possible to realize a multi-robot system of dynamic size, and that such teams are well-suited to perform reconnaissance and surveillance without constant observation by a human operator.

Having robots as reliable and robust mobile sensor platforms in unknown environments is getting more and more attractive. The control of each robot as a single machine is often complicated enough. But if there are more than one robots in a given scenario, the task get’s even harder. The operators then do not only have to care about steering their robot, but they also have to cooperate with each other. In this paper we describe the results of a research project regarding control strategies for a group of heterogeneous, autonomous robots. The swarm receives orders from a central control station, that uses a Battle Management Language, which is abstract, but human readable. An abstract language postulates a certain degree of intelligence within each robot of the cooperation, because the orders are mostly more complex than simple moves from A to B. There is a command hierarchy within the swarm, but every robot implements its own control strategies, to fulfill the overall goal of the orders. The technical as well as the operational realizations are described and discussed mostly with the focus on the unmanned aerial robots.

Multi-Robot Systems have become an important research topic. One of the main questions, when looking at usability of a MRS, is how it can be controlled. In this paper we describe an approach were the commanding is done by using an artificial language very similar to English, the Battle Management Language (BML). The orders can thus be created intuitively and on a high abstraction level. We developed a GUI to allow fast and efficient creating of orders for the robots system. On the robots we used the Robot Operating System (ROS). The interpretation and execution of the orders are controlled by ROS nodes. We created control nodes for every robot which handle the execution of a task for a single robot. We also created intelligent nodes for groups of robots. These nodes handle commands directed to a group of robots and split that BML order into BML orders for each robot. These orders are sent to the control nodes and executed by the robots. ROS provides numerous of libraries and tools which helps to create new robot applications. We mainly used the publish subscriber based communication capabilities. In this paper we concentrated on the architecture and how the translation of BML orders into basic ROS command is done and how feedback messages were sent back to the C2 System. This presented work is the result of cooperation between the Real Time Systems Group (RTS), Leibniz Universität Hannover, the Institute of Real-Time Learning Systems (EZLS), University of Siegen and the Fraunhofer Institute for Communication, Information Processing and Ergonomics.

Securing larger areas with multi-robot systems is a challenging task when you want to have multiple robots controlled by only one person in an efficient way. One way to do this is to let the commanding person express high-level tasks and to build an intelligent multi-robot system (MRS) that can execute these high-level tasks and provide aggregated feedback to the commander. We implemented such an approach by using Battle Management Language for defining high-level tasks. In this paper we will show how tasks are disaggregated, scheduled and distributed among the robots of the MRS.

Natural language is a very intuitive and efficient way to interact with robotic systems and to control mobile multi-robot systems (MRS). Because of this we designed a graphical user interface (GUI) which allows the creation of orders as sentences. The interface allows only sentences in a quasi-natural language that follow production rules and keeps them unambiguous. Because of these properties those sentences can be easily processed by a computer. This concept allows an intuitive input of high level tasks while we avoid natural language processing problems like complex, ambiguous sentences. In this paper we show how those high level tasks can be mapped into basic robot commands. This mapping can require complex conditions, e.g., spatial or temporal constrains, and depends on the capabilities of the MRS. In this paper we focus on how the result of this mapping can be scheduled. We used genetic algorithms to determine which task should be executed by which robot and in what order. A planning algorithm calculates the expected start and end times of every task and tries to satisfy the given constrains.

The paper presents work that has been done by three different research institutions. The aim was to realize an autonomous team of heterogeneous unmanned ground and aerial vehicles performing certain reconnaissance and surveillance tasks, where the tasks were set by an operator at a team level instead of controlling each vehicle seperately. To overcome the lack of a common middleware, the interfaces between vehicles and graphical user interface have been defined using Robot Operating System (ROS) and Battle Management Language (BML). We present approaches for autonomous control of the vehicles, focussing on the unmanned ground vehicle. Moreover, we conducted some large field experiments and present the results.

: Bolstering up a military unit, e.g., an infantry platoon on a recce mission, by robots often is a double-edged sword. On the one hand, the robots are able to support the soldiers in multiple ways, they can transport bulky equipment, they can enter risky spots, and they may have sensor suits that help to detect dangers of all kinds. On the other hand, robots need to be equipped with energy sources that are cumbersome by themselves. In addition, the robots must be commanded. In order to optimize the support for their unit, it is often necessary to have a team of heterogeneous robots, e.g., UAVs as well as UGVs all with different sensors and specific abilities. Such a team, however, is even harder to command than a team of homogeneous robots. Our research aims at simplifying commanding teams of heterogeneous robots. In order to achieve this aim, we use the standards BML (Battle Management Language) and ROS (Robot Operating System) to communicate with the robot team. BML is used since our approach to commanding robots is from the language point of view very similar to commanding simulated units. Thus, we use language constructions modeled on those we developed as part of the NATO research groups on BML, NATO MSG-048 and NATO MSG-085. Currently, we are testing to use one single mobile GUI, also modeled on our NATO research groups results. That GUI is implemented on a tablet and enables the controller to command a team of two UAVs and four UGVs. All the robots can be equipped with different sensor suits.