In this position paper we argue that robots do not have the status of moral agents and will not gain it in the foreseeable future; rather, moral responsibility for robot behaviors must be traced to humans. A military scenario is outlined, to illustrate how the domain is well-suited to research on ethical robotics systems, interpreted broadly to include robots deployed within a larger organizational framework with humans holding responsibilities in different roles and acting according to policy. We break down the scenario into parts that illustrate the challenges of ethical decision making with robots in the mix, and we identify areas in artificial intelligence that we believe are central to progress.

We describe a computational cognitive model intended to be a generalizable classifier that can provide context-based feedback to semantic perception in robotic applications. Many classifiers (including cognitive models of categorization) perform well at the task of associating features with objects. Underlying their performance is an effective selection of the features used during classification. This feature selection (FS) process is usually performed outside the boundaries of the models that learn and perform classification tasks, often by human experts. In contrast, the cognitive model we describe simultaneously learns which features to use, as it learns the associations between features and classes. This integration of FS and class learning in one model makes it complementary to other machine-learning techniques that generate feature-based representations (e.g., deep learning methods). But their integration in a cognitive architecture also provides a means for creating a dynamic context that includes disparate sources of information (e.g., environmental observations, task knowledge, commands from humans). This richer context, in turn, provides a means for making semantic perception goal-directed. We demonstrate automated FS, integrated with an instance-based learning approach to classification, in an ACT-R model of categorization by labeling facial expressions of emotion (e.g., happy, sad), and then generalizing the model to the classification of indoor public spaces (e.g., cafes, classrooms).

The Army Research Laboratory's Robotics Collaborative Technology Alliance is a program intended to change robots from tools that soldiers use into teammates alongside which soldiers can work. This requires the integration of fundamental and applied research in robotic perception, intelligence, manipulation, mobility, and human-robot interaction. In this paper, we present the results of assessments conducted in 2016 to evaluate the capabilities of a new robot, the Robotic Manipulator (RoMan), and of a cognitive architecture (ACT-R). The RoMan platform was evaluated on its ability to conduct a search and grasp task under a variety of conditions. Specifically, it was required to search for and recognize a gas can placed on the floor, and then pick it up. The RoMan showed the potential to be a good platform for autonomous manipulation, but the autonomy used in these experiments will require improvement to make full use of the platform's capabilities. The cognitive architecture was evaluated as to how well it could learn to select an appropriate set of features for a classification task. The task was to classify emotions that had been encoded using the Facial Action Coding System, with ACT-R learning to select the most effective set of features for correct classification. ACT-R leaned rules which required it to observe about half of the available features to make a decision, and the subsequent decisions had an accuracy ranging from 76% to 93% (depending on the emotion).

Considering context and the relationship between objects and events in the environment is a key component to developing the situational awareness necessary to accomplish abstract goals or effectively share information. We extend state of the art perception algorithms, which can identify specific objects in the environment, using cognitive reasoning to develop a deeper understanding of the scene based on the both the objects and their spatial relationships. Such understanding could impact the choice of tasks, routes, or observation positions in a military mission.

We describe a computational cognitive model intended to be a generalizable classifier that can provide context-based feedback to semantic perception in robotic applications. Many classifiers (including cognitive models of categorization) perform well at the task of associating features with objects. Underlying their performance is an effective selection of the features used during classification. This Feature Selection (FS) process is usually performed outside the boundaries of the model that learns and performs the classification task, often by a human expert. In contrast, the cognitive model we describe simultaneously learns which features to use, as it learns the associations between features and classes. This integration of FS and class learning in one model makes it complementary to other Machine-Learning (ML) techniques that automate the FS process (e.g., deep learning methods). But their integration in a cognitive architecture also provides a means for creating a dynamic context that includes disparate sources of information (e.g., environmental observations, task knowledge, commands from humans); this richer context, in turn, provides a means for making semantic perception goal-directed. We demonstrate automated FS, integrated with an Instance-Based Learning (IBL) approach to classification, in an ACT-R model of categorization by labeling facial expressions of emotion (e.g., happy, sad) from a set of relevant, irrelevant, distinct, and overlapping facial action features.

This work focuses on a robot's task of predicting the navigation intent of human teammates using Inverse Reinforcement Learning. The purpose of this study is to introduce the On-the-fly Maximum Margin Planner (OTF-MMP) method which estimates a predictive navigation model in real-time from the observed actions of a human teammate. We include an experiment to test the predictive ability of the method using simulation.

Smartphones have put powerful sensor arrays in nearly everyone's pockets. Fusing the data from these sensors it is possible to estimate the phone's current orientation. In this study we utilize a 3 axis gimbal to compare the performance of multiple orientation estimation algorithms. Controlling the position of the gimbal allows us to compare the known device orientation to the estimated orientation. Using this same method we determine where each algorithm's faults lie, and where they begin to break down. Then repeating these movements we are able to compare each algorithm to each other.

The Robotics Collaborative Technology Alliance (RCTA) program focuses on four overlapping technology areas: Perception, Intelligence, Human-Robot Interaction (HRI), and Dexterous Manipulation and Unique Mobility (DMUM). In addition, the RCTA program has a requirement to assess progress of this research in standalone as well as integrated form. Since the research is evolving and the robotic platforms with unique mobility and dexterous manipulation are in the early development stage and very expensive, an alternate approach is needed for efficient assessment. Simulation of robotic systems, platforms, sensors, and algorithms, is an attractive alternative to expensive field-based testing. Simulation can provide insight during development and debugging unavailable by many other means. This paper explores the maturity of robotic simulation systems for applications to real-world problems in robotic systems research. Open source (such as Gazebo and Moby), commercial (Simulink, Actin, LMS), government (ANVEL/VANE), and the RCTA-developed RIVET simulation environments are examined with respect to their application in the robotic research domains of Perception, Intelligence, HRI, and DMUM. Tradeoffs for applications to representative problems from each domain are presented, along with known deficiencies and disadvantages. In particular, no single robotic simulation environment adequately covers the needs of the robotic researcher in all of the domains. Simulation for DMUM poses unique constraints on the development of physics-based computational models of the robot, the environment and objects within the environment, and the interactions between them. Most current robot simulations focus on quasi-static systems, but dynamic robotic motion places an increased emphasis on the accuracy of the computational models. In order to understand the interaction of dynamic multi-body systems, such as limbed robots, with the environment, it may be necessary to build component-level computational models to provide the necessary simulation fidelity for accuracy. However, the Perception domain remains the most problematic for adequate simulation performance due to the often cartoon nature of computer rendering and the inability to model realistic electromagnetic radiation effects, such as multiple reflections, in real-time.

An autonomous mobile robot, working with human teammates, should be equipped to intelligently react to changes in team behavior without relying on directives from human team members. To respond appropriately to changes in team behavior, the robot should detect when these situations occur, and correctly classify the new team behavior. We demonstrate a method for detecting and classifying behavior changes in a simulated team, using the team's focus of attention. The method draws from Kim et al. (2010), who developed an algorithm for propagating the motion of soccer players through a vector field in order to predict locations of future action in a soccer game. Using this propagation method, our implementation extends this work by extracting statistical features from the motion information, and, looking back over a window of prior feature values, detects changes in the team behavior and classifies group activity according to a set of possible behaviors.

Indoor scene recognition remains a challenging problem for autonomous systems. Recognizing public spaces e.g., libraries, classrooms, which contain collections of commonplace objects e.g., chairs, tables, is particularly vexing; different furniture arrangements imply different types of social interaction, hence different scene labels. If people arrange rooms to support social interactions of one type or another, then object relationships that reflect the general notion of social immediacy may resolve some of the ambiguity encountered during scene recognition. We thus describe an approach to indoor scene recognition that uses the context provided by inferred social affordances as input to a hybrid cognitive architecture ACT-R that can represent, apply and learn knowledge relevant to classifying scenes. To provide common ground, we demonstrate how sub-symbolic learning processes in ACT-R, which plausibly give rise to human cognition, can mimic the performance of a simple, widely used machine learning technique k-nearest neighbor classification.

To support the missions and tasks of mixed robotic/human teams, future robotic systems will need to adapt to the dynamic behavior of both teammates and opponents. One of the basic elements of this adaptation is the ability to exploit both long and short-term temporal data. This adaptation allows robotic systems to predict/anticipate, as well as influence, future behavior for both opponents and teammates and will afford the system the ability to adjust its own behavior in order to optimize its ability to achieve the mission goals.This work is a preliminary step in the effort to develop online entity behavior models through a combination of learning techniques and observations. As knowledge is extracted from the system through sensor and temporal feedback, agents within the multi-agent system attempt to develop and exploit a basic movement model of an opponent. For the purpose of this work, extraction and exploitation is performed through the use of a discretized two-dimensional game. The game consists of a predetermined number of sentries attempting to keep an unknown intruder agent from penetrating their territory. The sentries utilize temporal data coupled with past opponent observations to hypothesize the probable locations of the opponent and thus optimize their guarding locations.

Autonomous exploration and mapping is a vital capability for future robotic systems expected to function in arbitrary complex environments. In this paper, we describe an end-to-end robotic solution for remotely mapping buildings. For a typical mapping system, an unmanned system is directed to enter an unknown building at a distance, sense the internal structure, and, barring additional tasks, while in situ, create a 2-D map of the building. This map provides a useful and intuitive representation of the environment for the remote operator. We have integrated a robust mapping and exploration system utilizing laser range scanners and RGB-D cameras, and we demonstrate an exploration and meta-cognition algorithm on a robotic platform. The algorithm allows the robot to safely navigate the building, explore the interior, report significant features to the operator, and generate a consistent map all while maintaining localization.

In theory, autonomous robotic swarms can be used for critical Army tasks, including accompanying vehicle convoys to provide security and enhance situational awareness. However, the Soldier providing swarm supervisory control must be able to correct swarm actions, especially in disrupted or degraded conditions. Dynamic map displays are visual interfaces that can be useful for swarm supervisory control tasks, because they can show the spatial positions of objects of interest (e.g., people, robots, swarm members, and vehicles), at different locations (e.g., on roads and intersections), while allowing user commands as well as world changes, often in real time. In this study, multimodal speech and touch controls were designed for a U.S. Army Research Laboratory dynamic map display to allow users to provide supervisory control of a simulated robotic swarm. This experiment explored the use of sequential multimodal touch and speech commands for placement of swarm-related map objects at different map locations. The criterion variable was temporal binding, the time between the onset of each command in the sequence, relative to the system's ability to fuse the two sequential commands into a unitary response. User preference of modality for the first command was also measured. These concepts were tested in a laboratory study using 12 male Marine volunteers with a mean age of 19 years. Results indicated significant differences in temporal binding for different map objects and map locations. Additionally, nine out of 12 Marines used speech commands approximately 75% or more of the time, while the remaining three Marines used touch commands first approximately 75% or more of the time. Temporal binding was significantly shorter for touch-first than for speech-first commands. Suggestions for future research and future applications to robotic command and control systems are described.

This paper introduces a method to integrate Unmanned Aircraft Systems (UASs) into a highly functional manned/unmanned team through the design and implementation of 3D distributed formation/flight control algorithms with the goal to act as wingmen for a manned aircraft. The proposed algorithms are designed to increase UAS autonomy, dynamically modify formations, utilize standard operating formations to reduce pilot resistance to integration, and support splinter groups for surveillance and/or as safeguards between potential threats and manned vehicles. The proposed work coordinates UAS members by utilizing artificial potential functions whose values are based on the state of the unmanned and manned assets including the desired formation, obstacles, task assignments, and perceived intentions. The overall unmanned team geometry is controlled using weighted potential fields. Individual UASs utilize fuzzy logic controllers for stability and navigation as well as a fuzzy reasoning engine for predicting the intent of surrounding aircrafts. Approaches are demonstrated in simulation using the commercial simulator X-Plane and controllers designed in Matlab/Simulink. Experiments include staggered trail and right echelon formations as well as splinter group surveillance.

In this paper, we present a novel strategy for organizing groups of robots into a formation utilizing artificial potentials which behave like a bar magnet. A mathematical surface is defined that will “pull” the robots into formation controlling the overall geometry. Nonlinear limiting functions are defined to attract and hold members in a geometric formation. By adjusting control parameters, the shape and the extent of the formation is controlled. Formations can dynamically change and adapt accordingly by making the control parameters time varying. This approach is computationally efficient and scales well to large team sizes. Simulation studies are presented for ten robots in wedge, inverted ‘vee’, and column formations.

In this paper, we present a strategy for organizing swarms of unmanned vehicles into a formation by utilizing artificial potential fields that were generated from normal and sigmoid functions. These functions construct the surface on which swarm members travel, controlling the overall swarm geometry and the individual member spacing. Nonlinear limiting functions are defined to provide tighter swarm control by modifying and adjusting a set of control variables that force the swarm to behave according to set constraints, formation, and member spacing. The artificial potential functions and limiting functions are combined to control swarm formation, orientation, and swarm movement as a whole. Parameters are chosen based on desired formation and user-defined constraints. This approach is computationally efficient and scales well to different swarm sizes, to heterogeneous systems, and to both centralized and decentralized swarm models. Simulation results are presented for a swarm of 10 and 40 robots that follow circle, ellipse, and wedge formations. Experimental results are included to demonstrate the applicability of the approach on a swarm of four custom-built unmanned ground vehicles (UGVs).

This paper addresses the problem of controlling and coordinating heterogeneous unmanned systems required to move as a group while maintaining formation. We propose a strategy to coordinate groups of unmanned ground vehicles (UGVs) with one or more unmanned aerial vehicles (UAVs). UAVs can be utilized in one of two ways: (1) as alpha robots to guide the UGVs; and (2) as beta robots to surround the UGVs and adapt accordingly. In the first approach, the UAV guides a swarm of UGVs controlling their overall formation. In the second approach, the UGVs guide the UAVs controlling their formation. The unmanned systems are brought into a formation utilizing artificial potential fields generated from normal and sigmoid functions. These functions control the overall swarm geometry. Nonlinear limiting functions are defined to provide tighter swarm control by modifying and adjusting a set of control variables forcing the swarm to behave according to set constraints. Formations derived are subsets of elliptical curves but can be generalized to any curvilinear shape. Both approaches are demonstrated in simulation and experimentally. To demonstrate the second approach in simulation, a swarm of forty UAVs is utilized in a convoy protection mission. As a convoy of UGVs travels, UAVs dynamically and intelligently adapt their formation in order to protect the convoy of vehicles as it moves. Experimental results are presented to demonstrate the approach using a fully autonomous group of three UGVs and a single UAV helicopter for coordination.

In this paper, we present algorithms and display concepts that allow Soldiers to efficiently interact with a robotic swarm that is participating in a representative convoy mission. A critical aspect of swarm control, especially in disrupted or degraded conditions, is Soldier-swarm interaction-the Soldier must be kept cognizant of swarm operations through an interface that allows him or her to monitor status and/or institute corrective actions. We provide a control method for the swarm that adapts easily to changing battlefield conditions, metrics and supervisory algorithms that enable swarm members to economically monitor changes in swarm status as they execute the mission, and display concepts that can efficiently and effectively communicate swarm status to Soldiers in challenging battlefield environments.