One key area of research in Human-Robot Interaction is solving the human-robot correspondence problem, which asks how a robot can learn to reproduce a human motion demonstration when the human and robot have different dynamics and kinematic structures. Evaluating these correspondence problem solutions often requires the use of qualitative surveys that can be time consuming to design and administer. Additionally, qualitative survey results vary depending on the population of survey participants. In this paper, we propose the use of heterogeneous time-series similarity measures as a quantitative evaluation metric for evaluating motion correspondence to complement these qualitative surveys. To assess the suitability of these measures, we develop a behavioral cloning-based motion correspondence model, and evaluate it with a qualitative survey as well as quantitative measures. By comparing the resulting similarity scores with the human survey results, we identify Gromov Dynamic Time Warping as a promising quantitative measure for evaluating motion correspondence.

Robotics and automation technology has experienced an incredible growth period in the past decade. The barrier for entry across many economic sectors has been reduced due to cheaper computing, sensing, and actuation coupled with advances in machine learning algorithms and open source tools. Many companies have sprouted up to create myriad components and robotic systems that solve different domain problems. However, managing the complexity of a robotic system is becoming more challenging as all of these hardware and software components lack standards to ensure that integration works smoothly. Several years ago, the IEEE Robotics and Automation Society established the Standing Committee for Standards Activities (SCSA; https://www.ieee-ras.org/industry-government/standards ) to develop multiple standards that promote common definitions and measures that support the growing robotics and automation market.

As robot sensors, mechanisms, and artificial intelligence algorithms increase in capability and availability, the performing arts has incorporated robots into different creative works. These performances often focus on humans and robots as separately controlled entities that interact at specific points in time. In this paper, we propose, implement, and evaluate a human-robot teaming platform that enables the live choreography of human-robot teams that work together within an improvisational performance context. This platform unifies robots and humans using a novel integration of programming abstractions, wireless haptics, and autonomous behaviors that provide a foundation to construct and manage non-dyadic human-robot teams. Furthermore, we demonstrate this platform in a live human-robot choreographic experience with close audience interaction. We use this performance to gather qualitative data about audience sentiment and performer experience that informs how these human-robot teams might be perceived when operating in close proximity.

Urban Air Mobility (UAM) is an emerging transport solution for passengers or cargo at lower altitudes within urban and suburban areas using electric Vertical Take-off and Landing (eVTOL) air vehicles. The complexity of UAM technology strains design-time safety-assurance methods, which is driving the use of dynamic monitoring methods such as runtime verification to ensure safety of intended functionality (SOTIF). Although cyber-physical system (CPS) testing and design-assurance literature exists [1], there is little focus on UAM testbed concepts that evaluate runtime safety assurance. Such testbeds would help ensure safety coverage across a range of failure scenarios. In this paper, we present a new testbed architecture that systematically integrates formal runtime verification methods and tools into an Unmanned Aerial Vehicle (UAV) flight simulator. The value of this testbed is to help evaluate and transition in-time hazard detection and mitigation methods (in realistic operational contexts) to higher levels of technology maturity. To the best of our knowledge this is the first testbed that integrates formal runtime verification tools into a eVTOL simulation testbed.

Path-constrained trajectory optimization research normally focuses on time or energy optimality. However, some applications seek reference trajectories that satisfy other constraints. In this paper, we formulate a control-minimal time-assigned path-constrained trajectory optimization problem: a mobile ground robot must traverse a given path in a specific amount of time using minimal control effort. Through a nonlinear change of variables, we solve this problem using convex optimization. We evaluate our solution with an intelligent transportation scenario where an autonomous vehicle must cross an intersection in a specific amount of time while following the turn lane's geometric center.

This paper presents an experiment orchestration platform, called VirtualLab@OpenCyberCity, that supports the research and education of smart city technologies. This platform will allow researchers and students to provision distributed experiments across the cyber-physical agents within OpenCyberCity. These new capabilities will support building a cyber-physical systems workforce with hands-on-experience using technologies that will be incorporated into smart city solutions. Virtual-Lab@OpenCyberCity will (a) provide a learning ecosystem of advanced CPS technologies, (b) inform the employment of advanced technologies and intelligent management systems for smart city planners, and (c) foster fruitful collaboration among academia, industry, and government stakeholders to build a smart city innovation workforce.

Advanced Air Mobility (AAM) and Urban Air Mobility (UAM) are emerging concepts that capitalize on urban airspace for commercial transportation of passengers and cargo as well as augment surface transportation infrastructure. A major part of the solution towards achieving the UAM/AAM vision will be assured and trusted autonomy that enable human operators and passengers to interact with autonomous systems that transport humans and cargo with high assurance of safety, security and reliability. Given the complex interactions among the autonomy algorithms, human, environment and UAM control, there is a need for vehicle-level monitoring that detects emerging hazard scenarios. The data gathered from UAM systems that would help detect emerging hazards will be diverse: from highly-regular information (onboard flight controls) to irregular streaming information (weather, real time population dynamics). This paper presents a systematic approach for developing model- and data-driven hazard monitoring framework which we call pervasive monitoring. The aim of pervasive monitoring is to “comprehensively" observe a Cyber Physical System (CPS) at different architectural levels to ensure its safety and security with respect to its intended operation. Since there is no single monitor type that solves complex in-time hazard detection problems for UAM, we assert that several classes of monitors are needed to address this challenge. In this paper, we present a methodology based on STPA that partitions the UAM hazard monitoring challenge into two problems: a context monitoring problem and vehicle monitoring problem. We present preliminary results on deriving monitors from STPA, and realization of the monitors using the NASA Runtime Verification language Co-pilot.

Unsignalized intersections are often sources of congestion and collisions. When human-driven vehicles arrive simultaneously, the drivers typically creep out into the intersection or wave each other through to break stalemates. While intuitive for human drivers, this approach would be challenging for autonomous vehicles (AVs). Current AVs typically operate in isolation without explicitly communicating their intentions to others. In this paper, we propose an auction-based intersection management system (IMS) to determine a crossing schedule. Vehicles bid for crossing time using a cost function over different possible crossing times, and the IMS assigns crossing times that maximize social utility. We evaluate our system with an ambiguous crossing scenario and demonstrate its usefulness in determining socially-optimal crossing schedules.

Runtime verification (RV) has the potential to enable the safe operation of safety-critical systems that are too complex to formally verify, such as Robot Operating System 2 (ROS2) applications. Writing correct monitors can itself be complex, and errors in the monitoring subsystem threaten the mission as a whole. This paper provides an overview of a formal approach to generating runtime monitors for autonomous robots from requirements written in a structured natural language. Our approach integrates the Formal Requirement Elicitation Tool (FRET) with Copilot, a runtime verification framework, through the Ogma integration tool. FRET is used to specify requirements with unambiguous semantics, which are then automatically translated into temporal logic formulae Ogma generates monitor specifications from the FRET output, which are compiled into hard-real time C99. To facilitate integration of the monitors in ROS2, we have extended Ogma to generate ROS2 packages defining monitoring nodes, which run the monitors when new data becomes available, and publish the results of any violations. The goal of our approach is to treat the generated ROS2 packages as black boxes and integrate them into larger ROS2 systems with minimal effort.

This paper presents a prototype framework for developing real-time human-robot performances and its application within a recent choreographic work for the stage. The framework allows dancers to teach new motions to robots, while also giving choreographers the ability to compose performances from pre-built and learned behaviors. To achieve this capability, we combine behavior-based robotics and learning from demonstration approaches to construct behaviors and compose them into a performance. Our learning algorithm, dancing-from-demonstration (DfD), allows dancers and choreographers to teach new phrases to the robot and specify choreographic motifs for the performance. This collaborative work culminated in a human-robot duet, where the robot incorporates a new, learned motion into its choreography within live performance. These capabilities create a baseline for choreographers and dancers to eventually compose and perform in more dynamic, reactive choreo-robotic performances.

A smart city environment utilizes different types of Internet of Things (IoT) devices, i.e., sensors and actuators, to collect and analyze millions of data to manage assets, resources, and services efficiently to improve the quality of life. However, the data collection process is one of the most challenging tasks for smart cities. To meet the requirements of smart city applications, a robust and efficient data collection process is needed to collect data from IoT devices then deliver it to data centers. This paper proposes an end-to-end data collection architecture from IoT devices for different smart city applications, such as smart buildings, transportation, smart grids, healthcare, and others, through advanced communication technologies and protocols and database management approaches. The proposed architecture includes five main components, (1) IoT Networks, (2) Server-side MQTT Interfacing, (3) Server-side Streaming, (4) NoSQL Database, and (5) Web Server. Finally, this paper provides a case study to demonstrate the effectiveness of the proposed architecture and discuss each design component in detail.

This paper presents an open architecture testbed for smart cities, called OpenCity, which is hosted at Virginia Commonwealth University (VCU). The OpenCity platform consists of data collection and processing units, database management, distributed performance management algorithms, and real-time data visualization. This smart city testbed aims to support various educational and research activities related to smart city development. The testbed provides a near-real-life platform to allow students to learn about the unique features of smart cities and explore supporting technologies. In addition, it allows researchers to develop, deploy, and validate new techniques, tools, and technologies to support future smart city developments. The OpenCity platform will support various ongoing important research directions in smart cities, including smart homes and buildings, urban mobility, smart grid, and water management. In addition, it will be extendable to include other potential applications and components as needed. The testbed will be validated by developing and deploying a management system that focuses on users’ experience and resource efficiency. The management system incorporates learning techniques and model-based predictive control approaches to take into account the current and future information of uncertain parameters as well as the subjective data (e.g., user-related data) in the design. The OpenCity management structure enables real-time control and monitoring of complex components in the testbed.

Reinforcement learning in heterogeneous multi-agent scenarios is important for real-world applications but presents challenges beyond those seen in homogeneous settings and simple benchmarks. In this work, we present an actor-critic algorithm that allows a team of heterogeneous agents to learn decentralized control policies for covering an unknown environment. This task is of interest to national security and emergency response organizations that would like to enhance situational awareness in hazardous areas by deploying teams of unmanned aerial vehicles. To solve this multi-agent coverage path planning problem in unknown environments, we augment a multi-agent actor-critic architecture with a new state encoding structure and triplet learning loss to support heterogeneous agent learning. We developed a simulation environment that includes real-world environmental factors such as turbulence, delayed communication, and agent loss, to train teams of agents as well as probe their robustness and flexibility to such disturbances.

This paper presents a cloud-computing inspired framework that facilitates the programming of a deployed cyber-physical system. This framework, PhysiCloud, uses a novel combination of abstractions that hide the implementation details of the underlying cyber-physical system. Additionally, the framework is designed to operate on low-power, mobile systems with resiliency to network failures. Using this system, a controls application developer can focus on their algorithm development and its information dependencies, rather than issues of low level scheduling and communication.

The development of control applications for multi-agent robot and sensor networks is complicated by the heterogeneous nature of the systems involved, as well as their physical capabilities (or limitations).We propose a software framework that unifies these networked systems, thus facilitating the development of multiagent control across multiple platforms and application domains. This framework addresses the need for these systems to dynamically adjust their actuating, sensing, and networking capabilities based on physical constraints, such as power levels. Furthermore, it allows for sensing and control algorithms to migrate to different platforms, which gives multi-agent control application designers the ability to adjust sensing and control as the network evolves. This paper describes the design and implementation of our software system and demonstrates its successful application on robots and sensor nodes, which dynamically modify their operational components.

In this paper, we consider the problem of going from high-level specifications of complex control tasks for cyber-physical systems to their actual implementation and execution on physical devices. This transition between abstraction levels inevitably results in a specification-to-execution gap, and we discuss two sources for this gap; namely model based and constraint based. For both of these two types of sources, we show how hybrid control techniques provide the tools needed to compile high-level control programs in such a way that the specification-to-execution gap is removed. The solutions involve introducing new control modes into nominal strings of control modes as well as adjusting the control modes themselves.

Many large-scale multi-agent missions consist of a sequence of subtasks, each of which can be accomplished separately by having agents execute appropriate decentralized controllers. However, many decentralized controllers have network topological prerequisites that must be satisfied in order to achieve the desired effect on a system. Therefore, one cannot always hope to accomplish the original mission by having agents naively switch through executing the controllers for each subtask. This paper extends the Graph Process Specification (GPS) framework, which was presented in previous work as a way to script decentralized control sequences for agents, while ensuring that network topological requirements are satisfied when each controller in the sequence is executed. Atoms, the fundamental building blocks in GPS, each explicitly state a network topological transition. Moreover, they specify the means to make that transition occur by providing a multi-agent controller, as well as a way to locally detect the transition. Scripting a control sequence in GPS therefore reduces to selecting a sequence of atoms from a library to satisfy network topological requirements, and specifying interrupt conditions for switching. As an example of how to construct an atom library, the optimal decentralization algorithm is used to generate atoms for agents to track desired multi-agent motions with when the network topology is static. The paper concludes with a simulation of agents performing a drumline-inspired dance using decentralized controllers generated by optimal decentralization and scripted using GPS.

This technical note investigates how to produce control programs for complex systems in a systematic manner. In particular, we present an abstraction-based approach to the specification and optimization of motion programs for controlling robot marionettes. The resulting programs are based on the concatenation of motion primitives and are further improved upon using recent results in optimal switch-time control. Simulations as well as experimental results illustrate the operation of the proposed method.

We explore the effect that bounded inputs have when we specifying sequences of control laws for driving a linear, dynamical system through intermediary equilibrium points. These sequences, given in the Motion Description Language (MDL) formalism, may not be feasible in that they may violate the bounded input constraints. To overcome this problem, we augment the MDL sequences by inserting extra control modes, resulting in a new motion program that does in fact satisfy the input constraints. We first consider the problem of “compiling” the motion program for feasibility, and then propose a method that constructs new motion programs based on the input bounds, system dynamics, and task design goals.