Evaluating perception sensor systems for on-road autonomous vehicles is still fraught with ambiguity due to the varying and dynamic environmental factors that these vehicles encounter. The critical safety of on-road AV operations is heavily dependent on perception sensors, systems, and technologies which form the “eyes” for these vehicles. For this reason, it is important that these systems provide high fidelity data with low uncertainties and latencies, and that they are robust and tamper-proof. This work is an effort to address some of those pressing needs. The needs are first addressed by developing a testbed with a common data acquisition platform, and second by developing various procedures to evaluate automotive lidars in a static environment.

With the advent of technologies to support autonomous vehicles (AVs), there is a proliferation of different AV technologies from a variety of companies and organizations. With this increase in options comes the need to evaluate the operation of AV technologies to ensure safety and accuracy. Of particular note for physical evaluation involves the perception systems of AVs. However, there is a lack of standard methods to physically evaluate the perception system of AVs. A set of test artifacts can be used to compare the performances of perception systems, but the artifacts must be usable with different types of perception sensors. This article presents the development of an artifact that has both undetectable and detectable edge cases for light detection and ranging (LiDAR) and radar sensors. Specifically, different physical properties were investigated to design the proposed artifact with the desired capabilities of achieving detectable and undetectable edge cases under different conditions. With rigorous testing, a final design for the test artifact was completed where its detectable component reflects at least 7.47 times more radio wave energy and results in at least 1.92 times the amount of LiDAR points as compared with the undetectable component. The test artifact was further tested in outdoor conditions in addition to misaligned positions to demonstrate the versatility and potential weakness of the test artifact, respectively. The demonstrated test artifact in this research can therefore be used to compare the performance of different LiDAR and radar models within AV perception systems.

ARIAC is a robotic simulation competition promoted by NIST annually since 2017, aiming to present competitors’ with contemporary industry problems to be solved using agile robotics. For the 2023 competition, ARIAC competitors must perform assembly and kitting tasks by controlling four autonomous ground vehicles (AGVs), one floor-based robot, and one ceiling-based (Gantry) robot in an attempt to overcome a range of agility challenges in the supplied simulated environment, itself based on the Robot Operating System (ROS 2) and Gazebo. The 2023 competition also included a “human” agility challenge, comprising a (simulated) human operator working among robots on the factory floor. This development was motivated by the fact that, while robots and automation play an increasingly significant role in modern manufacturing, there still remains a close relationship between machines and humans. They should complement each other’s strengths and cover each other’s limitations while also observing any required safety rules. For example, the ISO standard “Robots and Robotic Devices – Collaborative robots” (ISO 15066:2016) prescribes the distances required between humans and robots. Within the ARIAC simulation environment, each human operator is controlled using autonomous Belief-Desire-Intention (BDI) agents. At the same time, competitors can monitor the position of each human operator at any time by subscribing to the relevant ROS topic. In this article, we analyse the effects of this (simulated) human presence in the 2023 ARIAC competition and perform a detailed analysis of how the three different human personalities that were implemented affect the assembly tasks undertaken at the four different locations of the assembly stations. Given how the system is currently implemented, it appears that the influence of each encoded personality on the competitors is not as predictable as anticipated. We expand on why this may be a problem when addressing real collaborative spaces involving humans and industrial robots and the improvements that can be undertaken to mitigate the ensuing problems.

In all types of communication, the ability to share information is often hindered because the meaning of information can be drastically affected by the context in which it is viewed and interpreted. This is true in manufacturing, because of the growing complexity of manufacturing information and the increasing need to exchange this information among various software applications. This is particularly true in construction, because of the number of the actors involved in the construction process and the diversity of the information handled during the construction stages (design, construction, operation of the building). A solution to this problem is the development of a common language enabling all the actors of the construction process to share the same semantic concepts intrinsic to the capture and exchange of information related to the construction process. The aim of this paper is to present the PSL language, its main features and the underlying ontology, and to analyse its " applicability " to the construction sector.

The IEEE Robotics and Automation Society’s (RAS) standards working groups continue to grow.

The "Agile Robotics for Industrial Automation Competition" (ARIAC) is an international robotic competition carried out in a simulated factory floor using ROS 2 (Robot Operating System)/Gazebo. Competitors control one gantry robot, four AGVs, and many other elements/devices, overcoming a range of agility challenges in this simulated environment, and are provided with a scoring system to evaluate their performance during the tasks. This paper describes one of the agility challenges in ARIAC 2023, which pertains to a simulated human operator on the factory floor. In undertaking manufacturing tasks, competitors must avoid close proximity between the gantry robot and the human not to get penalized. The human operator is implemented as a Belief-Desire-Intention (BDI) agent in Jason. It is provided with a range of different potential types of behaviour in what concerns with how such human reacts when in proximity to the gantry robot. Three different personalities are presented, ranging from a minimally intrusive up to a very intrusive one. A preliminary analysis was conducted to evaluate the impact of using the developed Jason agent in the ARIAC 2023 competition.

The future of robotics foresees autonomous behavior that can complete tasks intelligently, with a focus on adaptability, flexibility, and versatility. In such systems, it is critical for robots to quickly and safely perform an operation. However, such aptitude is not limited to the speed of solving tasks, but also requires other qualities such as adeptly detecting and recovering from task irregularities, overcoming unforeseen task barriers by replanning to achieve stated goals, and adroitly adapting to dynamic environments such as changing light illumination, noisy sensors, or unexpected conditions. These intelligent characteristics define robot agility (not to be confused with robot agility akin to dexterity), and refer to approaches that allow robotic systems to be flexible and capable of re-tasking in the face of a changing and often unpredictable environment. Because robot task agility requires sophisticated dynamic and continuous planning and replanning, the Gwendolen intelligent agent programming language is studied as a high-level robot planner. In this report, we develop a manufacturing kitting case study to research the operation of Gwendolen planning. The case study uses the combination of Gwendolen, Canonical Robot Command Language (CRCL), Robot Operating System (ROS), and Gazebo software components to simulate and evaluate robot planning. Several Agile Robotics for Industrial Applications Competition (ARIAC) kitting agility challenges are used to evaluate Gwendolen planning under various levels of operational duress. Both the benefits and shortcomings will be reviewed.

The IEEE Robotics and Automation Society’s standards working groups continue to grow.

On-road autonomous vehicles are expected to significantly influence key aspects of everyday life. However, these complex systems can pose a safety risk in the event of unexpected system performance. Therefore, NIST held the Standards and Performance Metrics for On-Road Autonomous Vehicles Workshop to solicit stakeholder feedback with respect to challenges and opportunities in developing standards and performance metrics for this complex interdisciplinary field. In addition, this workshop aimed to foster a community consisting of stakeholders from a variety of disciplines and domains in autonomous vehicles. This workshop was conducted virtually over 2 days (March 8th-9th, 2022) consisting of introductory remarks, panels, breakout sessions, and concluding remarks. This report is a summary of the technical topics discussed by participants during the workshop.

Task agility is an increasingly desirable feature for robots in application domains such as manufacturing. The Canonical Robot Command Language (CRCL) is a lightweight information model built for agile tasking of robotic systems. CRCL replaces the underlying complex proprietary robot programming interface with a standard interface. In this paper, we exchange the automated planning component that CRCL used in the past for a rational agent in the Gwendolen agent programming language, thus providing greater possibilities for formal verification and explicit autonomy. We evaluate our approach by performing agile tasking in a kitting case study.

Presents information on the RAS October 2020 Terminology Harmonization Meeting.

Manufacturers are looking for intelligent solutions to increase quality and productivity. Smart manufacturing envisions production empowered by autonomous robots that can complete tasks intelligently, with the focus on adaptability, flexibility, and versatility. In such systems, agile tasking plays an important role, as it is critical for robots to be quickly tasked to perform an operation. However, task agility is not limited to the speed of tasking robots, but also includes other features such as handling task failure, planning for new goals, interchangeability of data and task plans between different robots, and adapting to dynamic environments. Because robot task agility requires sophisticated dynamic and continuous planning and replanning, the Gwendolen agent programming language was chosen to evaluate as the agile robot planner. In this paper, we develop a manufacturing kitting case study and provide a list of kitting performance metrics to evaluate performance. The case study uses Gwendolen, Canonical Robot Command Language (CRCL), Robot Operating System (ROS) and Gazebo software components in combination to simulate and evaluate kitting. We explore the strengths of Gwendolen agile tasking to assess the operation against the kitting performance metrics.

A thriving manufacturing sector is the essential heart of a vibrant and balanced economy in the United States (U.S.). Small and medium-sized manufacturers (SMMs) constitute an important sector in the U.S. manufacturing but they are currently facing increasing competition due to economic globalization. To survive and thrive in this highly competitive environment, SMMs have to rely on automation and robotics, which bring with them a whole series of techniques to improve the quality and productivity of a manufacturing process. However, robotic systems need to be agile for them to be useful to SMMs so they can offer more automated customization of high-mix/low-volume production. This paper focuses mainly on the metrics used in the Agile Robotics for Industrial Automation Competition (ARIAC). The goal of the competition along with its associated metrics is to promote advances in research by assessing the performance of industrial robotic systems in manufacturing settings.

While multi-robot cells are being used more often in industry, the problem of work-piece position optimization is still solved using heuristics and the human experience and, in most industrial cases, even a feasible solution takes a considerable amount of trials to be found. Indeed, the optimization of a generic performance index along a path is complex, due to the dimension of the feasible-configuration space. This work faces this challenge by proposing an iterative layered-optimization method that integrates a Whale Optimization and an Ant Colony Optimization algorithm, the method allows the optimization of a user-defined objective function, along a working path, in order to achieve a quasi-optimal, collision free solution in the feasible-configuration space.

Manufacturing and Industrial Robotics have reached a point where to be more useful to small and medium sized manufacturers, the systems must become more agile and must be able to adapt to changes in the environment. This paper describes the process for creating and the lessons learned over multiple years of the Agile Robotics for Industrial Automation Competition (ARIAC) being run by the National Institute of Standards and Technology.

ARIAC (Agile Robotics for Industrial Automation Competition) is a robotic competition which aims to advance robotic agility in industry. Participants in this competition are required to implement a robot control system to overcome agility challenges in a simulated environment. ARIAC comes with a set of score metrics to evaluate the performance of each control system during task execution. In this paper we show how such task-oriented evaluation can be problematic and how the addition of runtime monitors to verify properties given in ISO/TS safety standards can help in reducing the resulting reality gap.