End user development (EUD) is an emerging paradigm in human–robot interaction, involving end users both creating and maintaining robot programs themselves. Formal methods encompass a set of techniques that are useful to EUD, emphasizing the correctness of programs that are often created using natural, on-the-fly and sometimes even sloppy end-user input. These techniques include, for example, correct-by-construction synthesis of robot programs given incomplete developer input, automated repair of these applications and formal verification of these applications against sets of correctness criteria. Despite the increasing popularity of EUD and formal methods individually in robotics, there is no comprehensive review of how both have been successfully combined. The primary objective is thereby to review current practices for applying formal methods within robot EUD. To achieve this objective, the authors include adjacent areas of EUD research, such as natural language programming, and adjacent approaches to formal methods, such as automated planning. Recognizing that the integration of EUD and formal methods in robotics is still a nascent topic, the secondary objective is to identify opportunities for future research. This review thereby provides a primer on how to conduct further integration of formal methods in robot EUD.
Motion prediction, recently popularized under the term world models, refers to anticipating the future states of agents or the future evolution of a scene, which is rooted in human cognition to bridge perception and decision-making, enabling us to anticipate, adapt, and act within an everchanging world. It lies at the core of intelligent autonomous systems, such as robotics and self-driving cars, to safely operate in dynamic and human-robot-mixed environments, and also informs broader time-series challenges. With advances in methods, representations, and datasets, the field has seen rapid progress, reflected in rapidly updated benchmark performance. However, when state-of-the-art methods are deployed in the real world, they are often found to struggle to generalize to open-world settings and fall short of deployment standards. This reveals a gap between reality and benchmarks, which are often idealized or ill-posed, and fail to capture real-world complexity. To address the pressing need for problem settings that better reflect real-world challenges and guide future research, this paper focuses on revisiting the generalization and applicability of motion prediction models, with an emphasis on robotics, autonomous driving, and human motion applications. We first provide a comprehensive taxonomy of motion prediction methods, covering representations, modelling methods, application domains, and evaluation protocols. We then revisit two fundamental problems: 1) how to push motion prediction models to be deployable to realistic deployment standards, where motion prediction does not act in a vacuum, but functions as one module of closed-loop autonomy stacks – it takes input from the localization and perception, and informs downstream planning and control. 2) how to generalize motion prediction models from limited seen scenarios/datasets to the open-world settings. We conclude by highlighting crucial challenges and open problems for future research. By doing so, we aim to recalibrate the community’s efforts, fostering progress that is not only measurable but also meaningful for real-world applications. The project webpage corresponding to this paper can be found here Link to the website of Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectives
Soft robots are more human-friendly than rigid-bodied robots. This is due to their soft nature that does not harm humans while interacting. This work provides the developments and trend maps for soft continuum manipulators (SCM) and soft snake robots (SSR). Although they are functionally different, they are structurally similar in appearance. The way the robot is modelled, the materials it is made up of, and the various manufacturing methods for the soft continuum robots (SCR) are presented. Most importantly, the soft sensors and soft actuators used for these SCRs are discussed. Then, the manipulation and locomotion of these SCRs are presented. The trend map for the SCM and SSR are laid out and described in detail. We feel that this work will introduce the readers to the soft continuum world and give an overview of the future of SCM and SSR.
Robotics has dramatically increased our ability to gather data about our environments, creating an opportunity for the robotics and algorithms communities to collaborate on novel solutions to environmental monitoring problems. To understand a taxonomy of problems and methods in this realm, we present the first comprehensive survey of decision-theoretic approaches that enable efficient sampling of various environmental processes. We investigate representations for different environments, followed by a discussion of using these presentations to solve tasks of interest, such as learning, localization, and monitoring. To efficiently implement the tasks, decision-theoretic optimization algorithms consider: (1) where to take measurements from, (2) which tasks to be assigned, (3) what samples to collect, (4) when to collect samples, (5) how to learn environment; and (6) who to communicate. Finally, we summarize our study and present the challenges and opportunities in robotic environmental monitoring.
Since the birth of the first industrial robot in the early 1960s, robotics has often replaced humans for tedious and repetitive tasks in the industrial world. To meet these challenges, industrial robots have needed to become specialized. They have been designed according to the task that needs to be performed. In the early 1980s, the ambition to equip robots with robotic hands with universal capabilities led to the development of robotic grasping research. The emergence of more agile industry and also collaborative robotics requires the development of new generation grippers: more versatile, with not only adaptive grasping capabilities but also dexterous manipulation capabilities. The development of flexible multi-fingered grippers with both adaptive grasping and in-hand manipulation capabilities remains a complex issue for human-like dexterous manipulation. After four decades of research in dexterous manipulation, many robotic hands have been developed. The development of these hands however remains a key challenge, as the dexterity of robot hands is far from human capabilities. The aim of this monograph is, through the evolution of robotics from industrial and manufacturing robotics to service and collaborative robotics, to show the evolution of the grasping function. From industrial grippers to dexterous robot hands, and the stakes inherent today to new robotic applications in open, dynamic environments where humans are likely to evolve.
Outlier-robust estimation is a fundamental problem and has been extensively investigated by statisticians and practitioners. The last few years have seen a convergence across research fields towards "algorithmic robust statistics", which focuses on developing tractable outlier-robust techniques for high-dimensional estimation problems. Despite this convergence, research efforts across fields have been mostly disconnected from one another. This monograph bridges recent work on certifiable outlier-robust estimation for geometric perception in robotics and computer vision with parallel work in robust statistics. In particular, we adapt and extend recent results on robust linear regression (applicable to the low-outlier regime with << 50% outliers) and list-decodable regression (applicable to the high-outlier regime with >> 50% outliers) to the setup commonly found in robotics and vision, where (i) variables (e.g., rotations, poses) belong to a non-convex domain, (ii) measurements are vector-valued, and (iii) the number of outliers is not known a priori. The emphasis here is on performance guarantees: rather than proposing radically new algorithms, we provide conditions on the input measurements under which modern estimation algorithms (possibly after small modifications) are guaranteed to recover an estimate close to the ground truth in the presence of outliers. These conditions are what we call an "estimation contract". Besides the proposed extensions of existing results, we believe the main contributions of this monograph are (i) to unify parallel research lines by pointing out commonalities and differences, (ii) to introduce advanced material (e.g., sum-of-squares proofs) in an accessible and self-contained presentation for the practitioner, and (iii) to point out a few immediate opportunities and open questions in outlier-robust geometric perception.
Sampling-based methods are widely adopted solutions for robot motion planning. The methods are straightforward to implement, effective in practice for many robotic systems. It is often possible to prove that they have desirable properties, such as probabilistic completeness and asymptotic optimality. Nevertheless, they still face challenges as the complexity of the underlying planning problem increases, especially under tight computation time constraints, which impact the quality of returned solutions or given inaccurate models. This has motivated machine learning to improve the computational efficiency and applicability of Sampling- Based Motion Planners (SBMPs). This survey reviews such integrative efforts and aims to provide a classification of the alternative directions that have been explored in the literature. It first discusses how learning has been used to enhance key components of SBMPs, such as node sampling, collision detection, distance or nearest neighbor computation, local planning, and termination conditions. Then, it highlights planners that use learning to adaptively select between different implementations of such primitives in response to the underlying problem’s features. It also covers emerging methods, which build complete machine learning pipelines that reflect the traditional structure of SBMPs. It also discusses how machine learning has been used to provide data-driven models of robots, which can then be used by a SBMP. Finally, it provides a comparative discussion of the advantages and disadvantages of the approaches covered, and insights on possible future directions of research. An online version of this survey can be found at: https://prx-kinodynamic.github.io
Interactive Imitation Learning (IIL) is a branch of Imitation Learning (IL) where human feedback is provided intermittently during robot execution allowing an online improvement of the robot's behavior. In recent years, IIL has increasingly started to carve out its own space as a promising data-driven alternative for solving complex robotic tasks. The advantages of IIL are its data-efficient, as the human feedback guides the robot directly towards an improved behavior, and its robustness, as the distribution mismatch between the teacher and learner trajectories is minimized by providing feedback directly over the learner's trajectories. Nevertheless, despite the opportunities that IIL presents, its terminology, structure, and applicability are not clear nor unified in the literature, slowing down its development and, therefore, the research of innovative formulations and discoveries. In this article, we attempt to facilitate research in IIL and lower entry barriers for new practitioners by providing a survey of the field that unifies and structures it. In addition, we aim to raise awareness of its potential, what has been accomplished and what are still open research questions. We organize the most relevant works in IIL in terms of human-robot interaction (i.e., types of feedback), interfaces (i.e., means of providing feedback), learning (i.e., models learned from feedback and function approximators), user experience (i.e., human perception about the learning process), applications, and benchmarks. Furthermore, we analyze similarities and differences between IIL and RL, providing a discussion on how the concepts offline, online, off-policy and on-policy learning should be transferred to IIL from the RL literature. We particularly focus on robotic applications in the real world and discuss their implications, limitations, and promising future areas of research.
No human drives a car in a vacuum; she/he must negotiate with other road users to achieve their goals in social traffic scenes. A rational human driver can interact with other road users in a socially-compatible way through implicit communications to complete their driving tasks smoothly in interaction-intensive, safety-critical environments. This paper aims to review the existing approaches and theories to help understand and rethink the interactions among human drivers toward social autonomous driving. We take this survey to seek the answers to a series of fundamental questions: 1) What is social interaction in road traffic scenes? 2) How to measure and evaluate social interaction? 3) How to model and reveal the process of social interaction? 4) How do human drivers reach an implicit agreement and negotiate smoothly in social interaction? This paper reviews various approaches to modeling and learning the social interactions between human drivers, ranging from optimization theory and graphical models to social force theory and behavioral & cognitive science. We also highlight some new directions, critical challenges, and opening questions for future research.
Recently, the robotics industry celebrated its 60-year anniversary. We have used robots for more than six decades to empower people to do things that are typically dirty, dull and/or dangerous. The industry has progressed significantly over the period from basic mechanical assist systems to fully autonomous cars, environmental monitoring and exploration of outer space. We have seen tremendous adoption of IT technology in our daily lives for a diverse set of support tasks. Through use of robots we are starting to see a new revolution, as we not only will have IT support from tablets, phones, computers but also systems that can physically interact with the world and assist with daily tasks, work, and leisure activities. The present document is a summary of the main societal opportunities identified, the associated challenges to deliver desired solutions and a presentation of efforts to be undertaken to ensure that US will continue to be a leader in robotics both in terms of research innovation, adoption of the latest technology, and adoption of appropriate policy frameworks that ensure that the technology is utilized in a responsible fashion.
Robotics and related technologies are central to the ongoing digitization and advancement of manufacturing. In recent years, a variety of strategic initiatives around the world including "Industry 4.0", introduced in Germany in 2011 have aimed to improve and connect manufacturing technologies in order to optimize production processes. In this work, we study the changing technological landscape of robotics and "internet-of-things" (IoT)-based connective technologies over the last 7-10 years in the wake of Industry 4.0. We interviewed key players within the European robotics ecosystem, including robotics manufacturers and integrators, original equipment manufacturers (OEMs), and applied industrial research institutions and synthesize our findings in this paper. We first detail the state-of-the-art robotics and IoT technologies we observed and that the companies discussed during our interviews. We then describe the processes the companies follow when deciding whether and how to integrate new technologies, the challenges they face when integrating these technologies, and some immediate future technological avenues they are exploring in robotics and IoT. Finally, based on our findings, we highlight key research directions for the robotics community that can enable improved capabilities in the context of manufacturing.
Robotics is becoming more and more ubiquitous, but the pressure to bring systems to market occasionally goes at the cost of neglecting security mechanisms during the development, deployment or while in production. As a result, contemporary robotic systems are vulnerable to diverse attack patterns, and an a posteriori hardening is at least challenging, if not impossible at all. This book aims to stipulate the inclusion of security in robotics from the earliest design phases onward and with a special focus on the cost-benefit tradeoff that can otherwise be an inhibitor for the fast development of affordable systems. We advocate quantitative methods of security management and design, covering vulnerability scoring systems tailored to robotic systems, and accounting for the highly distributed nature of robots as an interplay of potentially very many components. A powerful quantitative approach to model-based security is offered by game theory, providing a rich spectrum of techniques to optimize security against various kinds of attacks. Such a multi-perspective view on security is necessary to address the heterogeneity and complexity of robotic systems. This book is intended as an accessible starter for the theoretician and practitioner working in the field.
This publication describes how robot innovations are adopted during a disaster using the COVID-19 response, both as a natural experiment and a case study. While the reports do not provide full details to ascertain gaps in specific algorithms or specific subsystems, the size and the pervasiveness of the data permits examination of three questions, namely: how the need for a robot arises during a disaster; whether those needs are met with existing technically mature robots, adapting existing robots, or innovating new robots; and what are the major barriers to inserting robots into use during a disaster. The analysis utilizes a novel formal framework consisting of a sociotechnical work domain analysis, an extended demand analysis, and a rating of the technical maturity of each instance using the NASA Technical Readiness Assessment (TRA) ranking. Also covered are related works in modeling the adoption of robots and prior summative of the use of robots for the coronavirus pandemic, as well as a novel framework for analysis is discussed in detail. An analysis is also made of all Heritage systems, Engineering systems, and New systems. The paper concludes with findings for disaster robotics, then uses the model of adoption to make four recommendations for roboticists interested in developing and deploying technology for a disaster. This publication provides an in-depth overview and practical examples of how robots can be deployed during a disaster. As such it should prove useful for roboticists, researchers in various fields related to robotics as well as industry professionals who are responsible for saving lives and mitigating societal impacts in times of disaster.
In January 2021, the Hellenic Institute of Advanced Study (HIAS) assembled a panel including world leading roboticists from the Hellenic diaspora, who volunteered their scientific expertise to provide a vision for Robotics in Greece. This monograph, entitled “Robotics in the Artificial Intelligence (AI) era,” will hopefully trigger a dialogue towards the development of a national robotics strategy. Our vision is that Robotics in the AI era will be an essential technology of the future for the safety and security of the Hellenic nation, its environment and its citizens, for modernizing its economy towards Industry 4.0, and for inspiring and educating the next generation workforce for the challenges of the 21st century. To contribute towards making this vision a reality, after reviewing global trends in robotics and assessing the Greek robotics ecosystem, we arrived at the following key findings and recommendations: Firstly, we think that Greece should develop a national Hellenic Robotics Initiative that serves as the nation’s long-term vision and strategy across the entire Greek robotics ecosystem. Also, certain societal drivers should be key in the areas of focus. Safety and security is an area of national importance necessitating a national initiative, while agrifood, maritime and logistics provide opportunities for internationally leading innovation. We recommend the establishment of a mission-driven, government-funded, organization advancing unmanned vehicles in societal drivers of national importance, and Greece should leverage its unique geography and become a living testbed of robotics innovation turning the country into a development site for exportable technologies. In our opinion, universities should create Centers of Excellence in robotics and AI as well as consider innovation-leading research institutes such as the Italian Institute of Technology. We recommend investing in robotics education using Maker Spaces in order to prepare the workforce with 21st century skills to become Industry 4.0 innovators. Furthermore, we believe that the government should collect, measure, and analyze data on the robotics industry, robotics uses, labor shifts, and brain gain and promote awareness via a Hellenic Robotics Day. And finally, the government should regulate robot safety without stifling innovation, provide safe experimentation areas and mechanisms for certifying safety of locally developed robots. This monograph has many additional suggestions that enhance the above main recommendations. As authors, we advocate bringing the robotics ecosystem together in order to sharpen and expand these findings to an ambitious, long-term, and detailed national strategy and roadmap for robotics in the AI era.
For robots to navigate and interact more richly with the world around them, they will likely require a deeper understanding of the world in which they operate. In robotics and related research fields, the study of understanding is often referred to as semantics, which dictates what does the world "mean" to a robot, and is strongly tied to the question of how to represent that meaning. With humans and robots increasingly operating in the same world, the prospects of human-robot interaction also bring semantics and ontology of natural language into the picture. Driven by need, as well as by enablers like increasing availability of training data and computational resources, semantics is a rapidly growing research area in robotics. The field has received significant attention in the research literature to date, but most reviews and surveys have focused on particular aspects of the topic: the technical research issues regarding its use in specific robotic topics like mapping or segmentation, or its relevance to one particular application domain like autonomous driving. A new treatment is therefore required, and is also timely because so much relevant research has occurred since many of the key surveys were published. This survey therefore provides an overarching snapshot of where semantics in robotics stands today. We establish a taxonomy for semantics research in or relevant to robotics, split into four broad categories of activity, in which semantics are extracted, used, or both. Within these broad categories we survey dozens of major topics including fundamentals from the computer vision field and key robotics research areas utilizing semantics, including mapping, navigation and interaction with the world. The survey also covers key practical considerations, including enablers like increased data availability and improved computational hardware, and major application areas where...
Physical embodiment is a required component for robots that are structurally coupled with their real-world environments. However, most socially interactive robots do not need to physically interact with their environments in order to perform their tasks. When and why should embodied robots be used instead of simpler and cheaper virtual agents? This paper reviews the existing work that explores the role of physical embodiment in socially interactive robots. This class consists of robots that are not only capable of engaging in social interaction with humans, but are using primarily their social capabilities to perform their desired functions. Socially interactive robots provide entertainment, information, and/or assistance; this last category is typically encompassed by socially assistive robotics. In all cases, such robots can achieve their primary functions without performing functional physical work. To comprehensively evaluate the existing body of work on embodiment, we first review work from established related fields including psychology, philosophy, and sociology. We then systematically review 65 studies evaluating aspects of embodiment published from 2003 to 2017 in major peer-reviewed robotics publication venues. We examine relevant aspects of the selected studies, focusing on the embodiments compared, tasks evaluated, social roles of robots, and measurements. We introduce three taxonomies for the types of robot embodiment, robot social roles, and human-robot tasks. These taxonomies are used to deconstruct the design and interaction spaces of socially interactive robots and facilitate analysis and discussion of the reviewed studies. We use this newly-defined methodology to critically discuss existing works, revealing topics within embodiment research for social interaction, assistive robotics, and service robotics.
As robots and other intelligent agents move from simple environments and problems to more complex, unstructured settings, manually programming their behavior has become increasingly challenging and expensive. Often, it is easier for a teacher to demonstrate a desired behavior rather than attempt to manually engineer it. This process of learning from demonstrations, and the study of algorithms to do so, is called imitation learning. This work provides an introduction to imitation learning. It covers the underlying assumptions, approaches, and how they relate; the rich set of algorithms developed to tackle the problem; and advice on effective tools and implementation. We intend this paper to serve two audiences. First, we want to familiarize machine learning experts with the challenges of imitation learning, particularly those arising in robotics, and the interesting theoretical and practical distinctions between it and more familiar frameworks like statistical supervised learning theory and reinforcement learning. Second, we want to give roboticists and experts in applied artificial intelligence a broader appreciation for the frameworks and tools available for imitation learning.
This monograph surveys and informs the design and usage of nonverbal signals for human-robot interaction. With robots increasingly being utilized for tasks that require them to not only operate in close proximity to humans but to interact with them as well, there has been great interest in the communication challenges associated with the varying degrees of interaction in these environments. The success of such interactions depends on robots’ ability to convey information about their knowledge, intent, and actions to co-located humans. The monograph presents a comprehensive review of literature related to the generation and usage of nonverbal signals that facilitate legibility of non-humanoid robot state and behavior. To motivate the need for these signaling behaviors, it surveys literature in human communication and psychology and outlines target use cases of non-humanoid robots. Specifically, the focus is on works that provide insight into the cognitive processes that enable humans to recognize, interpret, and exploit nonverbal signals. From these use cases, information is identified that is potentially important for non-humanoid robots to signal and organize it into three categories of robot state. The monograph then presents a review of signal design techniques to illustrate how signals conveying this information can be generated and utilized. It concludes by discussing issues that must be considered during nonverbal signaling and open research areas, with a focus on informing the design and usage of generalizable nonverbal signaling behaviors for task-oriented non-humanoid robots.
