Dexterous manipulation of objects through fine control of physical contacts is essential for many important tasks of daily living. A fundamental ability underlying fine contact control is compliant control, i.e., controlling the contact forces while moving. For robots, the most widely explored approaches heavily depend on models of manipulated objects and expensive sensors to gather contact location and force information needed for real-time control. The models are difficult to obtain, and the sensors are costly, hindering personal robots' adoption in our homes and businesses. This study performs model-free reinforcement learning of a normal contact force controller on a robotic manipulation system built with a low-cost, information-poor tactile sensor. Despite the limited sensing capability, our force controller can be combined with a motion controller to enable fine contact interactions during object manipulation. Promising results are demonstrated in non-prehensile, dexterous manipulation experiments.

Narrow passages in free space pose great challenges to many sampling-based motion planners. In problems with high-dimensional configuration spaces (C-spaces), narrow passages in free space (C-free) can be hard to find and navigate and, sometimes, even counterintuitive. In this article, we present an algorithm to construct cones of available instantaneous velocities (“free velocity cones” or, briefly, “free cones”) on the boundary of C-free to facilitate finding and navigating narrow passages. This is accomplished by first developing free cones and associated measures of local C-space narrowness for a single rigid link. These results are then extended to kinematic chains (open and closed) of arbitrary degrees of freedom. It turns out that the degeneracy of the free cones dictates the existence of C-space narrow passages, and the locations and orientations of the links in the workspace relate intimately to the corresponding connected component of C-free. This observation leads us to a modified probabilistic roadmap (M-PRM) algorithm and a modified rapidly exploring random vine (M-RRV) algorithm of combining the enumeration of topological components and random sampling of the free cones. Experimental results from applying our algorithms to several challenging examples show that our new algorithms are more efficient than some variants of the PRM algorithms, and several variants of the rapidly exploring random tree (RRT) algorithms, such as RRT-CONNECT and RRV.

Recently, the robotics industry celebrated its 60-year anniversary. We have used robots for more than six decades to empower people to do things that are typically dirty, dull and/or dangerous. The industry has progressed significantly over the period from basic mechanical assist systems to fully autonomous cars, environmental monitoring and exploration of outer space. We have seen tremendous adoption of IT technology in our daily lives for a diverse set of support tasks. Through use of robots we are starting to see a new revolution, as we not only will have IT support from tablets, phones, computers but also systems that can physically interact with the world and assist with daily tasks, work, and leisure activities. The present document is a summary of the main societal opportunities identified, the associated challenges to deliver desired solutions and a presentation of efforts to be undertaken to ensure that US will continue to be a leader in robotics both in terms of research innovation, adoption of the latest technology, and adoption of appropriate policy frameworks that ensure that the technology is utilized in a responsible fashion.

The ever-changing nature of human environments presents great challenges to robot manipulation. Objects that robots must manipulate vary in shape, weight, and configuration. Important properties of the robot, such as surface friction and motor torque constants, also vary over time. Before robot manipulators can work gracefully in homes and businesses, they must be adaptive to such variations. This survey summarizes types of variations that robots may encounter in human environments and categorizes, compares, and contrasts the ways in which learning has been applied to manipulation problems through the lens of adaptability. Promising avenues for future research are proposed at the end.

This paper presents a robotic assembly methodology for the manufacturing of large segmented composite structures. The approach addresses three key steps in the assembly process: panel localization and pick-up, panel transport, and panel placement. Multiple stationary and robot-mounted cameras provide information for localization and alignment. A robot wrist-mounted force/torque sensor enables gentle but secure panel pick-up and placement. Human-assisted path planning ensures reliable collision-free motion of the robot with a large load in a tight space. A finite state machine governs the process flow and user interface. It allows process interruption and return to the previous known state in case of error condition or when secondary operations are needed. For performance verification, a high resolution motion capture system provides the ground truth reference. An experimental testbed integrating an industrial robot, vision and force sensors, and representative laminated composite panels demonstrates the feasibility of the proposed assembly process. Experimental results show sub-millimeter placement accuracy with shorter cycle times, lower contact force, and reduced panel oscillation than manual operations. This work demonstrates the versatility of sensor guided robotic assembly operation in a complex end-to-end tasks using the open source Robot Operating System (ROS) software framework.

This paper investigates the motion planning problem of planar m-link (m≥4) closed chains among point obstacles with extension to arbitrary convex 2-D obstacles. The configuration space (C-space) of closed chains is embedded into two copies of m-3 dimensional tori. Two structural sets, the C-boundaries and the C-obstacles, are analyzed based upon the C-spaces of recursively constructed lower-dimensional closed chains. They contain essential structural information about the connectivity of the collision-free portion (C-free) of the C-space. By approximating each workspace obstacle by a set of points on the boundary after dilation, its corresponding C-obstacle is guaranteed to be covered by the C-obstacle of the convex hull of the point set. This permits a resolution-complete roadmap algorithm that puts specific bias for sampling the structural sets. Several benchmark examples are presented that compare the performance between our algorithm and the traditional algorithms. Animation videos and source codes are also provided which demonstrate the effectiveness of our method for closed chains of up to 20 links.

The last five years marked a surge in interest for and use of smart robots, which operate in dynamic and unstructured environments and might interact with humans. We posit that well-validated computer simulation can provide a virtual proving ground that in many cases is instrumental in understanding safely, faster, at lower costs, and more thoroughly how the robots of the future should be designed and controlled for safe operation and improved performance. Against this backdrop, we discuss how simulation can help in robotics, barriers that currently prevent its broad adoption, and potential steps that can eliminate some of these barriers. The points and recommendations made concern the following simulation-in-robotics aspects: simulation of the dynamics of the robot; simulation of the virtual world; simulation of the sensing of this virtual world; simulation of the interaction between the human and the robot; and, in less depth, simulation of the communication between robots. This Perspectives contribution summarizes the points of view that coalesced during a 2018 National Science Foundation/Department of Defense/National Institute for Standards and Technology workshop dedicated to the topic at hand. The meeting brought together participants from a range of organizations, disciplines, and application fields, with expertise at the intersection of robotics, machine learning, and physics-based simulation.

Over the past several decades, as affordable computational power has increased, simulation has become increasingly important in robot analysis, planning, and control. Smooth robot dynamics can be simulated efficiently and accurately, and therefore, readily used in model-based control schemes. However, some of the most difficult and important problems in robotics, such as running, grasping, and parts assembly, involve intermittent frictional contacts, which introduce extreme nonlinearities into the dynamics. Numerous approaches have been developed to simulate contact dynamics. Even though real bodies are not rigid, idealized rigid contact models have been used widely and productively for decades. However, the resulting nonsmooth dynamics can be computationally difficult to solve or use in model-based planning and control, motivating researchers to propose various relaxations of the idealized contact models. The varied origins and formulations of these approaches can obscure their similarities and differences. In this letter, we identify and explain differences between four contact models. We present the models in the context of one solver that is applicable to all of them, namely Projected Gauss-Seidel, in order to highlight their common structure and to avoid confounding their comparison with differences in the solution methods. Simulation results from sliding, wedging, grasping, and stacking experiments illustrate consequences of the differences. The results can help inform roboticists when comparing and selecting models for their specific applications.

This paper investigates the motion planning problem of planar m-link closed-chains with (point or convex non-point) obstacles. The configuration space (C-space) of closed-chains is embedded into two copies of m − 3 dimensional torii. The boundary varieties (Cboundaries) of the C-space are sets of configurations where a pair of links overlaps. They not only determine the portion of the aforementioned torii on which the loop can be closed, but also provide necessary transitional configurations from one torus to the other. By dilating each workspace obstacle and sampling the resulting boundaries by a set of points (called dilated point obstacle set), the original configuration space obstacles (C-obstacles) are contained in the union of the semi-algebraic sets cut out by the C-obstacles of the dilated point obstacle set (called dilated C-obstacles). It turns out that the C-boundaries and the dilated Cobstacles contain essential structural information about the connectivity of the collision-free portion of C-space (C-free), which allows us to propose an explicit version of exact cell decomposition and roadmap algorithm specifically for planar closed-chains, along with an efficient sampling-based roadmap algorithm. Simulation results along with animation videos and source codes are provided which demonstrate the effectiveness of our method. KewordsPath planning, closed chains, boundary variety, C-space, C-obstacle, C-free, sampling algorithm.

Geometric models are crucial for many robotics applications. Current robotic 3D reconstruction systems only focus on specific reconstruction goals which make them hard to adapt to different tasks. In this paper we present a next-best-view framework which allows robots to construct a geometric model incrementally through consecutive sensing actions. Instead of limiting the type and total number of sensors, in each sensing step we evaluate actions from all available sensors and pick the best to execute. Our framework is more comprehensive since the model building process can be designed to best accomplish different tasks. The system has been demonstrated in two experiments on 3D reconstruction and weld seam inspection, yielding promising results.

Due to the limitations of the robotic sensors, during a robotic manipulation task, the acquisition of the object's state can be unreliable and noisy. Combining an accurate model of multi-body dynamic system with Bayesian filtering methods has been shown to be able to filter out noise from the object's observed states. However, efficiency of these filtering methods suffers from samples that violate the physical constraints, e.g., no penetration constraint. In this paper, we propose a Rao-Blackwellized Particle Filter (RBPF) that samples the contact states and updates the object's poses using Kalman filters. This RBPF also enforces the physical constraints on the samples by solving a quadratic programming problem. By comparing our method with methods that does not consider physical constraints, we show that our proposed RBPF is not only able to estimate the object's states, e.g., poses, more accurately but also able to infer unobserved states, e.g., velocities, with higher precision.

We present a framework for object recognition using robotic skins with embedded arrays of tactile sensing elements. Our approach is based on theoretical foundations in compressed sensing and compressed learning. In our framework, tactile data is compressed during acquisition, potentially in-hardware, and we perform recognition directly on the compressed data. This dimensionality reduction allows for accurate recognition with a small number of training samples, reducing the time and computational effort needed to train the classifier. In addition, for tasks where the full-resolution tactile array signal is needed, it can be recovered efficiently from the compressed signal. We evaluate our method using data generated from a tactile array simulator. We also demonstrate the effectiveness of our framework in recognizing surface roughness using data from a physical system. Evaluation results show our approach achieves high recognition accuracy, even with a compression ratio of 64:1.

We present BubbleTouch, an open source quasi-static simulator for robotic tactile skins. BubbleTouch can be used to simulate contact with a robot's tactile skin patches as it interacts with humans and objects. The simulator creates detailed traces of contact forces that can be used in experiments in tactile contact activities. We summarize the design of BubbleTouch and highlight our recent work that uses BubbleTouch for experiments with tactile object recognition.

The potential of large tactile arrays to improve robot perception for safe operation in human-dominated environments and of high-resolution tactile arrays to enable human-level dexterous manipulation is well accepted. However, the increase in the number of tactile sensing elements introduces challenges including wiring complexity, data acquisition, and data processing. To help address these challenges, we develop a tactile sensing technique based on compressed sensing. Compressed sensing simultaneously performs data sampling and compression with recovery guarantees and has been successfully applied in computer vision. We use compressed sensing techniques for tactile data acquisition to reduce hardware complexity and data transmission, while allowing fast, accurate reconstruction of the full-resolution signal. For our simulated test array of 4096 taxels, we achieve reconstruction quality equivalent to measuring all taxel signals independently (the full signal) from just 1024 measurements (the compressed signal) at a rate over 100Hz. We then apply tactile compressed sensing to the problem of object classification. Specifically, we perform object classification on the compressed tactile data based on a method called compressed learning. We obtain up to 98% classification accuracy, even with a compression ratio of 64:1.

We present a formulation of nonpenetration constraint between pairs of polytopes which accounts for all possible combinations of active contact between geometric features. This is the first formulation that exactly models the body geometries near points of potential contact, preventing interpenetration while not overconstraining body motions. Unlike many popular methods, ours does not wait for penetrations to occur as a way to identify which contact constraints to enforce. Nor do we overconstrain by representing the free space between pairs of bodies as convex, when it is in fact nonconvex. Instead, each contact constraint incorporates all feasible potential contacts in a way that represents the true geometry of the bodies. This ensures penetration-free, physically correct configurations at the end of each time step while allowing bodies to accurately traverse the free space surrounding other bodies. The new formulation improves accuracy, dramatically reduces the need for ad hoc corrections of constraint violations, and avoids many of the inevitable instabilities consequent of other contact models. Although the dynamics problem at each time step is larger, the inherent stability of our method means that much larger time steps can be taken without loss of physical fidelity. As will be seen, the results obtained with our method demonstrate the effective elimination of interpenetration, and as a result, correction-induced instabilities, in multibody simulations.

Tactile robotic skins consist of thousands to millions of tiny sensing elements that cover the surface of a robot. The data loads, timing requirements, and hardware constraints in tactile skins make real-time tactile data acquisition challenging. In previous work, we developed a compressed sensing tactile data acquisition system to address these challenges. In this work, we propose a method to adaptively select a basis for each compressed tactile signal to improve the overall signal reconstruction accuracy. A classifier is trained offline using a set of candidate bases and training signals. When each new compressed signal is acquired, the classifier identifies which basis to use in reconstructing the full tactile signal from the compressed one. We evaluate our method using data generated by our tactile skin simulation system. Our evaluations show that our adaptive basis selection method consistently outperforms approaches that use a single basis for signal reconstruction.

1.1 Robotic manipulations in the DARPA Robotics Challenge........ 1 1.2 A robot picks up a package from a shelf in the Amazon Picking Challenge [2].................................... 3 1.3 A diagram of our proposed solution..................... 5 3.1 Coordinate frames for the ith contact between two colliding bodies. Λ ji and Λ ki are the contact frames attached to the contact points and Ψ in is the signed distance in the contact normal direction between the two contact points of the ith contact. n i, t i and ô i are the contact normal unit vector and the contact tangent vectors respectively.......... 21

This chapter discusses numerous topics related to simulating multi-rigid bodies undergoing contact, including rigid and pseudo-rigid models of contact, complementarity problems, the Coulomb friction model, rigid body impacts, coordinate selection for rigid bodies and multibodies, integrating the equations of motion, constructing Jacobian matrices for unilateral and bilateral constraints and time-stepping and event-driven simulation methods, and determining contact data from geometric representations of rigid bodies. The material is approached starting from foundational models and moves toward practical implementation. The chapter concludes with further reading, which includes both current research directions and open problems.

Whole body tactile perception via tactile skins offers large benefits for robots in unstructured environments. To fully realize this benefit, tactile systems must support real-time data acquisition over a massive number of tactile sensor elements. We present a novel approach for scalable tactile data acquisition using compressed sensing. We first demonstrate that the tactile data is amenable to compressed sensing techniques. We then develop a solution for fast data sampling, compression, and reconstruction that is suited for tactile system hardware and has potential for reducing the wiring complexity. Finally, we evaluate the performance of our technique on simulated tactile sensor networks. Our evaluations show that compressed sensing, with a compression ratio of 3 to 1, can achieve higher signal acquisition accuracy than full data acquisition of noisy sensor data.

The potential of large tactile arrays to improve robot perception for safe operation in human-dominated environments and of high-resolution tactile arrays to enable human-level dexterous manipulation is well accepted. However, the increase in the number of tactile sensing elements introduces challenges including wiring complexity, power consumption, and data processing. To help address these challenges, we previously developed a tactile sensing technique based compressed sensing that reduces hardware complexity and data transmission, while allowing accurate reconstruction of the full-resolution signal. In this paper, we apply tactile compressed sensing to the problem of object classification. Specifically, we perform object classification on the compressed tactile data. We evaluate our method using BubbleTouch, our tactile array simulator. Our results show our approach achieves high classification accuracy, even with compression factors up to 64.