Whole-body manipulation is a powerful yet underexplored approach that enables robots to interact with large, heavy, or awkward objects using more than just their end-effectors. Soft robots, with their inherent passive compliance, are particularly well-suited for such contact-rich manipulation tasks, but their uncertainties in kinematics and dynamics pose significant challenges for simulation and control. In this work, we address this challenge with a simulation that can run up to 350x real time on a single thread in MuJoCo and provide a detailed analysis of the critical tradeoffs between speed and accuracy for this simulation. Using this framework, we demonstrate a successful zero-shot sim-to-real transfer of a learned whole-body manipulation policy, achieving an 88

Achieving seamless human-robot collaboration requires a deeper understanding of how agents manage and communicate forces during shared tasks. Force interactions during collaborative manipulation are inherently complex, especially when considering how they evolve over time. To address this complexity, we propose a taxonomy of decomposed force and torque components, providing a structured framework for examining haptic communication and informing the development of robots capable of performing meaningful collaborative manipulation tasks with human partners. We propose a standardized terminology for force decomposition and classification, bridging the varied language in previous literature in the field, and conduct a review of physical human-human interaction and haptic communication. The proposed taxonomy allows for a more effective and nuanced discussion of important force combinations that we expect to occur during collaborative manipulation (between human-human or human-robot teams). We also include example scenarios to illustrate the value of the proposed taxonomy in describing interactions between agents.

Large-scale soft robots have the capability and potential to perform highly dynamic tasks such as hammering a nail into a board, throwing items long distances, or manipulating objects in cluttered environments. This is due to their joints being underdamped and their ability to store potential energy. The soft robots presented in this article are pneumatically actuated and thus have the ability to perform these tasks without the need for large motors or gear trains. However, getting soft robots to perform highly dynamic tasks requires controllers that can track highly dynamic trajectories to complete those tasks. For soft robots, this is a difficult problem to solve due to the uncertainty in their shape and their complicated dynamics and kinematics. This article presents a formulation of a model reference adaptive controller (MRAC) that causes a three-link soft robot arm to behave like a highly dynamic 2nd-order critically damped system. Using the dynamics of a 2nd-order system, we also present a method to generate joint trajectories for throwing a ball to a desired point in Cartesian space. We demonstrate the viability of our joint-level controller in simulation and on hardware with a reported maximum root mean square error of 0.0872 radians between a reference and executed trajectory. We also demonstrate that our combined MRAC controller and trajectory generator can, on average, throw a ball to within 25-28% of a desired landing location for a throwing distance of between 1.5 and 2 m on real hardware.

Scaling tactile sensing for robust whole-body manipulation is a significant challenge, often limited by wiring complexity, data throughput, and system reliability. This paper presents a complete architecture designed to overcome these barriers. Our approach pairs open-source, fabric-based sensors with custom readout electronics that reduce signal crosstalk to less than 3.3

Human teams intuitively and effectively collaborate to move large, heavy, or unwieldy objects. However, understanding of this interaction in literature is limited. This is especially problematic given our goal to enable human-robot teams to work together. Therefore, to better understand how human teams work together to eventually enable intuitive human-robot interaction, in this paper we examine four sub-components of collaborative manipulation (co-manipulation), using motion and haptics. We define co-manipulation as a group of two or more agents collaboratively moving an object. We present a study that uses a large object for co-manipulation as we vary the number of participants (two or three) and the roles of the participants (leaders or followers), and the degrees of freedom necessary to complete the defined motion for the object. In analyzing the results, we focus on four key components related to motion and haptics. Specifically, we first define and examine a static or rest state to demonstrate a method of detecting transitions between the static state and an active state, where one or more agents are moving toward an intended goal. Secondly, we analyze a variety of signals (e.g. force, acceleration, etc.) during movements in each of the six rigid-body degrees of freedom of the co-manipulated object. This data allows us to identify the best signals that correlate with the desired motion of the team. Third, we examine the completion percentage of each task. The completion percentage for each task can be used to determine which motion objectives can be communicated via haptic feedback. Finally, we define a metric to determine if participants divide two degree-of-freedom tasks into separate degrees of freedom or if they take the most direct path. These four components contribute to the necessary groundwork for advancing intuitive human-robot interaction.

Soft robotic actuators and their inherent compliance can simplify the design of controllers when operating in contact-rich environments. With such structures we can accomplish high-impact, dynamic, and contact-rich tasks that would be difficult using conventional rigid robots which might either break the robot or the object without careful modeling and design of high bandwidth controllers. In order to explore the benefits of structural passive compliance and exploit them effectively, we present a prototype robotic torso named Baloo, designed with a hybrid rigid-soft methodology, incorporating both adaptability from soft components and strength from rigid components. Baloo consists of two meter-long, pneumatically-driven soft robot arms mounted on a rigid torso and driven vertically by a linear actuator. We explore some challenges inherent in controlling this type of robot and build on previous work with rigid robots to develop a joint-level neural-network adaptive controller to enable high performance tracking of highly nonlinear, time-varying soft robot dynamics. We also demonstrate a promising use case for the platform with several hardware experiments performing whole-body manipulation with large, heavy, and unwieldy objects. A video of our results can be viewed at https://youtu.be/eTUvBEVGKXY.

Despite the existence of robots that can lift heavy loads, robots that can help people move heavy objects are not readily available. This paper makes progress towards effective human-robot co-manipulation by studying 30 human-human dyads that collaboratively manipulated an object weighing 27 kg without being co-located (i.e. participants were at either end of the extended object). Participants maneuvered around different obstacles with the object while exhibiting one of four modi–the manner or objective with which a team moves an object together–at any given time. Using force and motion signals to classify modus or behavior was the primary objective of this work. Our results showed that two of the originally proposed modi were very similar, such that one could effectively be removed while still spanning the space of common behaviors during our co-manipulation tasks. The three modi used in classification were quickly, smoothly and avoiding obstacles . Using a deep convolutional neural network (CNN), we classified three modi with up to 89% accuracy from a validation set. The capability to detect or classify modus during co-manipulation has the potential to greatly improve human-robot performance by helping to define appropriate robot behavior or controller parameters depending on the objective or modus of the team.

Aggressive and accurate control of complex dynamical systems, such as soft robots, is especially challenging due to the difficulty of obtaining an accurate and tractable model for realtime control. Learned dynamic models are incredibly useful because they do not require derivation of an analytical model, they can represent complex, nonlinear behavior directly from data, and they can be evaluated quickly on graphics-processing units (GPUs). In this paper, we present an open-source Python library to further current research in model-based control of soft robot systems. Our library for Modeling of Learned Dynamics (MoLDy), is designed to generate learned forward models of complex systems through a data-driven approach to hyperparameter optimization and learned model training. Included in the MoLDy library, we present an open-source version of NEMPC (Nonlinear Evo-lutionary Model Predictive Control), a previously published control algorithm validated on soft robots. We demonstrate the ability of MoLDy and NEMPC to accurately perform model-based control on a physical pneumatic continuum joint. We also present a benchmarking study on the effect of the loss metric used in model training on control performance. The results of this paper serve to guide other researchers in creating learned dynamic models of novel systems and using them in closed-loop control tasks.

Human teams are able to easily perform collaborative manipulation tasks. However, for a robot and human to simultaneously manipulate an extended object is a difficult task using existing methods from the literature. Our approach in this paper is to use data from human-human dyad experiments to determine motion intent which we use for a physical human-robot co-manipulation task. We first present and analyze data from human-human dyads performing co-manipulation tasks. We show that our human-human dyad data has interesting trends including that interaction forces are non-negligible compared to the force required to accelerate an object and that the beginning of a lateral movement is characterized by distinct torque triggers from the leader of the dyad. We also examine different metrics to quantify performance of different dyads. We also develop a deep neural network based on motion data from human-human trials to predict human intent based on past motion. We then show how force and motion data can be used as a basis for robot control in a human-robot dyad. Finally, we compare the performance of two controllers for human-robot co-manipulation to human-human dyad performance.

Multi-agent human-robot co-manipulation is a poorly understood process with many inputs that potentially affect agent behavior. This paper explores one such input known as interaction force. Interaction force is potentially a primary component in communication that occurs during co-manipulation. There are, however, many different perspectives and definitions of interaction force in the literature. Therefore, a decomposition of interaction force is proposed that provides a consistent way of ascertaining the state of an agent relative to the group for multi-agent co-manipulation. This proposed method extends a current definition from one to four degrees of freedom, does not rely on a predefined object path, and is independent of the number of agents acting on the system and their locations and input wrenches (forces and torques). In addition, all of the necessary measures can be obtained by a self-contained robotic system, allowing for a more flexible and adaptive approach for future co-manipulation robot controllers.

In this paper, we present a modular pressure control system called PneuDrive that can be used for large-scale, pneumatically-actuated soft robots. The design is particularly suited for situations which require distributed pressure control and high flow rates. Up to four embedded pressure control modules can be daisy-chained together as peripherals on a robust RS-485 bus, enabling closed-loop control of up to 16 valves with pressures ranging from 0-100 psig (0-689 kPa) over distances of more than 10 meters. The system is configured as a C++ ROS node by default. However, independent of ROS, we provide a Python interface with a scripting API for added flexibility. We demonstrate our implementation of PneuDrive through various trajectory tracking experiments for a three-joint, continuum soft robot with 12 different pressure inputs. Finally, we present a modeling toolkit with implementations of three dynamic actuation models, all suitable for real-time simulation and control. We demonstrate the use of this toolkit in customizing each model with real-world data and evaluating the performance of each model. The results serve as a reference guide for choosing between several actuation models in a principled manner. A video summarizing our results can be found here: https://bit.ly/3QkrEqO.

Soft manipulators, renowned for their compliance and adaptability, hold great promise in their ability to engage safely and effectively with intricate environments and delicate objects. Nonetheless, controlling these soft systems presents distinctive hurdles owing to their nonlinear behavior and complicated dynamics. Learning-based controllers for continuum soft manipulators offer a viable alternative to model-based approaches that may struggle to account for uncertainties and variability in soft materials, limiting their effectiveness in real-world scenarios. Learning-based controllers can be trained through experience, exploiting various forward models that differ in physical assumptions, accuracy, and computational cost. In this article, the key features of popular forward models, including geometrical, pseudo-rigid, continuum mechanical, or learned, are first summarized. Then, a unique characterization of learning-based policies, emphasizing the impact of forward models on the control problem and how the state of the art evolves, is offered. This leads to the presented perspectives outlining current challenges and future research trends for machine-learning applications within soft robotics.

This paper details a reliable control method for highly nonlinear dynamical systems such as soft robots. We call this method model evolutionary gain-based predictive control or MEGa-PC. The method uses an evolutionary algorithm to optimize a set of controller gains via model predictive control. We demonstrate the performance of MEGa-PC in simulation for a single-link inverted pendulum and a three- link inverted pendulum, and on physical hardware for a three- joint continuum soft robot arm with six degrees of freedom. MEGa-PC is compared to prior work that used Nonlinear Evolutionary Model Predictive Control or NEMPC. The new method performs similarly to NEMPC in terms of accumulated cost over the entire trajectory, however, MEGa-PC generalizes better to real-world applications where safety is paramount, the dynamic model is uncertain, the system has significant latency, and where the previous sampling-based method (NEMPC) resulted in significant steady-state error due to model inaccuracy.

Soft robots offer more flexibility, compliance, and adaptability than traditional rigid robots. They are also typically lighter and cheaper to manufacture. However, their use in real-world applications is limited due to modeling challenges and difficulties in integrating effective proprioceptive sensors. Large-scale soft robots (≈ two meters in length) have greater modeling complexity due to increased inertia and related effects of gravity. Common efforts to ease these modeling difficulties such as assuming simple kinematic and dynamics models also limit the general capabilities of soft robots and are not applicable in tasks requiring fast, dynamic motion like throwing and hammering. To overcome these challenges, we propose a data-efficient Bayesian optimization-based approach for learning control policies for dynamic tasks on a large-scale soft robot. Our approach optimizes the task objective function directly from commanded pressures, without requiring approximate kinematics or dynamics as an intermediate step. We demonstrate the effectiveness of our approach through both simulated and real-world experiments.

In this paper we present a novel kinematic representation of a soft continuum robot to enable full shape estimation using a purely geometric solution. The kinematic representation involves using length varying piecewise constant curvature segments to describe the deformed shape of the robot. Based on this kinematic representation, we can use overlapping length sensors to estimate the shape of continuously deformable bodies without prior knowledge of the current loading conditions. We show an implementation that assumes one change in curvature along the length of a joint, using string potentiometers as an arc length sensor, and an orientation measurement from the tip of the continuum joint. For 56 randomized joint configurations, we estimate the shape of a 250 mm long continually deformable robot with less then 2.5 mm of average error. The average error is reported for each of the 10 different equally spaced points along the length, demonstrating the ability to accurately represent the full shape of the soft robot.

Potential applications for large-scale soft robots include interacting with humans while carrying a heavy load, navigating in clutter, executing impact tasks like hammering a nail into a wall, and so much more. Because of their compliance and lack of fragile gear trains, soft robots are uniquely suited to these tasks. However, we expect that path planning may be more constrained by soft robot kinematics and dynamics than traditional rigid robots. Generating dynamically feasible trajectories for soft robots (especially large-scale soft robots with higher payloads) is critical to the success of low-level controllers tracking reference trajectories. This paper introduces an optimization method to generate task and joint space trajectories for soft robots that satisfy kinematic and dynamic constraints which are unique to large-scale soft robots. The method presented in this paper is an offline trajectory generator that is then fed to a low-level PID joint angle controller. We conduct two experiments to validate this method on a continuum pneumatic soft robot of length 1.19 meters in both simulation and on hardware. We show that this is a viable method of planning trajectories for soft robots with a reported median magnitude of error of 0.032 meters between the planned and actual end effector trajectories.

Legged robots have the potential to cover terrain not accessible to wheel-based robots and vehicles. This makes them better suited to perform tasks such as search and rescue in real-world unstructured environments. In addition, pneumatically-actuated, compliant robots may be more suited than their rigid counterparts to real-world unstructured environments with humans where unintentional contact or impact may occur. In this work, we define design metrics for legged robots that evaluate their ability to traverse unstructured terrain, carry payloads, find stable footholds, and move in desired directions. These metrics are demonstrated and validated in a multi-objective design optimization of 10 variables for a 16 degree of freedom, pneumatically actuated, continuum joint quadruped. We also present and validate approximations to preserve numerical tractability for any similar high degree of freedom optimization problem. Finally, we show that the design trends uncovered by our optimization hold in two hardware experiments using robot legs with continuum joints that are built based on the optimization results.

In this paper, we propose a new tractable ordinary differential equation formulation for dynamic simulation of fabric- reinforced inflatable soft robots. The method performs a lumped-parameter discretization of the continuum robot into discrete discs (inertia), spring elements, and threads (representing the inextensible fabric reinforcement). Using the repetition in the structure of the Lagrangian formulation of the dynamic equations of motion, a method is developed that outputs machine- readable analytical expressions for the equations of motion. The method does not require symbolic computation of derivatives. The recursive nature allows us to scale the model to an arbitrary number $N$ discs, and can represent buckling, twisting, and pleating that is commonly seen in very soft robots. The expressions generated were validated against manually-derived equations of motion for the two-disc case using both Lagrangian and Newton-Euler means. A simulation environment which parses and evaluates the analytical expressions generated at run-time was used to numerically integrate and predict the response of a four-disc example robot. Trajectories observed varied smoothly and plausibly predicted the behavior envisioned in robots like these.

Because of the complex nature of soft robots, formulating dynamic models that are simple, efficient, and sufficiently accurate for simulation or control is a difficult task. This paper introduces an algorithm based on a recursive Newton-Euler (RNE) approach that enables an accurate and tractable lumped parameter dynamic model. This model scales linearly in computational complexity with the number of discrete segments. We validate this model by comparing it to actual hardware data from a three-joint continuum soft robot (with six degrees of freedom represented in a constant curvature kinematic model). The results show that this RNE-based model can be computed faster than real-time. We also show that with minimal system identification, a simulation performed using the dynamic model matches the real robot data with a median error of 3.15 degrees.

In this paper we present a novel approach to accomplishing soft robot configuration estimation and control using RGB-D cameras and SLAM-based methods. By placing cameras on the unactuated sections of our large-scale (approximately 2 meters long) pneumatic soft robot, we can map an environment and then estimate the orientation of the robot links using landmark-based localization. Using the orientations of each camera we can solve for the joint configurations between them. We first show that this method works for a traditional rigid robot (Baxter) where we can compare against the ground truth encoder values. For Baxter, the median joint angle error was on the order of 1-2 ◦ . We then show that the SLAM-based method provides estimates for soft robot configuration that are within 1 ◦ when compared to our past methods of using a HTC Vive Tracker. While HTC Vive Trackers and commonly used motion capture systems require externally mounted sensors placed in the robot’s environment, the SLAM-based estimation method presented here works in any visually feature-rich environment. Finally we show that this method of estimation is effective for closed-loop control of soft robots by controlling our large-scale soft robot through a series of joint configurations.