Learning Coarsened Dynamic Graph Representations for Deformable Object Manipulation

Manipulation of deformable objects has long been a challenging task in robotics. Their high-dimensional configuration space and complex dynamics make them difficult to consider for tasks such as robotic manipulation. In this paper, we address the problem of learning efficient representations of deformable objects which lend themselves better suitable for downstream robotics tasks. In particular, we consider graph-based representations of deformable objects which arise naturally from their point-cloud representation. Through manipulation, we learn to coarsen this graph into a simpler representation which still captures the necessary dynamics of the object. Our model consists of (a) a Cluster Assignment Model which takes the initial graph and coarsens it, (b) a Coarsened Dynamics Model that approximates the dynamics of the coarsened graph and (c) a Forward Prediction Model which predicts the next state of the ground truth graph. After end-to-end training, the Cluster Assignment Model learns to build coarse representations which better capture the dynamics compared to conventional clustering methods such as K-means. We evaluate our method on three sets of experiments: rigid objects, rigid objects with pair-wise interactions and a simulated dataset of a shirt.