Modal-graph 3D shape servoing of deformable objects with raw point clouds

Deformable object manipulation (DOM) with point clouds has great potential as nonrigid 3D shapes can be measured without detecting and tracking image features. However, robotic shape control of deformable objects with point clouds is challenging due to: the unknown point correspondences and the noisy partial observability of raw point clouds; the modeling difficulties of the relationship between point clouds and robot motions. To tackle these challenges, this paper introduces a novel modal-graph framework for the model-free shape servoing of deformable objects with raw point clouds. Unlike the existing works studying the object's geometry structure, we propose a modal graph to describe the low-frequency deformation structure of the DOM system, which is robust to the measurement irregularities. The modal graph enables us to directly extract low-dimensional deformation features from raw point clouds without extra processing of registrations, refinements, and occlusion removal. It also preserves the spatial structure of the DOM system to inverse the feature changes into robot motions. Moreover, as the framework is built with unknown physical and geometric object models, we design an adaptive robust controller to deform the object toward the desired shape while tackling the modeling uncertainties, noises, and disturbances online. The system is proved to be input-to-state stable (ISS) using Lyapunov-based methods. Extensive experiments are conducted to validate our method using linear, planar, tubular, and volumetric objects under different settings.