Vision-based Control of Supernumerary Robotic Limbs for Grasping Tasks

Accuracy is a crucial factor when robots interact with humans in certain industrial tasks, such as grasping tasks. Therefore, the main goal of this paper is to propose a method to improve the accuracy of supernumerary robotic limbs (SRL) for assisting humans to complete tasks. First, the kinematics of the supernumerary robotic limbs is analyzed, and the kinematics of the six degrees of freedom are analyzed to lay the theoretical foundation for the subsequent experiments. Using Kinect as an image collection device, the YOLOv5 method without candidate regions is trained with a training set to recognize object information. Additionally, a Generative Residual Convolutional Neural Network (GRCNN) method is used to predict the parameters such as position and angle of the grasping task. After coordinate transformation, location of crawl is obtained through MoveIt for trajectory planning, and then controlling the mechanical claw of SRL to perform the grasping task. Simulation results verify the effectiveness of the proposed approach.