Learning to predict grasping interaction with geometry-aware 3 D representations

Learning object interaction is an essential problem in artificial intelligence that involves perception, motion planning and control. In this paper we present our results on the problem of grasp prediction from a single-view RGBD as well as the camera view matrix. We show that learning geometry is at the heart of this type of interaction and propose a geometry-aware grasping procedure with which first we predict a 3D volumetric representation of an object from an image, and then use this together with the image and a grasp pose proposal to predict the grasp success/failure (see Figure 1). We compare our results with a vanilla approach where outcome is a high-order mapping from image and action [3, 4, 2, 1] (see Figure 2a and b).