Material Classification Using Visio-Tactile Sensor for Haptic Feedback Generation

The flexibility, versatility and enhanced perception of visio-tactile sensors could be beneficial for advanced robotic systems and other applications requiring precise haptic feedback. In this paper, we present a comprehensive framework that combines material classification and haptic feedback through the use of GelSight sensors. The study includes the creation of a diverse material dataset, consisting of 13 material classes of 42 distinct indoor and outdoor items, each item with multiple video samples captured over different regions and pressing conditions by human-held GelSight mini sensors. We introduce a method for detecting pressing events from recorded video samples and extracting key frames that capture important material features. We employ both traditional and deep learning-based feature extraction techniques to model material characteristics. These features are then used to classify materials with high accuracy through supervised learning methods using different image resolutions. For the traditional approach, Histogram of Oriented Gradients (HOG) feature descriptor combined with SVM gives 95.41