Practical 3-D Object Detection Using Category and Instance-Level Appearance Models

Effective robotic interaction with household objects requires the ability to recognize both object instances and object categories. The former are often characterized by locally discriminative texture cues (e.g., instances with prominent brand names and logos), and the latter by salient global shape properties (plates, bowls, pots). We describe experiments with both types of cues, combining a template-and-deformable-parts detector to capture overall shape properties with a local feature Naive-Bayes nearest neighbor model to capture local texture properties. We base our implementation on the recently introduced Kinect sensor, which provides reliable depth estimates of indoor scenes. Depth cues provide segmentation and size constraints to our method. Depth affinity is used to modify the appearance term in a segmentation-based proposal step, and size priors are imposed on object classes to prune false positives. We address the complexity of scanning window HOG search using multi-class pruning schemes, first applying a generic object detection scheme to prune unlikely windows, and then focusing only on the most likely class per remaining window. Our method is able to handle relatively cluttered scenes involving multiple objects with varying levels of surface texture, and can efficiently employ multi-class scanning window search.