Learning External Point-Set Context for Point Cloud Segmentation

Visual context is essential for point cloud semantic segmentation. The contextual information captures the semantic relationship between 3-D points, providing helpful hints for reasoning the category labels of points. Most current methods harness the internal context from the parts of the same object (or from the things within the same scene). In contrast, we propose the external point-set context (EPSC), allowing a richer context of external points learned across various objects/scenes to assist the segmentation task. We employ an external memory with multiple sets to store the EPSC representations learned from the training data. Each representation is a cluster feature, which captures the relationship between adjacent 3-D points for recognizing the semantic category of the center point. During the inference phase, the external memory releases the EPSC representations, providing rich and relevant context for segmenting the target point cloud. We extensively evaluate our method on Stanford Large-Scale 3-D Indoor Spaces (S3DIS), ScanNetv2, and ShapeNetPart datasets, where we achieve the result of effective improvement.