OmniSeg3D: Omniversal 3D Segmentation via Hierarchical Contrastive Learning
CVPR 2024(2023)
摘要
Towards holistic understanding of 3D scenes, a general 3D segmentation method
is needed that can segment diverse objects without restrictions on object
quantity or categories, while also reflecting the inherent hierarchical
structure. To achieve this, we propose OmniSeg3D, an omniversal segmentation
method aims for segmenting anything in 3D all at once. The key insight is to
lift multi-view inconsistent 2D segmentations into a consistent 3D feature
field through a hierarchical contrastive learning framework, which is
accomplished by two steps. Firstly, we design a novel hierarchical
representation based on category-agnostic 2D segmentations to model the
multi-level relationship among pixels. Secondly, image features rendered from
the 3D feature field are clustered at different levels, which can be further
drawn closer or pushed apart according to the hierarchical relationship between
different levels. In tackling the challenges posed by inconsistent 2D
segmentations, this framework yields a global consistent 3D feature field,
which further enables hierarchical segmentation, multi-object selection, and
global discretization. Extensive experiments demonstrate the effectiveness of
our method on high-quality 3D segmentation and accurate hierarchical structure
understanding. A graphical user interface further facilitates flexible
interaction for omniversal 3D segmentation.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要