FreeZe: Training-free zero-shot 6D pose estimation with geometric and vision foundation models
arxiv(2023)
摘要
Estimating the 6D pose of objects unseen during training is highly desirable
yet challenging. Zero-shot object 6D pose estimation methods address this
challenge by leveraging additional task-specific supervision provided by
large-scale, photo-realistic synthetic datasets. However, their performance
heavily depends on the quality and diversity of rendered data and they require
extensive training. In this work, we show how to tackle the same task but
without training on specific data. We propose FreeZe, a novel solution that
harnesses the capabilities of pre-trained geometric and vision foundation
models. FreeZe leverages 3D geometric descriptors learned from unrelated 3D
point clouds and 2D visual features learned from web-scale 2D images to
generate discriminative 3D point-level descriptors. We then estimate the 6D
pose of unseen objects by 3D registration based on RANSAC. We also introduce a
novel algorithm to solve ambiguous cases due to geometrically symmetric objects
that is based on visual features. We comprehensively evaluate FreeZe across the
seven core datasets of the BOP Benchmark, which include over a hundred 3D
objects and 20,000 images captured in various scenarios. FreeZe consistently
outperforms all state-of-the-art approaches, including competitors extensively
trained on synthetic 6D pose estimation data. Code will be publicly available
at https://andreacaraffa.github.io/freeze.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要