Unsupervised Keypoints from Pretrained Diffusion Models
CVPR 2024(2023)
摘要
Unsupervised learning of keypoints and landmarks has seen significant
progress with the help of modern neural network architectures, but performance
is yet to match the supervised counterpart, making their practicability
questionable. We leverage the emergent knowledge within text-to-image diffusion
models, towards more robust unsupervised keypoints. Our core idea is to find
text embeddings that would cause the generative model to consistently attend to
compact regions in images (i.e. keypoints). To do so, we simply optimize the
text embedding such that the cross-attention maps within the denoising network
are localized as Gaussians with small standard deviations. We validate our
performance on multiple datasets: the CelebA, CUB-200-2011, Tai-Chi-HD,
DeepFashion, and Human3.6m datasets. We achieve significantly improved
accuracy, sometimes even outperforming supervised ones, particularly for data
that is non-aligned and less curated. Our code is publicly available and can be
found through our project page: https://ubc-vision.github.io/StableKeypoints/
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要