In-N-Out: Faithful 3D GAN Inversion with Volumetric Decomposition for Face Editing
arxiv(2023)
摘要
3D-aware GANs offer new capabilities for view synthesis while preserving the
editing functionalities of their 2D counterparts. GAN inversion is a crucial
step that seeks the latent code to reconstruct input images or videos,
subsequently enabling diverse editing tasks through manipulation of this latent
code. However, a model pre-trained on a particular dataset (e.g., FFHQ) often
has difficulty reconstructing images with out-of-distribution (OOD) objects
such as faces with heavy make-up or occluding objects. We address this issue by
explicitly modeling OOD objects from the input in 3D-aware GANs. Our core idea
is to represent the image using two individual neural radiance fields: one for
the in-distribution content and the other for the out-of-distribution object.
The final reconstruction is achieved by optimizing the composition of these two
radiance fields with carefully designed regularization. We demonstrate that our
explicit decomposition alleviates the inherent trade-off between reconstruction
fidelity and editability. We evaluate reconstruction accuracy and editability
of our method on challenging real face images and videos and showcase favorable
results against other baselines.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要