Animatable and Relightable Gaussians for High-fidelity Human Avatar Modeling
arxiv(2023)
摘要
Modeling animatable human avatars from RGB videos is a long-standing and
challenging problem. Recent works usually adopt MLP-based neural radiance
fields (NeRF) to represent 3D humans, but it remains difficult for pure MLPs to
regress pose-dependent garment details. To this end, we introduce Animatable
Gaussians, a new avatar representation that leverages powerful 2D CNNs and 3D
Gaussian splatting to create high-fidelity avatars. To associate 3D Gaussians
with the animatable avatar, we learn a parametric template from the input
videos, and then parameterize the template on two front back canonical
Gaussian maps where each pixel represents a 3D Gaussian. The learned template
is adaptive to the wearing garments for modeling looser clothes like dresses.
Such template-guided 2D parameterization enables us to employ a powerful
StyleGAN-based CNN to learn the pose-dependent Gaussian maps for modeling
detailed dynamic appearances. Furthermore, we introduce a pose projection
strategy for better generalization given novel poses. To tackle the realistic
relighting of animatable avatars, we introduce physically-based rendering into
the avatar representation for decomposing avatar materials and environment
illumination. Overall, our method can create lifelike avatars with dynamic,
realistic, generalized and relightable appearances. Experiments show that our
method outperforms other state-of-the-art approaches.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要