MI-NeRF: Learning a Single Face NeRF from Multiple Identities
CoRR(2024)
摘要
In this work, we introduce a method that learns a single dynamic neural
radiance field (NeRF) from monocular talking face videos of multiple
identities. NeRFs have shown remarkable results in modeling the 4D dynamics and
appearance of human faces. However, they require per-identity optimization.
Although recent approaches have proposed techniques to reduce the training and
rendering time, increasing the number of identities can be expensive. We
introduce MI-NeRF (multi-identity NeRF), a single unified network that models
complex non-rigid facial motion for multiple identities, using only monocular
videos of arbitrary length. The core premise in our method is to learn the
non-linear interactions between identity and non-identity specific information
with a multiplicative module. By training on multiple videos simultaneously,
MI-NeRF not only reduces the total training time compared to standard
single-identity NeRFs, but also demonstrates robustness in synthesizing novel
expressions for any input identity. We present results for both facial
expression transfer and talking face video synthesis. Our method can be further
personalized for a target identity given only a short video.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要