Chrome Extension
WeChat Mini Program
Use on ChatGLM

Language-driven Object Fusion into Neural Radiance Fields with Pose-Conditioned Dataset Updates

CVPR 2024(2024)

Cited 6|Views16
Abstract
Neural radiance field (NeRF) is an emerging technique for 3D scene reconstruction and modeling. However, current NeRF-based methods are limited in the capabilities of adding or removing objects. This paper fills the aforementioned gap by proposing a new language-driven method for object manipulation in NeRFs through dataset updates. Specifically, to insert an object represented by a set of multi-view images into a background NeRF, we use a text-to-image diffusion model to blend the object into the given background across views. The generated images are then used to update the NeRF so that we can render view-consistent images of the object within the background. To ensure view consistency, we propose a dataset update strategy that prioritizes the radiance field training based on camera poses in a pose-ordered manner. We validate our method in two case studies: object insertion and object removal. Experimental results show that our method can generate photo-realistic results and achieves state-of-the-art performance in NeRF editing.
More
Translated text
Key words
Radiance Field,Neural Radiance Fields,Image Object,Diffusion Model,3D Scene,Camera Pose,Multi-view Images,Convolutional Neural Network,Generative Adversarial Networks,Target Object,Background Image,Image Synthesis,Sneakers,Insertion Method,Image Inpainting,3D Bounding Box,Traditional Pipeline,Traditional 3D,Masked Area
PDF
Bibtex
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Data Disclaimer
The page data are from open Internet sources, cooperative publishers and automatic analysis results through AI technology. We do not make any commitments and guarantees for the validity, accuracy, correctness, reliability, completeness and timeliness of the page data. If you have any questions, please contact us by email: report@aminer.cn
Chat Paper
Summary is being generated by the instructions you defined