Leveraging Thermal Modality to Enhance Reconstruction in Low-Light Conditions
CoRR(2024)
摘要
Neural Radiance Fields (NeRF) accomplishes photo-realistic novel view
synthesis by learning the implicit volumetric representation of a scene from
multi-view images, which faithfully convey the colorimetric information.
However, sensor noises will contaminate low-value pixel signals, and the lossy
camera image signal processor will further remove near-zero intensities in
extremely dark situations, deteriorating the synthesis performance. Existing
approaches reconstruct low-light scenes from raw images but struggle to recover
texture and boundary details in dark regions. Additionally, they are unsuitable
for high-speed models relying on explicit representations. To address these
issues, we present Thermal-NeRF, which takes thermal and visible raw images as
inputs, considering the thermal camera is robust to the illumination variation
and raw images preserve any possible clues in the dark, to accomplish visible
and thermal view synthesis simultaneously. Also, the first multi-view thermal
and visible dataset (MVTV) is established to support the research on multimodal
NeRF. Thermal-NeRF achieves the best trade-off between detail preservation and
noise smoothing and provides better synthesis performance than previous work.
Finally, we demonstrate that both modalities are beneficial to each other in 3D
reconstruction.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要