Robust Depth Enhancement via Polarization Prompt Fusion Tuning
CVPR 2024(2024)
摘要
Existing depth sensors are imperfect and may provide inaccurate depth values
in challenging scenarios, such as in the presence of transparent or reflective
objects. In this work, we present a general framework that leverages
polarization imaging to improve inaccurate depth measurements from various
depth sensors. Previous polarization-based depth enhancement methods focus on
utilizing pure physics-based formulas for a single sensor. In contrast, our
method first adopts a learning-based strategy where a neural network is trained
to estimate a dense and complete depth map from polarization data and a sensor
depth map from different sensors. To further improve the performance, we
propose a Polarization Prompt Fusion Tuning (PPFT) strategy to effectively
utilize RGB-based models pre-trained on large-scale datasets, as the size of
the polarization dataset is limited to train a strong model from scratch. We
conducted extensive experiments on a public dataset, and the results
demonstrate that the proposed method performs favorably compared to existing
depth enhancement baselines. Code and demos are available at
https://lastbasket.github.io/PPFT/.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要