DiffDRNet: A Latent Diffusion Model-Based Method for Depth Completion

In robotic tasks involving object grasping, pose estimation using accurate depth information combined with RGB images and neural network models has become a standard solution. Industrial-grade structured light cameras can capture precise depth information of targets and generally exhibit strong adaptability and stability. However, they are often unsuitable for deployment on robots due to their large size, high power consumption, high cost, and poor real-time performance. Although consumer-grade structured light cameras are cost-effective and offer real-time capabilities, they suffer from large measurement errors and depth loss due to reflective materials, especially in dynamic scenes where movement occurs, which severely impacts downstream grasping tasks. To address these issues, we propose the DiffDRNet model, designed to mitigate the limitations of consumer-grade structured light cameras through neural network-based depth completion. DiffDRNet reformulates this task as a conditional denoising diffusion process, using RGB images and incomplete depth maps as guiding conditions to “denoise” random depth distributions into high-precision depth maps. The network leverages Swin Transformers to extract multi-scale features from RGB images and incomplete depth maps with varying degrees of noise. These features are then fused using Content-Guided Attention to produce a feature map with channel specificity and interactive information. By executing the diffusion process in latent space through dedicated encoder and decoder designs, DiffDRNet achieves high-resolution, efficient depth completion. Experimental results on the NYU-Depth-V2 dataset demonstrate that DiffDRNet achieves high-quality depth completion, showcasing the potential of diffusion models for depth completion tasks.