Depth Estimation Based on an Infrared Projector and an Infrared Color Stereo Camera by Using Cross-Based Dynamic Programming with Cost Volume Filter

This paper presents a method to estimate a depth map using an infrared projector and a pair of infrared color cameras that can capture infrared and color images simultaneously. The infrared projector projects a dot pattern so that the cameras capture infrared images of a scene textured by the dots with which depths to surfaces in the scene can be estimated regardless of whether they have visible textures. Cost volumes are calculated for each of the infrared and color stereo images and are processed with a cost volume filter that smoothes each cost map by a cross-based local multipoint filter. The filtered infrared and color cost volumes are then integrated into a single cost volume by selecting either the infrared or color cost for each pixel according to the size of the adaptive kernel used for the cross-based local multipoint filter. This improves the accuracy where the adaptive kernel is small. We propose a method that can find optimal local curved surfaces of adaptive kernels from the cost volume by using three dynamic programmings. We experimented this depth estimation method on real-world datasets that the infrared color stereo cameras captured. We also used it for color stereo matching and showed that it works with normal color stereo cameras as well.