CMWMF: Constant Memory Architecture of Weighted Mode/Median Filter for Extremely Large Label Depth Refinement

In this study, we propose a constant memory hardware architecture that can support weighted mode, median, and joint bilateral filters, which is referred to as CMWMF. This work aims to meet the high memory and computation requirements of processing depth maps with a large number of depth candidates. In the proposed architecture, we leverage the geometry smoothing characteristic of natural images to reduce the static random access memory (SRAM) size for hardware implementation. The architecture preserves a constant number of disparity values instead of depending on the label count and size of the local supporting window. A novel weighted median search procedure is proposed, which assigns a computation to each input cycle, thereby rendering the process hardware friendly. An index-checking technique is proposed to process out-of-order joint histograms. We adopted the above-mentioned techniques in our architecture as they consume a constant SRAM size and supports multiple types of filters. As a result, this architecture reduces the SRAM size by 92.4% with a negligible decrease in performance. According to our analysis on the KITTI, and Middlebury datasets, and with actual depth cameras, the preserved information is sufficient. The proposed architecture is one of the most suitable depth refinement architectures for scenarios having a large number of depth candidates.