PMMWMatcher: Weakly Supervised Feature Matching With Descriptor Enhancement for Passive Millimeter-Wave Images.

Concealed object detection in passive millimeter-wave (PMMW) image has achieved significant progress. Using the limited information in two-dimensional PMMW images, however, some intractable cases cannot be handled, such as the objects with grayscale values close to the human body. Mining three-dimensional (3D) structures of the concealed objects is beneficial to improve the performance and robustness of the detection. In this paper, we dedicate to establish the accurate correspondence between PMMW images, which is an essential step in 3D reconstructions. Such a task is especially challenging for textureless PMMW images with large noise. To this end, we introduce PMMWMatcher, a weakly supervised feature matcher that only leverages the camera pose as supervision, to promote the quality of current descriptors and establish the correspondence in a learnable manner. Concretely, we boost the representation of descriptors with self- and cross-attention. Then we introduce a novel local-to-global searching strategy that comprehensively exploits the epipolar constraint between image pairs derived from the camera poses. Such a strategy effectively improves the robustness of descriptors for local feature matching in real scenes. Finally, we design a matching module to adaptively set up more accurate correspondence through deep reinforcement learning than the hand-crafted methods. The experimental results show that PMMWMatcher can achieve a further enhancement in matching performance, with a +3.6% improvement in homography accuracy based on the existing PMMW feature points. The proposed method can be easily embedded in the existing keypoint detection pipelines, exhibiting great flexibilities in practical applications.