PMMWMatcher: Weakly Supervised Feature Matching With Descriptor Enhancement for Passive Millimeter-Wave Images

Concealed object detection in passive millimeter-wave (PMMW) image has achieved significant progress. Using the limited information in 2-D PMMW images, however, some intractable cases cannot be handled, such as the objects with grayscale values close to the human body. Mining 3-D structures of concealed objects is beneficial to improve the performance and robustness of the detection. In this article, we dedicate to establish the accurate correspondence between PMMW images, which is an essential step in 3-D reconstruction. Such a task is especially challenging for textureless PMMW images with large noise. To this end, we introduce PMMWMatcher, a weakly supervised feature matcher that only leverages the camera pose as supervision, to promote the quality of current descriptors and establish the correspondence in a learnable manner. Concretely, we boost the representation of descriptors with self- and cross-attention. Then, we introduce a novel local-to-global searching strategy that comprehensively exploits the epipolar constraint between image pairs derived from the camera poses. Such a strategy effectively improves the robustness of descriptors for local feature matching in real scenes. Finally, we design a matching module to adaptively set up more accurate correspondence through deep reinforcement learning (RL) than the handcrafted methods. The experimental results show that PMMWMatcher can achieve a further enhancement in matching performance, with a +3.6% improvement in homography accuracy based on the existing PMMW feature points. The proposed method can be easily embedded in the existing keypoint detection pipelines, exhibiting great flexibility in practical applications.