Self-Supervised Keypoint Detection Based on Multi-Layer Random Forest Regressor

This paper proposes a keypoint regressor (KeyReg), which consists of multi-layer random forest (MRF) regressor and single random forest (SRF) classifier modules. To increase the keypoints’ repeatability, the MRF regressor is applied to multi-scale images in a shared rules manner, and keypoints predicted at each scale are given a confidence score through the SRF for reliability measurement. Each candidate point is detected as the final keypoint through a non-maxima suppression process based on a confidence score. The MRF structure of KeyReg is designed to maintain a coarse-to-fine structure by varying the number of nodes per layer. In addition, the accuracy of the matching can be improved by removing less confidential keypoints through the continuous SRF classifier. KeyReg is the first approach to apply an MRF to the keypoint regression and is designed to run on a CPU rather than a GPU compared to DNN-based approaches. KeyReg training was conducted using COCO, and positive and negative examples were automatically obtained under a self-supervised learning method between the original image and a warped image. The proposed KeyReg showed superior performance in terms of repeatability, the accuracy of the homography, mean matching accuracy (MMA), and localization errors on HPatches dataset compared to state-of-the-art methods.