EventPointNet: Robust Keypoint Detection with Neuromorphic Camera Data

In this paper, we propose a new method for keypoint detection using neuromorphic camera data. Robust keypoint detection in diverse conditions is a major issue for visual simultaneous localization and mapping (SLAM), place recognition, and computer vision. Recently, many methods adopt supervised learning to solve the problem. However, it is hard to define the exact reference keypoints on natural scenes, so the training process can be ambiguous for the problem. To handle this issue, we propose a new method named EventPointNet which is trained from data collected from a neuromorphic camera, also known as an event-based camera. Since the event-based camera captures natural edge points from any scenes regardless of illumination and viewpoint changes, the data can be used as proper references for keypoint detection. Therefore, a network trained with these data can detect distinct keypoints on a gray-scale image captured from a conventional camera. The proposed method is validated by comparing with both handcrafted and learning-based approaches on HPatches dataset. The experimental results show that EventPointNet detects more valid keypoints than the other methods in terms of both qualitative and quantitative results, especially on the illumination conditions with 1.31% higher matching score compared to the second best method. We also perform the visual odometry experiments on the KITTI dataset to show that EventPointNet can be applied to robotic applications. In particular, EventPointNet shows a reduction of 30.74% in the trajectory error compared to the second best algorithm.