SiamKey: Siamese Local Keypoints Matching Network for Visual Tracking

Siamese Network has been widely researched in the tracking framework due to its simple structure and easy expansion. Most Siamese-based Network trackers use a depthwise cross-correlation (DW-XCorr) to embed the template information into the search branch. However, this method has many limitations in Siamese Network. On the one hand, the size of template needs to be set in advance. If pre-fixed size is larger than actual target size, lots of background noise will be brought in. On the contrary, there will be losing lots of foreground information. On the other hand, DW-XCorr is a handcrafted region-based correlation module, and lots of background information will be brought in and overwhelms the feature of the target, making it hard to distinguish the target from similar objects in background. In order to solve above problems, this paper proposes a simple local keypoints matching network (SiamKey) implementing keypoint-based correlation. We select top-K keypoints of illumination and rotation invariance from template feature by peakedness measurement, and embed kepoints feature into search branch adaptively. Our SiamKey can significantly reduce the background noise and improve the robustness without pre-fixed template size. Experiments on challenging benchmarks including VOT2018
^[1]
, GOT-10k
^[2]
and OTB-100
^[3]
, demonstrate that the proposed SiamKey outperforms many state-of-the-art trackers.