High Performance Visual Tracking With Siamese Actor-Critic Network

Object tracking is one of the fundamental tasks of computer vision and it is still a major challenge that trackers can balance between real-time speed and high performance. In this paper, a novel Siamese Actor-Critic network (SiamAC) is proposed to improve the accuracy and robustness of tracking while performing with real-time. Specifically, SiamAC consists of a fully convolutional siamese matching network for similarity learning and an Actor-Critic framework trained by reinforcement learning. The Actor is aimed to infer the optimal action in continuous space, while the Critic produces a Q-value to guide effectively the offline training of both Actor and Critic networks. During inference, according to the response map produced by the matching network, the most similar positions and scaled candidate patches are selected as the input of Actor-Critic. Subsequently, Actor can effectively search more precise location of these candidates and the Critic acts as a validator to decide the final tracking results with the highest confidence. Benefiting from this refinement, traditional multi-scale test and certain hyper-parameters in Siamese trackers can be discarded. Evaluations on popular benchmarks demonstrate that the proposed SiamAC achieves state-of-the-art performance with a real-time speed.