Two-Step Affine Transformation Prediction for Visual Object Tracking

The main drawback of correlation filter scheme is that it can only predict translation of the object, ignoring the other affine transformations such as rotation, aspect ratio change, and scale change. This paper tries to address this problem by a two-step affine transformation prediction method. The first step is to predict coarse target translation by the correlation filter model with adaptive model update rate generation. Compared with fixed update rate in existing correlation filter based trackers, the presented method generates the update rate by Long Short-Term Memory model with inputs of historical target templates. The second step maps well-designed features to affine transformation parameters. The designed features contain the information of affine transformation which makes the learning of non-linear mapping function possible. The training samples for on-line filter model update are transformed to the same pose, and they help the learnt filter to represent the object better than non-aligned samples. The proposed network is trained from end to end and achieves competitive performance when comparing with state-of-the-arts trackers on four benchmarks, OTB100, UAV123, VOT2018, and VOT2020.