Multiscale Spatio-Temporal Information Cascade Single-Object Visual Tracker

Recently, Transformer has been largely explored in object tracking, and shown state-of-the-art (SOTA) performance compared to convolutional neural networks (CNNs). Especially, single-object trackers based on pure Transformer and “CNN+Transformer” frameworks have achieved great success in terms of accuracy and speed. However, most methods do not fully exploit the temporal and spatial information of targets. Furthermore, the potential of trackers for spatial information interaction and propagation between the search area and templates remains underexplored. These all limit the further improvement in tracking performance. To address these issues, we propose a multiscale cascaded single-object tracking framework based on spatio-temporal information fusion (STIF), which more comprehensively integrates the temporal and spatial information of targets and more deeply interacts the information in the search area and templates. In particular, to establish extensive spatio-temporal feature correlations, the STIF network is introduced, which uses a Transformer-based encoder-decoder structure to cross-fuse the global nonlinear temporal and spatial information of the target search area with static and dynamic templates, effectively performing fusion-based propagation. To focus on rich spatial semantic information, we design a multiscale feature extraction (MFE) network, and a feature cascade aggregation (FCA) module based on the encoder-decoder structure, which can effectively carry out interaction-based propagation. Finally, a bounding box prediction head and an IoU score head are used to predict the exact location of the target and update the dynamic templates, respectively. Extensive experiments demonstrate that our method attains better tracking performance than the baseline method. Meanwhile, the proposed method also obtains comparable results with other SOTA trackers on six challenging benchmarks, including GOT-10k, TrackingNet, LaSOT, UAV123, NFS30, and OTB100, while running at real-time speed.