MSAT: Multi-scale Sparse Attention Target Tracking

Current object tracking task have seen significant performance improvements is due to the huge expansion of model parameters. The Transformer architecture with global features and modality fusion capabilities is a typical example. However, the complete attention mechanism used in this architecture has a quadratic complexity dependence on the length of the input sequence, making it impractical for many practical engineering scenarios. In this paper, we propose an MSAT tracker based on multi-scale sparse Transformer, which includes a multi-scale feature interaction backbone network, that performs multi-stage feature fusion to enhance anti-interference ability and robustness, and alleviate the interference accumulation effect of shallow features in the backbone feature extraction. Secondly, we design a feature fusion module based on the deformable attention mechanism and the bilateral sparse attention mechanism, achieves the effect of reducing the memory consumption of Transformer, improving the focus on target features, and weakening the interference of background information. Additionally, we propose a multi-stage template feature update strategy to ensure that reliable template information is obtained during tracking. Extensive experiments show that our tracker achieved efficient performance that surpassed the most advanced trackers on GOT-10k, OTB2015, TrackingNet and LaSOT, while runs a stable real-time tracking at about 45FPS.