Probabilistic Assignment with Decoupled IoU Prediction for Visual Tracking

Modern Siamese trackers mainly rely on classifying and regressing pre-defined anchor boxes or per-pixel points, which are assigned as positive and negative samples based on box intersection-over-union (IoU) or point distance with corresponding ground-truth for training. However, this rigid configuration potentially involves some noisy and ambiguous positive samples, leading to an inconsistency problem between classification and regression, which limits the tracking performance. In this paper, we propose a novel probabilistic assignment approach that dynamically determines positive/negative samples for each instance. To be specific, we first customize the confidence scores of positive candidates by comprehensively exploring the outputs from both classification and regression heads, and fit these scores as a probability distribution. Therefore, it is intuitive to conduct adaptive label assignment according to their probabilities. Then, we also consider dynamic re-weighting factor for each positive sample, jointly optimizing the classification and regression losses in a synchronized manner. Moreover, we introduce a decoupled IoU prediction branch to bridge the gap between the training and inference objectives for accurate tracking. Thanks to well-aligned procedures, our method significantly improves the performance of both CNN-based and Transformer-based trackers. Extensive experiments conducted on several tracking benchmarks including LaSOT and GOT-10k, demonstrate the effectiveness and efficiency of the proposed probabilistic assignment tracker.