A Simple but Effective Method for Balancing Detection and Re-Identification in Multi-Object Tracking

In recent years, joint detection and embedding (JDE) has become the research focus in multi-object tracking (MOT) due to its fast inference speed. JDE models are designed and widely utilized to train the detection task and the re-identification (Re-ID) task jointly. However, there exists a severe issue overlooked by previous JDE models, i.e., the detection task requires category-level features but the Re-ID task requires instance-level features. This could lead to feature conflict, which would hurt the performance of JDE models. Furthermore, inaccurate detection results can degrade the final tracking accuracy even when discriminative Re-ID features are provided. In this article, we propose a new balancing method for training JDE models, which monitors the training process of the detection task and adjusts the weights of the detection task and Re-ID task in the training phase. Our proposed balancing method ensures a well-trained detection model and a good trade-off between the detection task and Re-ID task. Comprehensive experiments on two public MOT benchmarks demonstrate the effectiveness and superiority of our proposed balancing method. In particular, our proposed balancing method could achieve new state-of-the-art results on MOT challenges without additional training data.