Balancing Multiple Object Tracking Objectives based on Learned Weighting Factors

Advanced multiple object tracking requires multi-task learning in order to solve object detection and data association tasks simultaneously. One fundamental characteristic of multi-task learning is that there is correlated information among tasks, and leveraging this property in training the model can result in better generalization performance. However, in multiple object tracking, most existing methods learn such property by treating multiple task losses equally and independently. In this paper, we take the weighting of multiple object tracking losses into consideration, relying on the related information among object detection and data association tasks. Firstly, this paper introduces a simple but effective Learned Weighting Factors (LWF) method where the weighting factors are learned through shallow neural networks. These learned factors are used to balance multi-task losses during training dynamically. Thus, our LWF method avoids manually tuning these weighting factors because this process is difficult and expensive caused by the high dimension of the search space. To the best of our knowledge, the proposed LWF method is a new and different perspective in the literature. Secondly, we conduct extensive experiments on two benchmark datasets, i.e., MOT16 and MOT20, surpassing state-of-the-art trackers without extra training samples. Video surveillance demos are available at https://bit.ly/3hkgBxo.