Multiple Cues Association for Multiple Object Tracking Based on Convolutional Neural Network

Tracking-by-detection is a popular framework for Multiple Object Tracking (MOT) where detectors produce a set of labeled detection to indicate the categories, the size, and the position of the objects. Most of the MOT approaches use the detection as points to do association. However, in this manner, these methods ignore the majority of useful association cues. As a result, the accuracy of the association is not optimal. In our approach, we combine multiple cues like object appearance feature, object size and position to improve the association confidence. Instead of treating the detection and tracking as two separate parts we directly extract the appearance features from the detector's feature map and append an extra association network to fuse the multiple cues. The architecture of our approach is an end-to-end detection-association network. The input of our network is the image sequence, and the output is the inter-frame target association matrix. We tested our network on MOT17 challenge dataset. The results show that our solution significantly improves the short term association accuracy when compared with the single-cues association methods while keeping a lower consumption of computing resources.