Fully Deep Simple Online Real-time Tracking: Efficient Re-Identification by Attention without Explicit Similarity Learning

Most existing Multi-Object Tracking methods consider detection and re-identification as two distinct steps. As a result, the re-identification cannot leverage object location and is only based on appearance, thus leading to ID merges when dealing with highly similar objects. The few works that combine detection and re-identification still generate an appearance descriptor for similarity computation. However, since the detection task conflicts with the tracking task, the network privileges the former and generates similar descriptors for objects of the same class, especially when class instances have a strong visual similarity. Besides, when using a motion model or a motion prediction recurrent neural network to delimit the search area and overcome the problem of ID merges, the rise of uncertainty occurring when those models are not updated often leads to ID switches. In this paper, we tackle these issues and propose to use the same model for detection and re-identification by leveraging attention between features of two frames. By doing so, the network can make motion predictions without providing any appearance descriptor and without computing any learned similarity, thus eliminating the need for any motion prediction model and making the tracking trainable end-to-end. Our experimental results support our main contributions and show that our fully DeepSORT significantly reduces the number of ID switches and merges, even when using non-class-agnostic non-maximum suppression. Besides, our model is more resistant to variations in time lapses between two images, leading to improved tracking results.