End-to-End Multiple Object Tracking based on Multi-Frame Regression

Multiple Object Tracking (MOT) is a fundamental and challenging task in the field of computer vision. Recently, the MOT community has turned its attention from the tracking-by-detection paradigm to a brand new one, namely joint detection and tracking (JDT). In this paper, we address the crowded tracking issue in the JDT paradigm and propose an end-to-end trainable multiple object tracking model based on multi-frame regression, which we term as MFR-tracker. Different from the traditional JDT paradigm which generates frame-level detections, our MFR-tracker takes short-term temporal information into account and directly predicts tracklets in a video clip. Specifically, we approximate a tracklet as an adaptive Bezier curve and learn the object’s arbitrary motion with only several control points. The simple yet effective motion modeling helps a lot in distinguishing different objects, especially when they are occluded somewhere. We conduct all experiments on MOT benchmarks of MOT16, MOT17, and MOT20, demonstrating its remarkable tracking performance.