An Effective Method for Semi-Online Multi-Object Tracking Refinement

In multi-object tracking(MOT), identity(ID) switches (i.e., single tracklet containing different objects) are common. Here, we propose a semi-online tracking refinement method, where the ID switches are detected by monitoring the changes in appearance similarity within a short duration temporal window. When an ID switch occurs, frames containing different object will firstly enter the window, causing a large drop in appearance similarity. As the window moves forward, the ID switch frame will exit the window, causing an increase in appearance similarity since the window is about to be solely filled with the switched object. This ‘drop-increase’ pattern in appearance similarity within the moving temporal window can be used to identify the ID switch point. Frames containing switched object are then split from the original tracklet and attached to other tracklets based on the similarities among their multiple representative prototypes. Comparing to the baseline, our refinement method can significantly improve the IDF1 score on MOT17 and MOT20 in a real-time manner.