Rethinking Motion Estimation: An Outlier Removal Strategy in SORT for Multi-Object Tracking With Camera Moving

Multi-Object Tracking (MOT) involves the simultaneous tracking of multiple targets in a scene, demanding accurate discrimination of foreground and background, as well as precise identification of feature distinctions among diverse objects. Simple Online and Real-Time Tracking (SORT) is a widely adopted method in MOT, leveraging dual-phase Kalman Filter (KF) for object state estimation and ensuring consistent tracker association throughout a video sequence. Recent advancements in SORT-like algorithms aim to address nonlinear object motion and reduce reliance on detection for association in SORT. Despite these improvements, existing SORT-like methods often overlook camera motion, resulting in suboptimal motion prediction under dynamic camera conditions. In this paper, we introduce a novel SORT-like approach, termed Outlier Removal-based SORT (OR-SORT), which introduces a novel triple-phase Kalman Filter, encompassing prediction, re-prediction, and update phases. This framework dissects the object motion state transition model into distinct components—linear velocity self-motion and camera motion. Additionally, our method employs outlier removal based on Mixed Integer Linear Programming (MILP) to enhance camera motion estimation accuracy. Experimental evaluations on the MOTChallenge datasets, including the scenarios with both moving cameras and high object densities, demonstrate our method’s superior performance, particularly in scenarios with moving cameras. Our approach achieves a state-of-the-art MOTA of 80.7% and IDF1 of 79.6% on MOT17, and 77.9% and IDF1 of 76.4% on MOT20.