Robust Multiobject Tracking Using Vision Sensor With Fine-Grained Cues in Occluded and Dynamic Scenes

Multiobject tracking (MOT) using vision sensors remains a challenging problem, particularly in dynamic backgrounds and severe occlusions. Existing methods, relying on holistic appearance or spatial cues, fail to capture detailed information within object regions and the background, resulting in inaccurate and inconsistent tracking. To address these issues, we propose a novel detail-driven MOT (DD-MOT) method that revisits and leverages fine-grained cues to discover the dynamics of both object regions and the background, facilitating the recovery and association of trajectories. Specifically, the proposed method consists of three key modules: 1) points trajectories generator (PTG) module; 2) camera otion and occlusion compensation (CMOC) module; and 3) fine- and coarse-grained association (FCGA) module. The PTG module is responsible for generating fine-grained cues by sampling an initial set of points, generating point trajectories, and refining the initial set of points. The CMOC module utilizes the background and object point trajectories to correct background motion and recover occluded object bounding boxes. Finally, the FCGA module leverages point trajectory cues to assist in establishing a more effective association strategy, combining both fine-grained and coarse-grained spatial cues from the object bounding boxes. We evaluate our DD-MOT method on several benchmark datasets, including MOT17 and MOT20, demonstrating that it consistently outperforms state-of-the-art (SOTA) methods in key metrics such as HOTA (64.2, 62.6), MOTA (80.8, 78.1), and IDF1 (78.8, 77.1).