Multi-Granularity Tracking with Modularlized Components for Unsupervised Vehicles Anomaly Detection

Anomaly detection on road traffic is a fundamental computer vision task and plays a critical role in video structure analysis and urban traffic analysis. Although it has attracted intense attention in recent years, it remains a very challenging problem due to the complexity of the traffic scene, the dense chaos of traffic flow and the lack of fine-grained abnormal labeled data. In this paper, we propose a multi-granularity tracking approach with modularized components to analyze traffic anomaly detection. The modularized framework consists of a detection module, a background modeling module, a mask extraction module, and a multi-granularity tracking algorithm. Concretely, a box-level tracking branch and a pixel-level tracking branch is employed respectively to make abnormal predictions. Each tracking branch helps to capture abnormal abstractions at different granularity levels and provide rich and complementary information for the concept learning of abnormal behaviors. Finally, a novel fusion and backtracking optimization is further performed to refine the abnormal predictions. The experimental results reveal that our framework is superior in the Track4 test set of the NVIDIA AI CITY 2020 CHALLENGE, which ranked first in this competition, with a 98.5% F1-score and 4.8737 root mean square error.