2D-to-3D Mutual Iterative Optimization for 3D Multi-camera Multiple People Tracking

Multi-camera Multiple People Tracking (MMPT) is a challenging task in advanced visual monitoring systems. The main challenge of MMPT is how to accurately match the single-camera trajectories generated from different viewpoints and establish one global and complete cross-camera trajectory for each target, i.e., the multi-camera trajectory matching problem. In this paper, we propose a novel framework to solve this problem using a scene-aware multiple object tracking. Furthermore, unlike most existing methods that purely use single-camera trajectories for multiple object tracking, we introduce a new multiple camera compensation mechanism 2D-to-3D Mutual Iterative Optimization for MMPT (MIO-MMPT) to enhance the person tracking results, which exploits the crucial multi-camera relationships among the human trajectories appearing in different cameras both robustly and automatically. Based on the camera calibration, we can project the 2D coordinate into 3D coordinate to achieve more reliable tracking results for each person. Once we have the 3D tracking results, we can re-project to 2D coordinate of each camera to solve the missing detection issues from occlusion or the blind spot of the camera. According to our experimental results, the proposed method achieves a new state-of-the-art on ICCV 2021 MMPT dataset with MOTA of 95% and IDF1 of 96%.