Unified multi-camera detection and tracking using region-matching

We describe an algorithm for detecting and tracking multiple people in a cluttered scene using multiple synchronized cameras located far away from each other. This camera arrangement results in multiple wide-baseline camera systems. We segment each image, and then, for each pair we compare regions across the views along epipolar lines. The centers of the matching segments are then back-projected to identify 3D points in the scene potentially corresponding to people. These 3D points are then projected onto the ground plane. The results from these wide-baseline camera systems are then combined using a scheme that rejects outliers and gives very robust estimates of the 2D locations of the people. These estimates are then used to track people across time. We have found that the algorithm works quite well in practice in scenes containing multiple people even when they occlude each other in every camera view.