Spatial-temporal consistent labeling for multi-camera multi-object surveillance systems

For an intelligent multi-camera multi-object surveillance system, object correspondence across time and space is important to many smart visual applications. In this paper, we propose a temporal and spatial consistent labeling algorithm for this demand. In the algorithm, an object corresponding database records the temporal and spatial consistency information for each segmented mask. With the database, the object-mask correlations are propagated through the propagation rules by analyzing mask splitting/merging conditions. In the spatial consistent labeling method, the homography warping and the earth mover’s distance are adopted to match same objects across different views. The earth mover’s distance solves the double matching problem, allows the algorithm to work normally under a small deviation of detected object locations, and makes pairing results have minimum global matching distances. The concept trusting-former-pairs-more is also adopted to avoid frequent pair switching if two objects are too close. The correct spatial labeling rate is about 89.25% in average. For online processing applications, the algorithm need not trace back to the past frames. The overall processing speed is about 10.24 frame per second (fps) with CIF size video running on a 2.8GHz general purpose CPU.