Spatio-temporal segmentation for video surveillance

Automatic extraction of moving objects and construction of site models are key problems in surveillance systems. We present an approach to segmenting moving objects from static scenes as well as building site models automatically. With the video sequence captured by a static camera, we describe a robust algorithm to estimate the background image as the 2D site environment map. Then, we can label each pixel in an image volume formed by a video sequence as either "foreground" or "background". Clustered foreground pixels are used as a cue to seed selection when we perform 3D segmentation in the image volume. Unlike some conventional segmentation approaches, our algorithm utilizes the spatio-temporal information instead of spatial information only. By a uniform 3D segmentation we can refine the estimate of the static environment map and detect the moving objects automatically. Experimental results on veal video sequences demonstrate the robustness and accuracy of the algorithm.