Spatiotemporal algorithm for joint video segmentation and foreground detection

We present a novel algorithm for segmenting video sequences into objects with smooth surfaces. The segmentation of image planes in the video is modeled as a spatial Gibbs-Markov random field, and the probability density distributions of temporal changes are modeled by a Mixture of Gaussians approach. The intensity of each spatiotemporal volume is modeled as a slowly varying function distorted by white Gaussian noise. Starting from an initial spatial segmentation, the pixels are classified using the temporal probabilistic model and moving objects in the video are detected. This classification is updated by Markov random field constraints to achieve smoothness and spatial continuity. The temporal model is updated using the segmentation information and local statistics of the image frame. Experimental results show the performance of our algorithm.