Multimode spatiotemporal background modeling for complex scenes

We present a new approach for modeling background in complex scenes that contain motions caused e.g. by wind over water surface, in tree branches, or over the grass. The background model of each pixel is defined based on the observation of its spatial neighborhood in a recent history, and includes up to K ≥ 1 modes, ranked in decreasing order of occurrence frequency. Foreground regions can then be detected by comparing the intensity of an observed pixel to the high frequency modes of its background model. Experiments show that our spatial-temporal background model is superior to traditional related algorithms in cases for which a pixel encounters modes that are frequent in the spatial neighborhood without being frequent enough in the actual pixel position. As an additional contribution, our paper also proposes an original assessment method, which has the advantage of avoiding the use of costly handmade ground truth sequences of foreground objects silhouettes.