A global probabilistic framework for the foreground, background and shadow classification task

Over the years, many works have been published on the two-dimensional foreground segmentation task, describing different methods that treat to extract that part of the scene containing active entities. In most of the cases, the stochastic background process for each pixel is modeled first, and then the foreground pixels are classified as an exception to the model or using maximum a posteriori (MAP) or maximum likelihood (ML). The shadow is usually removed in a later stage and salt and pepper noise is treated with connected component analysis or mathematical morphology. In this paper, we propose a global method that classifies each pixel by finding the best possible class (foreground, background, shadow) examining the image globally. A Markov random field is used to represent the dependencies between all the pixels and classes and the global optimal solution is approximated with the belief propagation algorithm. The method can extend most local methods and increase their accuracy. In addition, this approach brings a probabilistic justification of the classification problem and it avoids the use of additional post-processing techniques.