Hierarchical Improvement of Foreground Segmentation Masks in Background Subtraction

A plethora of algorithms have been defined for foreground segmentation, a fundamental stage for many computer vision applications. In this paper, we propose a post-processing framework to improve the foreground segmentation performance of background subtraction algorithms. We define a hierarchical framework for extending segmented foreground pixels to undetected foreground object areas and for removing erroneously segmented foreground. First, we create a motion-aware hierarchical image segmentation of each frame that prevents merging foreground and background image regions. Then, we estimate the quality of the foreground mask through the fitness of the binary regions in the mask and the hierarchy of segmented regions. Finally, the improved foreground mask is obtained as an optimal labeling by jointly exploiting foreground quality and spatial color relations in a pixel-wise fully connected conditional random field. Experiments are conducted over four large and heterogeneous data sets with varied challenges (CDNET2014, LASIESTA, SABS, and BMC) demonstrating the capability of the proposed framework to improve background subtraction results.