On Evaluating Video Object Segmentation Quality: A Perceptually Driven Objective Metric

The task of extracting objects in video sequences emerges in many applications such as object-based video coding (e.g., MPEG-4) and content-based video indexing and retrieval (e.g., MPEG-7). The MPEG-4 standard provides specifications for the coding of video objects, but does not address the problem of how to extract foreground objects in image sequences. Therefore, for specific applications, evaluating the quality of foreground/background segmentation results is necessary to allow for an appropriate selection of segmentation algorithms and for tuning their parameters for optimal performance. Many segmentation algorithms have been proposed along with a number of evaluation criteria. Nevertheless, formal psychophysical experiments evaluating the quality of different video foreground object segmentation results have not yet been conducted. In this paper, a generic framework for both subjective and objective segmentation quality evaluation is presented. An objective quality assessment method for segmentation evaluation is derived on the basis of perceptual factors through subjective experiments. The performance of the proposed method is shown on different state-of-the-art foreground/background segmentation algorithms and our method is compared to other objective methods which do not include perceptual factors. Moreover, on the basis of subjective results, weighting strategies are introduced into the proposed metric to meet the specificity of different segmentation applications e.g., video compression, video surveillance and mixed reality. Experimental results confirm the efficiency of the proposed approach.