Spatio-temporal video segmentation using a joint similarity measure

This paper presents a new morphological spatio-temporal segmentation algorithm. The algorithm incorporates luminance and motion information simultaneously and uses morphological tools such as morphological filters and watershed algorithm. The procedure toward complete segmentation consists of three steps: joint marker extraction, boundary decision, and motion-based region fusion. First, the joint marker extraction identifies the presence of homogeneous regions in both motion and luminance, where a simple joint marker extraction technique is proposed. Second, the spatio-temporal boundaries are decided by the watershed algorithm. For this purpose, a new joint similarity measure is proposed. Finally, an elimination of redundant regions is done using motion-based region fusion. By incorporating spatial and temporal information simultaneously, we can obtain visually meaningful segmentation results. Simulation results demonstrates the efficiency of the proposed method.