Cosegmentation and Cosketch by Unsupervised Learning

Co segmentation refers to the problem of segmenting multiple images simultaneously by exploiting the similarities between the foreground and background regions in these images. The key issue in co segmentation is to align common objects between these images. To address this issue, we propose an unsupervised learning framework for co segmentation, by coupling co segmentation with what we call ``co sketch''. The goal of co sketch is to automatically discover a codebook of deformable shape templates shared by the input images. These shape templates capture distinct image patterns and each template is matched to similar image patches in different images. Thus the co sketch of the images helps to align foreground objects, thereby providing crucial information for co segmentation. We present a statistical model whose energy function couples co sketch and co segmentation. We then present an unsupervised learning algorithm that performs co sketch and co segmentation by energy minimization. Experiments show that our method outperforms state of the art methods for co segmentation on the challenging MSRC and iciest datasets. We also illustrate our method on a new dataset called Coseg-Rep where co segmentation can be performed within a single image with repetitive patterns.