Segmentation directed inpainting

Inpainting is the art of synthesizing large portions of images using the available background information. Most commonly used inpainting techniques fill the target region or hole in concentric layers from outside inward by sampling values from the background block by block. In each iteration, decisions are made in a greedy fashion taking into account only the local information on the gradient and isophote changes. This might lead to irregular extensions of region boundaries inside the hole, degrading the overall visual appearance. To overcome this, we impose constraints on the search space for source-exemplars by first segmenting the image and coupling regions on either side of the hole that would otherwise have a tendency to connect within the hole. This is achieved by interpolating their mutual boundaries into the hole using Bézier interpolations in a preliminary step. The aim of hole-partitioning is to inpaint each sub-hole separately by constraining the search space to one or more regions adjacent to the particular sub-hole. We also present a new confidence update rule is proposed which tries to measure the amount of faith in a block by evaluating how anomalous the block is in comparison to other blocks in the corresponding search space. Most inpainting algorithms choose a fixed block size which is decided before the algorithm starts. Using the additional information provided by our hole-partitioning algorithm, we choose block sizes separately for each sub-hole. The block size for a sub-hole is determined by mining the region characteristics of the corresponding search space by segmenting it at multiple scales. The proposed method is tested on several images and satisfactory results are see in most cases.