Optimization of video synthesis by means of cost-guided multimodal photometric correction

In this paper, a novel texture synthesis approach with spatio-temporal boundary conditions is presented. The proposed method is non-parametric and patch-based. Blending between overlapping patches is optimized using a graph-cut technique. Furthermore, a set of photometric correction algorithms, namely Poisson [1] and covariant [2] cloning, is used simultaneously to fit continuation patches into a given neighborhood of the synthetic texture, which minimizes the visibility of patch transitions. The selection of the optimal correction mode is steered by a perceptual cost function. The Poisson and covariant cloning approaches are modified to fit the video synthesis framework proposed in this paper. Our experiments show that the synthesis quality of complex textures can be improved by up to 82% by using an online cost-guided selection of the optimal photometric correction method compared to graph-cut-only approach.