Video Background Completion Using Motion-Guided Pixel Assignment Optimization

Background completion for consumer videos captured by free-moving cameras is a challenging problem. In this paper, we present a new approach to complete the holes left by removing objects with motion-guided pixels assignment optimization. We first estimate the motion field in the holes by applying a two-step motion propagation method. Then, using estimated motion field as guidance, the missing parts of the video are completed by performing pixels assignment optimization based on the Markov random field, which optimally assigns available pixels from other neighboring video frames to the missing regions. Finally, we present an illumination-adjusting approach to eliminate the illumination inconsistency in the completed holes. We validate our method on a variety of videos captured by free-moving cameras. Compared with previous methods, our method works better to keep the completed background spatiotemporally coherent, to complete video background with much depth discontinuity and to make the illumination consistent in the completed region.