Synthesizing and Manipulating Natural Videos Using Image-to-Image Translation

Unpaired video-to-video translation is the mapping of elements of one video to another generated video using a neural network trained on unpaired data. While many methods have been developed in the attempt to generate naturalistic translated videos, many difficulties still exist. One major hurdle preventing the generation of naturalistic videos is the issue of texture consistency. This paper presents a modular video-to-video translation pipeline which leverages various fields of computer vision and image manipulation to produce high quality, texture-consistent translated videos. This paper also presents new paradigms for examining the performance of existing video-processing solutions, namely keyframe extraction methods, creating new metrics at which these methods are examined by. Experimental results show that the proposed method can increase the naturalistic appearance of these translated videos when compared to other existing unpaired methods.