Temporally Consistent Video Manipulation for Facial Expression Transfer

Facial expression transfer is an important problem in the computer vision community due to its applicability in human-computer interaction, facial animation, etc. Most of the existing approaches mainly deal with images. However, expression transfer to videos is yet to be explored. The extension of existing image-based approaches to videos often leads to flickering artifacts in the result. This is due to the temporal inconsistency that may have been produced due to the independent processing of frames. In this paper, we address this problem by proposing a combination of loss functions in association with a deep learning-based method. Image reconstruction loss takes care of spatial domain consistency. The flow and warp losses address the temporal consistency. By maintaining consistency spatially as well as temporally, our method produces visually plausible results. The efficacy of our method is further explored in different problems like video dehazing, and video colorization.