High Fidelity Motion Adaptive Face Reenactment

Along with the development of virtual reality and augmented reality technology, the requirement for high fidelity face reenactment methods increases gradually. Recent face reenactment works either use landmarks or Action Units (AUs) based motion representation as the guided signal. However, it is difficult to represent the complex facial geometry and movement completely only with landmarks or AUs, which leads to the limited effect of these methods. To address this challenge, we propose a high fidelity face reenactment framework that combines landmarks and AUs together for facial motion representation. Specifically, we design a Motion Alignment Converter (MAC) module to align the landmarks and AUs when driving and source identities are different. After that, an Adaptive Motion Controller (AMC) module is proposed to inject motion information into the face reenactor. Moreover, to optimize output quality, the proposed method processes background and face regions separately. Qualitive and quantitative experiments are conducted on VoxCeleb1 and CelebV datasets, the results show that the proposed method can generate face images with more details and better background than existing methods.