Attribute-Aware Head Swapping Guided by 3d Modeling

Face manipulation has ignited the interests of both academia and industry in very recent years. Existing face manipulation methods can be roughly categorized into two types: face attribute editing and face swapping. In this paper, we focus on swapping the identity. But unlike face swapping which only changes the face region, we attempt at a more challenging task: attribute-aware head swapping. Given a source video and a target video, we replace the whole target head with the whole source head while keeping the original target attributes. To address the inherent appearance gap (e.g., hairstyle, face shape), accompanying background incompatibility and lighting difference, our method consists of three key components: 1) a generative rendering-to-real-head model for source head modeling and attribute transfer; 2) a background modeling network to fix the background incompatibility during head swapping; 3) a deep harmonization network to fix remaining issues and makes the final composited result more realistic. We compare our approach to different face manipulation methods and the experimental results demonstrate its superiority for a lot of challenging cases.