A Facial Expression Transfer Method Based on 3DMM and Diffusion Models

Due to the complex geometric features of facial expressions, achieving realistic facial expression transfer is a challenging task. This paper proposes a two-stage facial expression transfer method based on 3D Morphable Model (3DMM) and diffusion models. In the first stage, 3DMM-based DECA model is employed to map the source identity face and the target expression face 2D image into 3D facial parameters. Subsequently, the expression parameters of the source identity face are substituted with the expression parameters obtained from the target expression face. This procedure reconstructs a 3D flame model of the face with the transferred expression. Finally, the 3D flame model, in conjunction with the source identity face rendering-related parameters, is fed into a differentiable renderer to generate a 2D image, completing a rough expression transfer. However, due to the inability of the 3DMM to reconstruct the hair and teeth in the face, the rough expression transfer 2D images obtained in the first stage have obvious artificial traces. To address the above issues, this paper uses the diffusion model in the second stage to repair artificial traces in 2D images, aiming to achieve a close resemblance to the source identity face after expression transfer.