High-Precision Face Generation and Manipulation Guided by Text, Sketch, and Mask

Recently, diffusion models have made remarkable progress in face image processing using more modality. We propose a novel framework for high-precision face generation and manipulation guided by text, sketch, and mask input. Our approach introduces a Dual Branch Diffusion Model that integrates residuals into the diffusion process, transforming the standard denoising diffusion model into a unified, interpretable model suitable for both image generation and manipulation tasks. Quantitative results show that our method achieves competitive performance across multiple metrics and excels in preserving identity features, manipulating local regions, and performing structured edits. Ablation studies further reveal the effectiveness of the Dual Branch and Latent Space Regularization in enhancing the quality and diversity of generated images while emphasizing the necessity of balancing these components under different task settings.