SynExpression: A Diffusion-Based Framework for Controllable Facial Expression Synthesis and Emotion Detection Using Facial Segmentation Pose Maps

Facial expression synthesis and emotion detection are crucial for advancing computer vision applications, including virtual reality and affective computing. This study introduces SynExpression, a novel framework that combines Stable Diffusion and ControlNet to generate diverse, realistic facial expressions guided by pose maps. Central to this framework is SegPoseMap, a cutting-edge segmentation method integrating DeepLabV3 and transformer-based self-attention layers. SegPoseMap enables precise segmentation and alignment of key facial landmarks, providing geometrically accurate masks that significantly improve the fidelity and realism of synthesized expressions. SegPoseMap provides precise delineation of facial landmarks, achieving an average precision (AP) of 96% across multiple intersection-over-union (IoU) thresholds and excelling in segmenting large facial regions with near-perfect precision (98%). Its superior facial feature localization performance significantly enhances synthesized expressions’ fidelity and geometric accuracy. Additionally, SegPoseMap’s segmentation outputs improve emotion detection by enabling precise tracking of subtle facial cues. Experimental results show that SynExpression achieves 99% accuracy and a 99.5% F1 score on a Diffusion-Based dataset. With applications in digital forensics, animation, and emotion analysis, SynExpression offers a robust, versatile, and high-precision solution for controllable facial expression generation and emotion detection.