Self-Attention-Masking Semantic Decomposition and Segmentation for Facial Attribute Manipulation

Many face attribute manipulation methods can only provide global attribute manipulation according to the attribute labels. In this paper, we propose a self-attention-masking semantic decomposition method which is able to learn an attribute attention mask for each attribute. User can adjust the strength and color of each attribute smoothly and more freely. We decouple the attention of different attributes and overcome the disadvantage of overlap between different attribute attention masks by an attention weighting module. Thanks to the attribute attention masks, our method allows manipulate facial attribute without generator after only once generation. Moreover, we can perform facial semantic segmentation without pixel level semantic labels. Experiments show that our method simultaneously improves the freedom of attribute manipulation and the authenticity of synthetic face. The mean intersection over union of semantic segmentation is over 65% for hair and skin. Our code is available at github.com/flyfeatherok/SAMSD.