Sparse Feature Representation Learning for Deep Face Gender Transfer

Why do people think Tom Hanks and Juliette Lewis look alike? Can we modify the gender appearance of a face image without changing its identity information? Is there any specific feature responsible for the perception of femininity/masculinity in a given face image? Those questions are appealing from both computer vision and visual perception perspectives. To shed light upon them, we propose to develop a GAN based approach toward face gender transfer and study the relevance of learned feature representations to face gender perception. Our key contributions include: 1) an architecture design with specially tailored loss functions in the feature space for face gender transfer; 2) the introduction of a novel probabilistic gender mask to facilitate achieving both the objectives of gender transfer and identity preservation; and 3) identification of sparse features (≈ 20 out of 256) uniquely responsible for face gender perception. Extensive experimental results are reported to demonstrate not only the superiority of the proposed face gender transfer technique (in terms of visual quality of reconstructed images) but also the effectiveness of gender feature representation learning (in terms of the high correlation between the learned sparse features and the perceived gender information). Our findings seem to corroborate a hypothesis about the independence between face recognizability and gender classifiability in the literature of psychology. We expect this work will stimulate more computational studies of different face perception attributes including race, age, attractiveness, and trustworthiness.