Classification of Synthetic Facial Attributes by Means of Hybrid Classification/Localization Patch-Based Analysis

Facial attributes editing, that is the manipulation of some specific attributes of a face image, is a new trend in the generation of synthetic images by GANs. Several recent studies have shown the possibility to detect the synthetic nature of such images by training a DL-based binary classifier. At the same time, the question about the specific face attributes that have been altered is typically disregarded, yet this may be a crucial information for forensic analysts. In this paper, we propose a new architecture whose objective is to identify the altered facial attributes of synthetic face images. To do so, we developed a hybrid classification-and-localization architecture. The local and global features are first extracted from the full image and from specific image patches, and then merged by using an attentional feature fusion module. The extensive experiments we have carried out involving 19 different facial attributes, manipulated by a StyleGAN2 network, show the good accuracy of the proposed method and its robustness against several image post-processing operators.

1. INTRODUCTION

Facial image manipulation using GAN architectures has gained popularity [1], [2], [3], [4], and is being increasingly used in a wide variety of real-world application scenarios [5]. The diffusion of manipulated images poses a serious threat to public trust, and many efforts have been made to distinguish fake images from real ones. However, in many cases, knowing that a face image is fake without a solid proof of why and where it has been manipulated is not sufficient. As an example, Figure 1 shows some real images reconstructed and partially corrupted by a StyleGAN2 architecture [2], [3], [6]. All images represent the same identity and though all of them have been manipulated, the goal of the manipulation is different for all the images, with a different facial expression artificially injected in all of them.