“A Leopard Cannot Change Its Spots”: Improving Face Recognition Using 3D-Based Caricatures

Caricatures refer to a representation of a person, in which the distinctive features are deliberately exaggerated, with several studies showing that humans perform better at recognizing people from caricatures than using original images. Inspired by this observation, this paper introduces the first fully automated caricature-based face recognition approach capable of working with data acquired in the wild. Our approach leverages the 3D face structure from a single 2D image and compares it with a reference model for obtaining a compact representation of face features deviations. This descriptor is subsequently deformed using a “measure locally, weight globally” strategy to resemble the caricature drawing process. The deformed deviations are incorporated in the 3D model using the Laplacian mesh deformation algorithm, and the 2D face caricature image is obtained by projecting the deformed model in the original camera view. To demonstrate the advantages of caricature-based face recognition, we train the VGG-face network from scratch using either original face images (baseline) or caricatured images and use these models for extracting face descriptors from the LFW, IJB-A, and MegaFace data sets. The experiments show an increase in the recognition accuracy when using caricatures rather than original images. Moreover, our approach achieves competitive results with the state-of-the-art face recognition methods, even without explicitly tuning the network for any of the evaluation sets.