SP-Net: A Novel Framework to Identify Composite Sketch

Identifying composite sketches with digital face photos is an important and challenging task for law enforcement agencies. It has attracted a wide research interest in the face recognition area. In this paper, we present a novel framework that identifies the photo corresponding to a given composite face sketch. A coupled deep convolutional neural network, named Sketch-Photo Net (SP-Net) is proposed, which is fed with a positive or negative photo-sketch pair. In the proposed SP-Net, the customized VGG-Face network is adopted as base model and is followed by two branches, namely S-Net and P-Net, for sketch and photo, respectively. The S-Net and the P-Net are able to learn discriminative features between the sketches and the photos, regardless of the appearance gap by introducing the concept of elastic learning. In other words, to extract the most important features from the input, the network needs to learn the relevant features along with the irrelevant ones. To do so, higher dimension layers are used after the three 512 layers from VGG-FaceNet. Since the network learns representative features, we decrease the dimension of the layers to produce the most representative features. In addition, contrastive loss is employed to discover the coherent visual structures between sketch and photo. Experimental results on E-PRIP face sketch dataset indicate that the proposed network significantly outperforms the state-of-the-art composite sketch identification methods.