Cross-Modality Multi-Task Deep Metric Learning for Sketch Face Recognition

Sketch face recognition is to match face sketch images to photo images. The main challenge of it lies in cross-modality differences. To address this challenge, a variety of methods were proposed to bridge cross-modality gap of different modalities. Specially, common subspace-based methods have achieved great performance in this task. These methods enable the data of different modalmes to be comparable by mapping this data into a new and common subspace. However, the problem of non-linear distribution of samples from different modalities has not been well solved by these methods. In this paper, we propose a cross-modality multi-task deep metric learning (CMTDML) approach to address this problem. Firstly, we design a two-channel neural network to extract non-linear features of photo modality and sketch modality, and the parameter sharing characteristics can reduce the differences of features between different modalities. Secondly, we develop the loss function to constrain the features in common space, where intra-class compactness and inter-class separability of features are promoted. In extensive experiments and comparisons with the state-of-the-art methods, the CMTDML approach achieves marked improvements in most cases.