Face Sketch Synthesis Using Regularized Broad Learning System

There are two main categories of face sketch synthesis: data- and model-driven. The data-driven method synthesizes sketches from training photograph–sketch patches at the cost of detail loss. The model-driven method can preserve more details, but the mapping from photographs to sketches is a time-consuming training process, especially when the deep structures require to be refined. We propose a face sketch synthesis method via regularized broad learning system (RBLS). The broad learning-based system directly transforms photographs into sketches with rich details preserved. Also, the incremental learning scheme of broad learning system (BLS) ensures that our method easily increases feature mappings and remodels the network without retraining when the extracted feature mapping nodes are not sufficient. Besides, a Bayesian estimation-based regularization is introduced with the BLS to aid further feature selection and improve the generalization ability and robustness. Various experiments on the CUHK student data set and Aleix Robert (AR) data set demonstrated the effectiveness and efficiency of our RBLS method. Unlike existing methods, our method synthesizes high-quality face sketches much efficiently and greatly reduces computational complexity both in the training and test processes.

I. Introduction

Face sketch synthesis has been widely applied in digital entertainment and law enforcement [1]–[3]. In terms of digital entertainment, everyone can become a painter and get easy access to sketches that are originally drawn by skilled artists. Another application is for assisting law enforcement. As the photographs of a criminal suspect are not available in most cases, the best way to find the suspect in a mug-shot database is to match sketches, which are drawn by skilled artists with the aid of eyewitnesses, with photographs. However, due to the different modalities between photographs and sketches as shown in Fig. 1, face sketch recognition encounters great difficulty. Thus, the solution is to transform them into the same modality. Fig. 1.

Examples of face photographs and sketches. (a) and (b) Photographs. (c) and (d) Corresponding sketches drawn by artists.