Grayscale Portrait Colorization using CNNs and Pretrained VGG-Face Descriptor

Colorization problem is a process of adding colors to a grayscale image. Traditionally it requires human-labeled color scribbles on the grayscale image and the image is colored by propagating the scribbled colors throughout the image using optimization techniques. It is not very long ago that colorization without the intervention of human was almost impossible. Recently colorization using Neural Networks has achieved big success due to the availability of huge datasets. This paper attempts to color only portrait images using Convolutional Neural Networks along with a pretrained VGGFace descriptor as a global feature extractor. It has been shown that using separate networks, one to extract 3x3 patch sized features or local features and another to extract global features, result in higher quality of colorization outputs. In portrait colorization, we used VGGFace descriptor as the global feature extractor that encodes a grayscale portrait into a vector representation which later can be used to train the system to have overall coloring effects on the grayscale portrait using fusion technique. To measure the colorfulness of the output image we introduce the use of colorfulness metric that has the potential to capture the perceptual colorfulness essence from the output images. To evaluate the model performance we used Mean Squared Errors (MSE) and Mean Absolute Errors (MAE).