Colorization Using Neural Network Ensemble

This paper investigates into the colorization problem, which converts a grayscale image to a colorful version. This is a difficult problem and normally requires manual adjustment to achieve artifact-free quality. For instance, it normally requires human-labeled color scribbles on the grayscale target image or a careful selection of colorful reference images. The recent learning-based colorization techniques automatically colorize a grayscale image using a single neural network. Since different scenes usually have distinct color styles, it is difficult to accurately capture the color characteristics using a single neural network. We propose a mixture learning model representing the presence of sub-color-style within an overall image data set. We, therefore, ensemble multiple neural networks to obtain better color estimation performance than could be obtained from any of the constituent neural network alone. A two-step colorization strategy is utilized as an adaptive color style clustering followed by a neural network ensemble. To ensure artifact-free quality, a joint bilateral filtering-based post-processing step is proposed. Numerous experiments demonstrate that our method generates high-quality results comparable with state-of-the-art algorithms.