Camera-specific image quality enhancement using a convolutional neural network

We propose a simple method to enhance the image quality of modern Bayer pattern based cameras that are offering an optional sub-pixel accurate sensor shift to capture full rgb images of static scenes. By capturing a series of image pairs of the same, unaltered scene, once captured with the Bayer pattern and once with the full rgb image data, a database of corresponding images can be generated in which the former contains artifacts resulting from the spatial interpolation which is absent in the latter. Using this data, we train a convolutional neural network (CNN) to generate a camera dependent image processing operation which reduces the image artifacts and enhances the image quality to approximate the quality of the full rgb image within a single exposure, even if moving scenes are captured. We present a simple, do-it-yourself method to capture and pre-process the data, train the network, and to enhance the images. An evaluation using several image quality assessment methods shows the effectiveness of the proposed method.