DCT-Based Residual Network for NIR Image Colorization

Colorization of Near-Infrared (NIR) image is a challenging problem due to the lack of low-level clues available in the luminance channel of visible images. Recently, deep learning has witnessed remarkable progress in the NIR image colorization approaches. However, we have observed that most research focuses on designing deeper and wider architectures to improve the quality of RGB images at the expense of computational burden and speed. Few studies have adopted lightweight but effective modules to improve the efficiency of NIR image colorization without affecting its performance. In this paper, we propose the Discrete Cosine Transform (DCT)-based Residual Network (DCT-RCAN) for NIR image colorization. Specifically, the output of our network is four coefficients generated by the Two-Dimensional (2D) 4×4 DCT of RGB images, which reduces the training difficulty of our network by explicitly separating low-frequency and high-frequency details into four subgroups. We adopt the Residual in Residual (RIR) module as a basic module in our network, which can reduce the complexity of the model. Thus, our method can focus on more crucial underlying patterns in channel dimension in a lightweight manner. Extensive experiments validate that our DCT-RCAN is computationally efficient and demonstrate competitive results against state-of-the-art NIR image colorization methods.