Multi-modal Image Super-resolution with Joint Coupled Deep Transform Learning

In this paper, we address the problem of multi-modal image super-resolution (MISR), which aims at improving the resolution of the target modality with the help of high resolution guidance image of another modality. A novel joint coupled deep transform learning framework (JCDTL) based on deep transform learning is proposed which combines the information from multiple modalities for achieving MISR. The formulation and the requisite solution steps are provided. Two publicly available datasets RGB/NIR and RGB/Multispectral are considered for performance evaluation. The proposed approach shows a considerable improvement in performance compared to the state-of-art techniques. Further, an average PSNR improvement of close to 2dB and 1.5dB on RGB/Multispectral and RGB/NIR datasets respectively is observed by increasing the number of layers from one to three.