Deep Gradient Projection Networks for Pan-sharpening

Pan-sharpening is an important technique for remote sensing imaging systems to obtain high resolution multi-spectral images. Recently, deep learning has become the most popular tool for pan-sharpening. This paper develops a model-based deep pan-sharpening approach. Specifically, two optimization problems regularized by the deep prior are formulated, and they are separately responsible for the generative models for panchromatic images and low resolution multispectral images. Then, the two problems are solved by a gradient projection algorithm, and the iterative steps are generalized into two network blocks. By alternatively stacking the two blocks, a novel network, called gradient projection based pan-sharpening neural network, is constructed. The experimental results on different kinds of satellite datasets demonstrate that the new network out-performs state-of-the-art methods both visually and quantitatively. The codes are available at https://github.com/xsxjtu/GPPNN.

1. Introduction

Multispectral images store multiple images corresponding to each band (or say, channel) in an optical spectrum, and they are widely utilized in literature of remote sensing. With the limitation of imaging devices, satellites how-ever often measure the low spatial resolution multispectral (LRMS) images [4], [21], [29]. Compared with the multispectral image, the panchromatic (PAN) image is characterized by the high spatial resolution but only one band. Lots of satellites carry both multispectral and panchromatic sensors to simultaneously measure the complementary images, such as Landsat8, GaoFen2 and QuickBird. To obtain the high resolution multispectral (HRMS) image, a promising way is to fuse the complementary information of the LRMS image and the PAN image. This technique is called as pansharpening [4], [21].