A Deep Error Removal Network for Pan-Sharpening

The phenomenon of nonoverlapping spectral responses is an inevitable but an overlooked problem in the deep-learning-based panchromatic (PAN) and multispectral (MS) images’ fusion task, which will introduce some error information from the PAN image. In light of this, we construct a novel prior model based on spectral response theory and develop a model-based pan-sharpening network. Specifically, we extract the initial error map from the PAN and interpolate the MS image as the initial pan-sharpened result. Then, two optimization problems regularized by the deep prior are formulated to update the error map and pan-sharpened image. By alternately optimizing the above subtasks, error information is gradually separated from PAN images and the lost texture information in MS images is gradually restored, which can effectively alleviate the negative impact of low coupling information from PAN and MS images. Plenty of experimental results on different kinds of satellite datasets demonstrate that the proposed method shows a better balance between interpretability and lightweight structure. The proposed method will be open-sourced in
https://github.com/jiaming-wang/DERN
.