Build General Pansharpening Model Through a Simple Sensor-Specific Model Selection Strategy

Deep-learning-based pansharpening methods have become a hot field in the remote sensing community. However, most existing deep learning pansharpening methods are conducted in a sensor-specific and supervised way. That is, pairs of panchromatic (Pan) and multispectral (MS) images acquired from a given sensor are fed into networks. The trained model can achieve a satisfactory fusion result only if the images are fused from the same sensor as the training images. The trained model from one sensor cannot be transferred to another. How to train a general model suitable for different sensors is a challenging problem. In this context, we propose a simple sensor-type detecting strategy, which can be integrated with any sensor-specific pansharpening methods and result in a general fusion framework across different sensors. More specifically, for a given deep-learning-based pansharpening method, sufficient Pan and MS image pairs from multiple sensors are fed into the network respectively, and models from multiple sensors are obtained to construct a model pool. Thereafter, a pair of local patches of MS and Pan from the test images are fed into existing sensor-specific models in the pool respectively. The pansharpening quality of outputs are evaluated with multiple fusing indices, such as quality without reference index (QNR). The model that achieves the best performance in terms of index is selected as the candidate model and further employs the images over the whole scene. To evaluate the effectiveness of the proposed framework, WSDFNet, a state-of-the-art pansharpening method was trained with GaoFen-1, QuickBird, and WorldView-3 datasets respectively. The QNR index is the optimal index for predicting the sensor type. With 100 images from three datasets as testing images, the correct accuracy rate of the statistical QNR index on sensor differentiation performance is 86, 69, and 62, respectively, which indicates this model is promising for distinguishing the sensor type. This proposed prediction strategy is simple and flexible, and it can be combined with existing deep-learning methods to form a complete fusion process without distinguishing the sensor type.