A Blind Full Resolution Assessment Method for Pansharpened Images Based on Multistream Collaborative Learning

Pansharpening aims to fuse a high spatial resolution (HR) panchromatic (PAN) image and a low spatial resolution (LR) multispectral (MS) image to obtain an HR-MS image. However, due to the lack of the real HR-MS reference image, determining pansharpened image quality at full resolution (FR) has always been a contentious issue in the community. We propose a blind FR assessment method for pansharpened images based on multistream collaborative learning. The proposed method designs a Siamese framework to collaboratively learn the spatial, spectral, and overall quality of the fused image. The parameters of the feature extraction layer in the spatial and spectral evaluation models are frozen for the overall evaluation model, thus improving accuracy and convergence speed. The proposed method was comprehensively tested and verified based on a large-scale dataset consisting of 13 620 fused images obtained by six pansharpening methods with four different thematic datasets. Furthermore, a large-scale subjective evaluation dataset, in which each of the 13 620 fused images was assessed by 28 participants, was utilized to comprehensively validate the proposed method. The experimental results demonstrated the superior performance of the proposed method to other state-of-the-art quality assessments.