Arbitrary-Resolution Hyperspectral Pansharpening Neural Operators

Arbitrary-resolution hyperspectral (ARHS) pansharpening seeks to enhance low-resolution hyperspectral (LRHS) images to any target spatial resolutions by collaborating with the connected panchromatic (PAN) images. The ARHS pansharpening task is significantly different from standard hyperspectral (HS) pansharpening, where ARHS pansharpening has to deal with the challenge that how to pansharpen upcoming LRHS images to any desired scales far beyond the training scales. Neural operators (NOs) form mapping between functional spaces and build continuous function relations, offering an alternative potential path to deal with ARHS pansharpening task. In this article, we are dedicated to the design of pansharpening NOs (PNOs) by the postulation that LRHS and PAN are observations of the input functional space while target high-resolution HS (HRHS) images are observed from the output functional space. Our PNOs are built with a paradigm comprising prepansharpening encoding, inverse discretization, and functional mapping subnetwork. More specifically, by use of kernel projection of frequency domain-related design and Galerkin design, we develop Fourier PNO (Fourier-PNO) and Galerkin PNO (Galerkin-PNO). In addition, we propose the concept of dynamic spectral-spatial Green’s function and accordingly develop an enhanced Fourier-PNO (eFourier-PNO), and further introduce a PAN-guided test space tactic to develop an enhanced Galerkin-PNO (eGalerkin-PNO). Finally, we discuss and explain some details such as the scale generalization ability of our PNOs. Our four PNOs are tested on three datasets, and the experimental results verify their excellent performances.