Revisiting Remote Sensing Cross-Sensor Single Image Super-Resolution: The Overlooked Impact of Geometric and Radiometric Distortion

In remote sensing, single image super-resolution (SISR) can be learned from large cross-sensor datasets with matched high resolution (HR) and low resolution (LR) satellite images, thus avoiding the domain gap issue that occurs when generating the LR image by degrading the HR one. Yet cross-sensor datasets come with their own challenges, caused by the radiometric and geometric discrepancies that arise from using different sensors and viewing conditions. While those discrepancies can be prominent, their impact has been vastly overlooked in the literature, which often focuses on pursuing more complex models without questioning how they can be trained and fairly evaluated in a cross-sensor setting. This article intends to fill this gap and provide insight on how to train and evaluate cross-sensor SISR deep learning models. First, it investigates standard image quality (IQ) metrics robustness to discrepancies and highlights which ones can actually be trusted in this context. Second, it proposes a complementary set of frequency domain analysis (FDA)-based metrics that are tailored to measure spatial frequency restoration performances. Metrics tailored for measuring radiometric and geometric distortion (GD) are also proposed. Third, a robust training and evaluation strategy is proposed, with respect to discrepancies. The effectiveness of the proposed strategy is demonstrated by experiments using two widely used cross-sensor datasets: Sen2Ven $\mu $ s and WorldStrat. Those experiments also showcase how the proposed set of metrics can be used to achieve a fair comparison of different models in a cross-sensor setting. The code will be publicly available at https://github.com/Evoland-Land-Monitoring-Evolution/sisr4rs.git