Study and Impact Analysis of Data Shift in Deep Learning Based Atmospheric Correction

Atmospheric corrections (AC) factors in adverse atmospheric effects while determining surface reflectance (SR) from remote sensed satellite images. Deep learning (DL)-based methods using only remote-sensed images are capable of learning nonlinear relations between the top of the atmosphere (TOA) and the bottom of the atmosphere (BOA) to provide consistent SR estimates. The changes over time in atmospheric conditions, geographical terrains and sensing processes has caused significant changes in the statistical properties of satellite images affecting the nature of the relationship between TOA and BOA, commonly referred to as a dataset shift. If this phenomenon is significant then over time the statistical properties of the data used to train DL models are vastly different from the test data for which we are performing AC. This distributional mismatch can cause DL models to perform poorly due to its inability to extend temporally. In this paper, we aim to ascertain the presence of data set shift using Landsat 8 images from 2013 and 2020. We create a joint distribution between the TOA and BOA for temporally separated training and testing sets, and study data set shift from three perspectives; 1. Use the Kolmogorov-Smirnov test to measure the distribution changes. 2. Apply Wasserstein distance over differences in TOA and BOA to measure the numerical drift directly. 3. Using the DL model to identify the drift and study model generalization. All three approaches indicate a dataset shift and provide strong evidence for its presence.