Comprehensive quality assessment of optical satellite imagery using weakly supervised video learning

Identifying high-quality (i.e., relatively clear) measurements of surface conditions is a near-universal first step in working with optical satellite imagery. Many cloud masking algorithms have been developed to characterize the likelihood that reflectance measurements for individual pixels were influenced by clouds, cloud shadows, and other atmospheric effects. However, due to the continuous density of the atmospheric volume, we argue that quantification of occlusion and corruption effects is better treated as a regression problem rather than a discretized classification problem as done in prior work. We propose a space-time context network trained using a bootstrapping procedure that leverages millions of automatically-mined video sequences informed by a weakly supervised measure of atmospheric similarity. We find that our approach outperforms existing machine learning and physical basis cloud and cloud shadow detection algorithms, producing state-of-the-art results for Sentinel-2 imagery on two different out-of-distribution reference datasets. The resulting product offers a flexible quality assessment (QA) solution that enables both standard cloud and cloud shadow masking via thresholding and more complex image grading for compositing or downstream models. By way of generality, minimal supervision, and scale of our training data, our approach has the potential to significantly improve the utility and usability of optical remote sensing imagery.