Improving Satellite Imagery Masking Using Multitask and Transfer Learning

Many remote sensing applications require masking of pixels in satellite imagery for further analysis. For instance, estimating water quality variables such as suspended sediment concentration (SSC) requires isolating pixels depicting water bodies unaffected by clouds, their shadows, terrain shadows, and snow and ice formation. A significant bottleneck is the reliance on multiple data products (e.g., satellite imagery and elevation maps) and lack of precision in individual processing steps, which degrade estimation accuracy. We propose a unified masking system that predicts all necessary masks from harmonized landsat and sentinel (HLS) imagery. Our model leverages multitask learning to improve accuracy while sharing computation across tasks for added efficiency. In this article, we explore recent deep learning architectures, demonstrating that masking performance benefits from pretraining on large satellite imagery datasets. We present a range of models offering different speed/accuracy tradeoffs: MobileNet variants provide the fastest inference while maintaining competitive accuracy, whereas transformer-based architectures achieve the highest accuracy, particularly when pretrained on large-scale satellite datasets. Our models provide a 9% $F1$ score improvement compared to previous work on water pixel identification. When integrated with an SSC estimation system, our models result in a 30× speedup while reducing estimation error by 2.64 mg/L, allowing for global-scale analysis. We also evaluate our model on a recently proposed cloud and cloud shadow estimation benchmark, where we outperform the current state-of-the-art model by at least 6% in $F1$ score.