PATCH-FILL: Multiscale Gap Filling for Earth System Data Cubes

Many applications in Earth system science require continuous, gap-free datasets. However, remote sensing data are plagued by gaps due to cloud obstruction, incomplete spatial coverage, and low-quality flags. Gap filling in remote sensing data with the help of machine learning often requires model architectures that are tailored specifically to underlying dataset characteristics such as scale, resolution, or range of values. This limits the transferability to other gap-filling scenarios. Training these models is further hindered by the lack of adequate training samples, as they must be gathered from gap-afflicted data themselves. In this work, we present a generic spatiotemporal, univariate, and multiscale gap-filling method. A modular implementation allows for the customization of system parameters so that the method can be adjusted and applied to different spatiotemporal datasets. By employing a patch-wise gap-filling approach, introducing masked loss functions, and providing effective methods for synthetic gap generation, we are able to leverage gap-afflicted datasets and gather large volumes of training samples from them. To demonstrate the flexibility of the system, we perform gap filling on multiple climatic variables from Earth system data cubes (ESDCs) using a 3-D convolutional neural network (3-D CNN) architecture, making this the first global-scale gap-filling solution on ESDC. By capturing both spatial and temporal relations, the model is able to generate predictions that are coherent on a large scale and across patches, thus demonstrating the potential of the patch-wise gap-filling framework and the use of 3-D CNN architectures for spatiotemporal gap-filling tasks. We validate the generalization capabilities of the model by performing spatial leave-one-out (SLOO) cross validation.