STANet: A Novel Predictive Neural Network for Ground-Based Remote Sensing Cloud Image Sequence Extrapolation

Cloud image sequence extrapolation plays an important role in ground-based remote sensing observation because allows the observation range to be extended in the spatiotemporal domain. Existing methods, primarily focus on characterizing spatial features and capturing temporal state transitions independently, ignoring the complex spatiotemporal dynamics of the real physical world, resulting in images extrapolated by them being less than expected. To break through this dilemma, we propose the Spatio-Temporal-Aware Network (STANet) for ground-based remote sensing cloud image sequence extrapolation. The method is a novel predictive neural network that deterministically and uniformly models the transient variations and cumulative trends embedded in cloud image sequences under the supervision of an attention mechanism to characterize complex spatiotemporal dynamics. The context-gated unit (CGU) is connected to the encoder and decoder, replenishing context features lost by downsampling while removing the “ghosting” effect prevalent in spatiotemporal prediction tasks. For the purpose of evaluation of the proposed method, a series of comparative experiments and ablation studies are conducted on our collected TSI-440-Sequence Dataset (TSISD) dataset. Experimental results indicate that the proposed method outperforms other existing methods.