Ringmo-SenseV2: Remote Sensing Foundation Model for Spatiotemporal Prediction Based on Multisource Heterogeneous Time-Series Data

The rapid development of remote sensing (RS) technology has generated a vast amount of heterogeneous time-series data from various sources, including drone videos, satellite time-series images, and multiobject trajectories. Effectively processing and analyzing this multisource heterogeneous data for accurate spatiotemporal prediction is crucial in fields such as environmental protection and disaster response. In this article, we propose a universal predictive foundation model named Ringmo-SenseV2 to learn the general evolutionary patterns of RS elements from massive heterogeneous data. Ringmo-SenseV2 features a mixture-of-heterogeneous-experts (MoHE) Transformer, which unifies the modeling of multisource heterogeneous time-series data. Additionally, to better capture the complex dependencies across different spatiotemporal locations, we introduce a hypergraph translator (HT), treating embeddings of different spatiotemporal locations as nodes and employing hypergraph convolution for information propagation. Furthermore, to enhance the model’s adaptability to different evolution speeds during pretraining, we implement the adaptive tube masking (AM) strategy, which controls prediction difficulty by adaptively setting mask proportions for sequences with varying evolution speeds. Extensive experiments demonstrate that Ringmo-SenseV2 exhibits outstanding performance across various RS prediction tasks. Further tests on scene graph generation for RS images showcase the model’s ability to extract image features, thereby enhancing image perception tasks.