CrossST: An Efficient Pre-Training Framework for Cross-District Pattern Generalization in Urban Spatio-Temporal Forecasting

Urban spatio-temporal forecasting is critical for modern urban governance, especially in traffic management, resource planning, and emergency response. Despite advancements in pre-trained models for natural language processing, challenges persist in urban spatio-temporal forecasting. Existing methods struggle to identify and generalize universal cross-district spatio-temporal patterns, while computational limitations hinder the extraction of complex patterns from large-scale data. In this study, we propose CrossST, an efficient pre-training framework designed to capture universal spatio-temporal patterns across large-scale, cross-district scenarios. Specifically, CrossST performs pre-training on various large-scale spatio-temporal datasets to learn and store diverse valuable patterns in its pattern bank. It captures temporal dependencies, including periodicity and trends, through frequency domain and time domain analysis, while leveraging graph attention mechanisms to identify dynamic spatial propagation patterns. During fine-tuning, a spatio-temporal disentanglement strategy separates universal patterns from diverse spatio-temporal patterns stored during pre-training, improving generalization to downstream tasks and enabling efficient cross-district knowledge transfer. Additionally, temporal information aggregation and spatial linear optimization strategies enhance CrossST's efficiency and scalability, significantly reducing computational costs. Extensive experiments demonstrate that CrossST outperforms state-of-the-art baselines, improving downstream task generalization while maintaining low computational overhead. The datasets and code are available at https://github.com/Aoyu-Liu/CrossST.