STCA-LLM: Spatial-Temporal Cross-Attention Large Language Model for Wind Speed Forecasting

Accurately forecasting wind speed is crucial for efficiently utilizing the renewable energy, stabilizing the energy system and advancing the progress of the decarbonization of our society. However, due to its inherently temporal volatility and intermittency, accurate wind speed forecasting in a wind farm is challenging. Recently, Large Language Models (LLMs) have demonstrated notable performance in abundant natural language processing and computer vision tasks. However, the conventional LLMs fail to learn the complex spatial and temporal correlations of the wind speed data at multiple turbines in a wind farm, which makes wind speed forecasting cant fully benefit from the significant breakthroughs of LLM. To fill in this gap, we propose a novel spatial-temporal cross-attention LLM framework for wind speed forecasting, namely STCA-LLM, composed of alignment phase, and fine-tuning phase. In detail, our contributions are given as follows. First, the alignment phase aligns the general-purpose LLM with task-specific data, i.e., training the LLM with wind speed data. Second, in the fine-tuning phase, two representation learning modules, i.e., convolution network and graph neural network (GNN) are respectively used to extract temporal features of intra time series in each turbine, and the correlation of inter time-series at multiple turbines in a wind farm. Moreover, the cross-attention module is innovatively proposed to establish the connections between spatial and temporal embeddings. Then, the spatial-temporal representation modules and the aligned LLM are fine-tuned in two-stage way. Finally, thorough experiments on real wind speed dataset demonstrate that our proposed STCA-LLM outperforms state-of-the-art time series forecasting models including Transformer-based models, spatial-temporal GNN-based models, and pertained LLM-based models. Our code is available at: https://github.com/Justinzzcj/STCA-LLM