Energy Demand Forecasting Model Based on Romember's Self-Attention Mechanism and Deep Learning

Power system transformation is crucial in a dual-carbon context. In this paper, we propose an optimized Transformer architecture, Romember model, which improves the accuracy of power consumption prediction by introducing rotational position coding. We construct a multidimensional evaluation framework to provide guidelines for the continuous optimization of the model. The Romember model contributes to balanced grid dispatch, optimal load allocation, and the promotion of energy green transition.