Temporal Fusion Transformer with Non-Intrusive Attention for Data-Driven Electricity Load Forecasting

Accurate and timely load prediction can provide reference for grid operation control, facilitating energy saving, emission reduction and fine management of grid dispatch. In this paper, we propose a data-driven electric load forecasting method that incorporates Transformer and Auto Regression (AR), where the Transformer model is responsible for predicting the nonlinear high-frequency components of load data, while the AR model predicts the linear low-frequency components, thus jointly improving the forecasting results. The encoder in the classical Transformer model is improved by adopting a hierarchical design. Through the way of “layer-by-layer distillation and multi-layer fusion”, the improved encoder can extract the cross-periodic features from the temporal data in a more focused manner. Furthermore, this paper proposes a non-intrusive attention mechanism, which allows the model to incorporate auxiliary information such as weather and holiday in a non-intrusive way. Under the premise of ensuring the relative independence of the primary information representation space corresponding to the load, using this attention mechanism, more reliable temporal dependence can be uncovered with the help of auxiliary information. The experimental results on real transformer bus load data show that the proposed method outperforms the existing methods in both Mean Square Error and Mean Absolute Error, which verifies the effectiveness of the proposed method.