RKTrans: Transformer with Improved Residual Connection Units for Power Forecasting

Existing Transformer-based time series forecasting models fail to balance the requirements of power forecasting, including long-range dependencies capture, real-time forecasting, and retention of sequential information simultaneously. To address this issue, we propose RKTrans, a Transformer-based forecasting model that associates the residual network in the Transformer with Euler discretization of solutions to Ordinary Differential Equations (ODE) and replaces classic residual units with the Runge-Kutta block for accurate and real-time forecasting. Our research provides insights for effectively applying Transformer models to power forecasting. Furthermore, RKTrans surpasses existing state-of-the-art Transformer models on public datasets with less inference time. Our code and dataset are available at https://github.com/KongMLin/RKTrans.