Electrical Power Consumption Forecasting with Transformers

Until recently, state of the art (SOTA) deep learning methods for time series prediction problems, such as electricity load forecasting, have been based on recurrent neural networks (RNN), convolutional neural networks (CNN), or combinations thereof. However, RNNs involve sequential computations that cannot be parallelized on machine learning accelerators, while CNNs require very deep networks to capture long distance relationships. This paper proposes a sparse transformer based method for time series prediction. The proposed model achieves comparable accuracy to the SOTA method, TSRNN, on the London Smart Meter dataset while achieving up to 10 times faster inference speed.