A Review for Pre-Trained Transformer-Based Time Series Forecasting Models

Transformer-based models have proven their superiority against recurrent networks in time series forecasting. Enhancing transformer-based forecasting models via pretraining tasks is a novel approach in the literature. In this paper, we are reviewing the most recent papers about pretraining aspects of time series as well as pretraining tasks that are used in transformer-based architectures.