IProbeTrans: A Long-Term Series Forecasting Method Based on Self-Supervised Learning

Long-term time series forecasting (LTSF) is essential for domains like meteorology and finance, requiring accurate predictions over many time points. While Transformer-based models have shown promise in capturing long-term dependencies, they face challenges due to potential information loss and quadratic scaling of time complexity, impacting efficiency. Thus, this work proposes a forecasting method called iProbTrans, which uses inverted embedding and probsparse self-attention to capture temporal representations and correlations between variables. It outperforms transformer-based methods on multivariate datasets, reducing MSE and MAE by 9.6% and 6.6%, respectively, compared to the FEDformer baseline. A self-supervised learning approach is proposed, decomposing time series into patches and applying random masking to reduce computational costs. This model, trained on mean squared error loss for mask reconstruction, improves upon supervised learning by reducing MSE and MAE by 26.3% and 27.5%. It excels in capturing long-term dependencies, enhancing generalization, and offers a solution to the limitations of Transformer-based methods in LTSF.