Enhancing the transformer-based forecasting models by training datasets permutation

Transformer-based machine learning models are gaining in popularity across a range of applications. In this investigation, we explore the potential of this architecture for time-series-based forecasting, with a particular focus on assessing the benefits of permuting multiple datasets during the training process. Our research utilizes a collection of 5 different datasets collected from electricity trading markets. We selected three datasets that were tested and verified that we could substantially improve the performance of the prediction model. The mean absolute errors for all three markets decreased by a minimum of 12.5%, shifting from 6.4C, 5.4C, and 3.1$ for the models based on a single dataset for the Belgian, French, and American markets to 4.2C, 4.7C and 2.7$ for models established using three datasets, respectively. Furthermore, for all three markets tested, the final transformer configuration resulted in superior metrics in comparison with the prepared baselines, exhibiting enhanced MAE, RMSE, and SMAPE scores when evaluated against Naıve and Vanilla models.