Are Deep Learning Models Practically Good as Promised? A Strategic Comparison of Deep Learning Models for Time Series Forecasting

Multivariate time series forecasting problem has attracted enormous research in recent years, and many deep learning models have been proposed and claimed to be effective in different tasks. We find that many of these models were tested in a simple one-step-ahead strategy, which does not apply to real scenarios requiring multistep forecasting. This paper compares the performance of three well-known deep learning models (DA-RNN, LSTNet, and TPA-LSTM) for multivariate time series forecasting under three forecasting strategies (One-Step-Ahead, Recursive, and MIMO) for multistep forecasting. We conducted our experiments on six datasets (NASDAQ 100 Stock Data, Beijing PM2.5 Data Set, Electricity, Exchange Rate, Solar Energy, and Traffic) with four forecasting horizons (3, 6, 12, and 24). Our results reveal that, under the Recursive strategy, these deep learning models constantly suffer from accumulated errors and thus cannot carry out real multistep forecasting tasks. However, combining them with the MIMO strategy can tackle this problem and thus enables one-step-ahead deep learning models for multistep forecasting.