Ensemble Transfer Learning for Time Series Forecasting: a Sensitivity Analysis Framework for a Shallow Neural Network

Transfer learning applied to time-series forecasting is a current topic of high interest to the machine learning community. This paper addresses the need for empirical studies, as identified in recent survey papers that advocate the necessity of practical guidelines for TL approaches and method design for time series forecasting. This paper's main contributions include proposing a comprehensive sensitivity analysis framework and methodology for TL schemes, starting with the introduction of four novel TL performance metrics. Our experiments validate the usability of the methodology and offer insightful information using an MLP shallow network as a use-case scenario. The results of the experiments demonstrate the advantages of using ensemble techniques for time series forecasting over using single models in transfer learning. The experiments also offer empirical insights into various parameters that impact the transfer learning gain while raising the question of network dimensioning requirements when designing a neural network for transfer learning. The project aims to culminate in the design of a pre-trained model that can benefit practitioners in time-series forecasting.

I. Introduction

Time series forecasting, the task of predicting future values based on historical data, has gained substantial importance in diverse domains such as finance, energy, healthcare, and transportation, [1], [2]. Accurate forecasting enables businesses and decision-makers to anticipate trends, make informed decisions, and optimize resource allocation. However, the inherent characteristics of time series data, including temporal dependencies, non-stationarity, and noise make time series forecasting be regarded as a difficult problem in the field of mathematics and machine learning. Thankfully, deep learning models have garnered considerable attention, in this area, and are lauded for their ability to capture the stochasticity and complexity in time series data.