Structural Combination of Neural Network Models

Forecasts combinations normally use point forecasts that were obtained from different models or sources ([1], [2], [3]). This paper explores the incorporation of internal structure parameters of feed-forward neural network (NN) models as an approach to combine their forecasts via ensembles. First, the generated NN models that could be part of the ensembles are subject to a clustering algorithm that uses the structure parameters and, from each of the clusters obtained, a small set of models is selected and their forecasts are combined in a two-stage procedure. Secondly, in an alternative and simpler implementation, a subset of the generated NN models is selected by using several reference points in the model structure parameter space. The choice of the reference points is optimised through a genetic algorithm and the models selected are averaged. Hourly electricity demand time series is used to assess multi-step ahead forecasting performance for up to a 12 hours horizon. Results are compared against several statistical benchmarks, the average of the individual forecasts and the best models in the ensembles. Results show that the cluster-based (CB) structural combinations do better than the genetic algorithm (GA) structural combinations in outperforming the average forecast, which is the traditional point forecast from an ensemble.