Ensemble of Deep Belief Network and Bayesian Adaptive Aggregation for Regression

Ensemble modeling of Neural Networks is a strategy where multiple alternative models (ensemble members) are constructed and then their forecasts are ensembled using various combination approaches. Ensemble of Neural Networks has proved the concept behind this strategy. Deep neural network offers potential opportunities to overcome traditional ensemble of neural networks. This paper proposes an deep belief networks ensemble (DBN). The ensemble members of DBN are constructed with different number of epochs Which results in superior generalization ability. The outputs of these DBNs are aggregated by a Bayesian model averaging method. The proposed Bayesian adopted ensemble of DBNs is evaluated on two benchmark data sets. Comparison of the proposed model is evaluated with simple averaging and single DBN over a number of forecasting measuring that shows better performance of the proposed model.