A Bayesian Optimization Approach of Ensemble and Decision Tree Learning Applied to Industrial Energy Consumption Prediction

This work contributes with a new approach for tuning hyperparameters of machine learning models, based on sequences of optimization studies based on an initial range of hyperparameters. Through the proposed methodology, each sequence of studies allows the delimitation of an optimal range of hyperparameters to be inserted and evaluated by a Bayesian optimization framework, Optuna, in search of better performance metrics for the model used. The technique developed in this work was applied for short-term electrical energy prediction, with 15-minute and 1-hour data, using energy consumption data from a steel industry. We used ensemble and decision tree learning models as predictors, including Random Forest Regressor, Support Vector Regressor and Cubist Regressor, which have already been used in the literature to predict energy consumption using the same database. In an unprecedented way, we used the XGBoost model as a predictor of energy consumption in the proposed context. The results obtained from each model surpassed the performance metrics previously obtained in the literature for the same prediction scenarios, even without the use of specific feature selection techniques or pre-processing. To predict the 15-minute and 1-hour energy consumption, we obtained a Root Mean Square Error of 0.175 kWh and 1.341 kWh for the test set, respectively, using the Cubist Regressor model.