Pandemic-Aware Day-Ahead Demand Forecasting Using Ensemble Learning

Electricity demand forecast is necessary for power systems’ operation scheduling and management. However, power consumption is uncertain and depends on several factors. Moreover, since the onset of covid-19, the electricity consumption pattern went through significant changes across the globe, which made the forecasting demand more challenging. This is mainly due to the fact that pandemic-driven restrictions changed people’s lifestyles and work activities. This calls for new forecasting algorithms to more effectively handle these conditions. In this paper, ensemble-based machine learning models are utilized for this task. The lockdown temporal policies are added to the feature set in order to make the model capable of correcting itself in pandemic situations and enhance data quality for the forecasting task. Several ensemble-based machine learning models are examined for the short-term country-level demand prediction model. Besides, the quantile random forest regression is implemented for a probabilistic point of view. For case studies, the models are trained for predicting Germany’s country-level demand. The results indicate that ensemble models, especially boosting and bagging-boosting models, are capable of accurate country-level demand forecast. Besides, the majority of these models are robust against missing the pandemic policy data. However, utilizing the pandemic policy data as features increases the forecasting accuracy during the pandemic situation significantly. Furthermore, the probabilistic quantile regression demonstrated high accuracy for the aforementioned case study.


I. INTRODUCTION
Electricity power infrastructure transfers a huge amount of energy to end-users in a fast, clean, and reliable manner. It is imperative to provide a reliable power supply in today's modern times as brief power outages of even a few minutes or less may have significant consequences for energy users. As such, generating, transporting, and delivering electrical energy remain complex and costly. Moreover, electrical power, unlike other forms of energy, cannot be stored in any substantial amount [1]. Thus, power generation must match total consumption. However, dispatchable generation units are subject to inter-temporal constraints (e.g., ramp rates).
The associate editor coordinating the review of this manuscript and approving it for publication was Guangya Yang . Therefore, the generation of these units should be scheduled in advance. This requires a precise forecast of the uncertainties associated with generation, transmission, and consumption ahead of time by which the day-ahead market is cleared. An accurate prediction of these uncertainties mitigates the need for expensive storage and increases grid flexibility.
The uncertainties associated with generation and transmission can be addressed through renewable power generation forecasting [2]- [4], and dynamic line rating [5]. On the other hand, demand is also another source of uncertainty in power grids [6]. Demand forecasting assists system operators in performing unit commitments and evaluating the stability of the power system. The more forecast in precise, the less real-time dispatch correction is required. In light of the electricity market's intense competitiveness, load forecasting may offer important information to aggregators when engaging in energy trading and dynamically controlling power demand [7]. This paper focuses on precise forecasting demand to enable system operators to make more rational decisions for the day-ahead market. This category of demand forecasting is known as short-term demand forecasting, as opposed to long-term demand forecasting, which is commonly used for resource planning problems. Based on the forecasting time horizon, the existing literature can be categorized into four groups [8]: • Very short-term load forecasting aims at forecasting load from a few seconds to a few minutes in advance.
• Short-term load forecasting targets to predict demand from a few minutes to a few hours ahead, which plays a vital role in power systems' operation.
• Medium-term load forecasting aims at predicting load from a few hours to a few months ahead.
• Long-term load forecasting targets a longer duration of demand forecasting, e.g., several years. This prediction is used for system expansion and planning studies.
Although demand forecasting is necessary for the secure operation of power systems, it is proved to be a difficult task as customers' electricity demand is uncertain to a great extent. Load profiles vary extensively with changes in weather conditions, time, and special occasions. To cope with these uncertainties, several approaches/methods have been developed by researchers. For example, an additive partially linear model is presented in [9] aiming at predicting daily electricity consumption. Alongside the improvement in computation capability of processors and Machine Learning (ML) and Deep Learning (DL) techniques in computer science, electrical engineering researchers find out the benefits of deploying ML and DL for demand forecasting. For example, [10]- [12] developed K-Nearest Neighbors (KNN), Support Vector Regression (SVR), and Artificial Neural Network (ANN) as data-driven demand forecast models at New York state, Northeastern China, and Pecan street, respectively. A deep convolution neural network is presented in [13] for demand forecast in three Chinese cities. The authors in [14] introduced a stacked hybrid machine deep learning-machine learning regression for predicting the city-level demand. Convolutional long short-term memory (ConvLSTM) integrated with bidirectional long short-term memory (BiLSTM) is used in [15] for predicting residential and commercial consumption. Dilated CNN based multi-step forecasting model is developed for the same purpose in [16].
One of the overlooked concerns in most ML-based load forecasting approaches is the robustness of the method against non-typical situations. An example of a non-typical situation is data quality degradation due to a cyber-attack or missing data. To cope with the abnormalities, a joint neural network and Gravitational Search Optimization approach for load forecasting is deployed in [17] for load and price prediction. A Long Short-Term Memory (LSTM) neural network model was presented in [18] for short-term demand and price prediction. The method was trained and test for Spanish and Pennsylvania-New Jersey-Maryland power markets. Authors in [19] developed an LSTM integrated with Convolution Neural Network (CNN) to forecast the Bangladesh power system's demand. Variational Mode Decomposition (VMD) and SampEn (SVDM) decomposition method is adopted in [20] to decompose the demand series to the summation of a trend series and a set of fluctuating sub-series. Then, a linear regression model is used for predicting the trend series, and Extreme Gradient Boosting (XGBoost) is used for predicting sub-series. In [21], the state-space prediction general problem is modeled with DL and then applied to probabilistic load forecasting.
Accurate forecasting under atypical situations has remained a consistent challenge in the previous studies as many forecasting algorithms fail to make a precise forecast facing abnormal situations. Experimental data shows that demand prediction in the recent pandemic situation went through a considerable error [22]. Therefore, the aforementioned forecasting models would fail to predict the demand accurately in this specific situation. Thus, a robust model capable of predicting demand in lockdown situations is missing in the literature. Reference [22] analyzes the load profile of three states of New York, California, and Florida in the Covid-19 lockdown situation. After careful investigation, they concluded that the demand ramp rate decrease in these states. As a result, demand pattern changes compared to the pre-pandemic situation. Authors in [23] compared Spain, Italy, Belgium, and UK's consumption patterns during the pandemic situation with the same interval in 2019. It has been observed that the demand profile changes depending on the policy of the government during the pandemic. Similar studies in [24] and [25] for Brazilian and Poland electrical energy demand demonstrated that the load decreases during the pandemic. The same has been concluded for Spain in [26]. The changes in residential, commercial, and industrial section demand in Lagos Nigeria have been studied separately in [27]. Though industrial and commercial consumption decreased, residential consumption increased. The references above analyzed the consumption trends changes during the Covid-19 tragedy. Reference [28] states that the Ontario State's electricity demand decreased by 25 percent in some days during April 2020 compared to the same days in 2019. The increase in total energy consumption while the peak demand is constant in Warsaw residential demand is demonstrated in [29]. Authors in [30] studied the demand change in the pandemic situation in several countries. Next, a conceptual framework of the temporal effect of lockdown measures on energy consumption has been developed.
All of the mentioned references in the previous paragraph adhere to the fact that a pandemic like the 2020 Covid-19 would dramatically change the electricity consumption patterns. However, none of them has presented a demand forecasting method for the pandemic situation. Although the planet is increasingly recovering from the current pandemic due to universal vaccines' availability, this is not a safeguard VOLUME 10, 2022 against future pandemics in the case of other viruses. Based on the above discussion, it is of great importance to develop pandemic situation-aware demand forecasting models. In this paper, we aim to develop decision tree ensemble-based pandemic-aware short-term load forecasting for the first time. First of all, it is discussed what factors affect the demand value, those which would be considered as the features of the demand forecasting models. Then, it will be investigated how to consider the pandemic situation in features. After that, we used decision tree ensemble-based models for 16-40 hours ahead demand forecast. Since the probabilistic demand forecast provides additional information about demand uncertainty, the quantile regression is deployed for load forecasting tasks in addition to deterministic ensemble models. The performance metrics of the models are evaluated for Germany's country-level demand forecasting. To summarize, the main contribution of this paper is as follows: • Analyzing the impact of covid-19 pandemic and lockdown measures on the energy consumption.
• Comparing the accuracy of several representative load forecasting models, including deterministic and probabilistic, bagging, boosting and bagging boosting decision tree ensembles, in the normal and pandemic no-history situation.
• Analyzing and evaluating the resilience of developed models for Germany's country-level demand forecasting in time of lockdown.
• Improving the performance of forecasting models by including lockdown temporal policies as new features.

II. METHODOLOGY
This study aims to forecast electricity demand 16 to 40 hours in advance by utilizing ML algorithms. Of all the ML models in the field, decision tree ensemble-based ML models are selected as it is highly capable of learning uncertainties and variabilities associated with load profiles, thereby providing high accuracy and generalizability. Besides, they are usually trained fast, which offers the opportunity to re-train it as we receive new electricity consumption data.

A. FEATURES
Every ML-based forecasting task requires to identify the variables that affect the target variable. Electricity consumption depends on several factors. The first influential factor that contributes to the value of demand is the time of use. The load demand at 1 p.m. is more than that of 4 a.m. due to the fact that most of the people sleep and do not have any activities at 4 a.m. Besides, the day of the week highly affects the load, such that load in weekends is less than the weekdays. Fig. 1 shows the electricity demand of four successive weeks [31] demonstrated in blue, orange, green, and red. This figure provides an illustrative example of the dependency of the demand on the hour and day. This figure reveals that the demand is substantially higher is day-time peak hours. In contrast, it decreases considerably at night when people are not awake. The daily peak demand decreases on weekends, especially on Sundays. Nevertheless, this figure also reveals slight differences between the same hours of the same days of different weeks. Therefore, the time is not the only influential factor. Weather factors are important as well. Electrical cooling or heating appliances have different consumption in different weather conditions. For example, refrigerators consume more demand during warmer days. The weather variables' actual values are not known 16 to 40 hours ahead necessitating weather condition forecasting. Another feature would be the special holidays such as Christmas. These occasions change the profile of the load due to the changes in people's activities. These features enable the model to predict the demand in typical situations. However, in pandemic situations, the trends of consumption go through significant changes. Therefore, it is necessary to add some features representing the changes in activities. If exact information is not available, a binary variable of lockdown situation is added to the model. Nevertheless, a single binary variable cannot fully represent the pandemic situation. If further information of detailed actions is available, each of the actions can be a feature. For example, the remote educational and academic activities reduce the daytime demand, whereas closing nightly public hobby centers lower the night demand.

B. ENSEMBLE-BASED MODELS
A decision tree [32] is a model that divides the feature domain into several sub-domains by binary branching, as shown in Fig. 2. In each sub-domain, a value is predicted for the output variable. The number of branches from the root to terminal roots is called the depth of the tree. The more the depth of the decision tree, the more the tree is capable of learning from the training set. However, big values for depth jeopardize the generalizability of the model. To minimize bias/variance and maximize the model's accuracy/precision, generalizability and robustness, decision tree ensembles are presented combining the prediction of different learners in parallel or serial. They can be categorized into three groups of Bagging, Boosting, and Bagging-Boosting [32], [33].  Bagging models average the output of several independent estimators that are trained on bootstraps of the original dataset in order to decrease the variance. To further clarify, consider that n sets of B 1 , B 2 , . . . , B n are independently generated as bootstraps of the training set T by sampling with replacement. The estimators M 1 , M 2 , . . . , M n are built according to (1). For predicting the value of the target variable, equation (2) is used. Fig. 3 demonstrates the bagging ensemble structure.
Random Forest is a more sophisticated form of Bagging in which not only training datasets are bootstraps of the original dataset, but also the features of each estimator is a bootstrap of the original feature set.
Boosting models use several serial decision trees, each aiming at compensating the error of the previous models. Therefore, the bias of the final model decreases in comparison to a single decision tree. This structure is mathematically formulated in (3) to (4). Fig. 4 also provides a visual representation of the boosting ensemble structure.
There are several boosting methods including Adaboost, Gradient Boosting, XGBoost, Light GBoost (LGBoost) and CatGboost. AdaBoost algorithm trains new predictors sequentially to compensate the error of previous predictors with more emphasis on instance with higher error. In Gradient boosting (GBoost), gradient decent algorithm is combined with AdaBoost learning algorithm. Extreme Gradient Boost or XGBoost is an improved version of GBoost which possess higher performance level and scalablity. An approximate  algorithm is utilized to find the best split point for continuous features. By implementing built-in regularization, it is also less prone to over-fitting [34].
LGBoost is another version of GBoost with higher accuracy. This algorithm exploit histogram to bucket continuous variables into several discrete bunch. As a result, this model trains pretty fast. Benchmark LGBoost implementation demonstrate 11 to 15 times faster than XGBoost. Besides, leaf-wise growth training procedure provides more accuracy [34]. Categorical boosting (Cat-Boost) is another version of gradient which boost can handle the categorical data as well. This algorithm can also be implemented on GPU [34]. XGBoost, CatBoost, and LGBoost's performances are high and very close.
Unlike bagging that provides a less variance model with competence in noisy data, boosting is less biased with competence in noise-free data. If both bias and variance need to be enhanced, a combination of bagging and boosting methods, which is called bagging-boosting, should be adopted. Fig. 5 provides a graphical illustration of ensemble models' performance compared to a single decision tree.

C. EVALUATION METRICS
Among several regression accuracy metrics, Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Root Mean Squire Error (RMSE), and R 2 -Score metrics can demonstrate the accuracy decently [5], [35]. MAE is defined as the average absolute value of error in (7). On the other hand, MAPE metric computes the average value of relative error in (8). RMSE measures the root of the average squire error according to (9). Since RMSE is the root of Mean Squire Error (MSE) and root is a monotonic function, it is no longer necessary to compare and compute MSE. Besides, RMSE has similar dimension and unit with the forecasted variable. VOLUME 10, 2022 Therefore, it provides more insight to the performance of the model. Finally, the R 2 -Score is an indicator of prediction to the observation. In these formulas, y i is the actual value of the N th sample, y i is the value of the N th sample's output, andȳ is the average value of the observed output over samples.

D. QUANTILE REGRESSION
Traditionally, regressors aim at finding a model that estimate the expected value of the dependent variable conditioned by the independent variables as asserted in (11) [36].
However, this single point estimation would not be sufficient in many engineering applications. In the case of load forecasting, it would be more instructive for the system operator to know his or her level of confidence in the estimation. Therefore, probabilistic regression model would be beneficial to indicate the most probable demand interval. Defining the conditional probability function as represented in (12), the quantile regressors aim at finding the value which is more than the output variable with the probability of α. This value is called α-quantile, and it is mathematically defined as (13). [36].
The loss function is defined in according with (10). In the training stage, the model is trained in a way in which the expected value of loss is minimized. The loss function can also be used for test evaluation. For the sake of space limitation, we refuse to delve into details of the training and structure of a quantile regressor. The respected reader can read [36] in order to study more about quantile random forest which is used in this paper.

III. CASE STUDIES
To evaluate the suggested forecasting framework's accuracy, robustness, and generalizability, the case of Germany's national-level load [37] prediction is investigated in this section.

A. PREPROCESSING
Before delving into training the machine learning models, it is of high importance to preprocess the raw data and analyze the main characteristic of the data. Maximum, minimum, average, and standard deviation of the Germany electricity demand consumption [37] is summarized in Table 1. The data contains the demand for each 15-minute interval. If a value is missing from the dataset, the average of the preceding and next samples is used to replace it. Only demand data until 26 th September 2020 was available at the time this paper was written. As Fig. 6 reveals, the scaled demand for the same days of year in 2020 is less than 2017 till 2019, especially for the days 100 to 200 when people's activities were strictly limited. In order to take the effects of these restrictions into account, pandemic lockdown actions should be considered as extra features for demand prediction. Since the actions data of Germany during the pandemic is provided in [38], a one-hot encoding binary feature variable vector is added for each action. Lockdown actions during 2020 are shown in Fig. 7. It is observed that several policies in different intervals have been regulated during the pandemic situation, and the level of lockdown differs from time to time. Generally speaking, three levels of highly restrict, restrict, and moderate lockdowns have been established in Germany. Each policy is modeled with a one-hot encoding vector. Fig. 7 depicts that restriction policies became partial in May and afterward. Then, most of the restrictions have been abandoned after July. Fig. 8 depicts the demand decrease for the first Monday of April, last Monday of May, and first Monday of August from 2019 to 2020. As it is observed, the demand decreases considerably. The greatest drop in demand occurs on day one, when extremely restrictive regulations are implemented. Days 2 and 3, which correspond to the limited and moderate policies, experience load decrease in the following level.
The following stage of preprocessing scales all characteristics and the output (demand) between zero and one.  The training set is comprised of demand data from 2017 to September 2020, with the exception of one month in 2019 and one month in 2020, which serve as the test sets.

B. CASE 1
In the first case, short-term demand forecasting models are trained using decision tree ensembles for two different feature sets. Set I contains the regular features and lockdown policy binary variables and Set II consists only the regular features. Train and test scores, reported in Table 2, demonstrate that boosting models outperformed baggings in terms of MAE, RMSE, MAPE and R 2 -Score metrics while all of them are trained well. Results indicate an enhanced performance in baggings by adopting boostings as their base learners, i.e. bagging-boostings. It can be seen from Table 2 that CatBoost, Light Gradient Boosting, Bagging XGBoost, Bagging CatBoost, and Bagging LGBoost provide close performance metrics. The base tree learner's depth is seven. Moreover, Fig. 9 corroborates the fact that including lockdown policies as additional features leads to better forecasts. Fig. 10 compares trained decision tree ensembles for forecasting Germany demand, where higher performance of boosting and bagging boosting models can be distinguished from their counterparts.

C. CASE 2
Load forecasts are inherently associated with uncertainty which makes it crucial to estimate and communicate this  uncertainty to forecast loads so that power systems operators can make optimal decisions under uncertainties like pandemic. Dissimilar to deterministic forecasting, probabilistic forecasting offers further information to measure the uncertainty associated with electricity demand. Hence, the Quantile Random Forest probabilistic model is deployed in this case for load forecasting tasks in addition to deterministic ensemble models. The interval between 97.5% and 2.5% for a day in test interval is shown with green color in Fig. 11. The average quantile loss for these two quantiles are 3017.232 MW and 1572.392 MW for the 2020 test set. Normalizing these two values with the average value of demand in this test set are 0.057 and 0.030; respectively. These low values indicate that the high performance of the probabilistic model. Approximately 99.22 percent of the demand values are between 97.5%-quantile and 2.5%-quantile. It is even more than 95% percent, which is desired.

D. CASE 3
The COVID-19 pandemic and consequent lockdown measures have led to an unprecedented decrease in the global electricity demand. The lockdown policies depressed commercial and industrial sector electricity consumption and increased electricity demand in the residential sector, creating an overall change in the shape of daily load profiles. Thus, this VOLUME 10, 2022   case addresses the resilience of forecasting systems against this form of uncertainty. In this regard, seven days of a test period in 2020 were examined, which affected by covid-19 pandemic. As Table 2 reveals, XGBoost and AdaBoost result in the best and worst forecasts when trained with feature set II. However, to improve the resiliency of models against the uncertainty attributed to the pandemic, it is necessary to consider lockdown policies as demand forecasting features during the pandemic situation. Results detailed in Table 2 substantiate that adding the lockdown policies as features in the load predicting model change the MAE, RMSE, MAPE, and R 2 -score metrics −31.26%, −24.19%, −31.81%, 3.26% on average. Fig. 12 also verifies that the developed models trained with feature set I, LGBoost as a representative decision tree ensemble depicted in Fig. 13, can resiliently forecast the demand in time of lockdown. The interval between 97.5% and 2.5% for a day in this test interval is shown with green color in Fig. 12. The average quantile loss for these two quantiles are 4352.549 MW and 2729.605 MW for the 2020 test set. Normalizing these two values with the average value of demand in this test set are 0.083 and 0.052; respectively. Approximately 89.57 percent of the demand values are between 97.5%-quantile and 2.5%-quantile. It is only 5.43% percent less than 95% percent, which is desired.

IV. CONCLUSION
Motivated by the substantial change in power consumption during pandemic situation, this paper developed pandemic aware day-ahead demand forecasting model. According to these models, if enough information about different types of pandemic-related policies is available, each type of policy would be a feature. Otherwise, a binary variable indicating lockdown situation is added to the feature set. Ensemblebased models were used for the day-ahead demand forecasting task. The proposed forecasting framework has been implemented for Germany's power demand as the case studies. Results revealed that the models, especially boosting and bagging-boosting models, are accurate and robust. Moreover, quantile random forest has also been adopted for demand prediction task in both normal and pandemic situations. With small loss value, it has shown a decent performance.
The methodology is based on the lockdown situation which impacts business, commercial, and entertainments. These impacts are reflected in demand profile and demand profile prediction in consequence. Biology of virus in the future pandemic does not influence these items. Therefore, the model is extendable for future pandemics.
Although demand is considerably affected by pandemic situation, there are also several other variables that went through substantial changes. For research in this area could include market price and available flexibility forecasting in pandemic situation.