A Bi-Level Framework for Optimal Energy Management of Electrical Energy Storage Units in Power Systems

This study presents a bi-level framework to obtain optimal energy management of electrical energy storage (EES) units in power systems by minimizing the operation cost of the system to satisfy daily demand and by maximizing the benefit of storage units’ owner. Two agents are considered consisting of the power system operator and the owner of EES units. The former seeks to determine the minimum operation cost of the system in providing the system load, while the latter seeks to provide its maximum profit. The power system operator has the option to supply energy by the thermal generation plants or from the storage units. The proposed bi-level model provides optimal operation strategies for both the EES owner in the outer level and the power system operator in the inner one. In other words, the decision making of the power system operator is taken into account explicitly in optimal charge/discharge scheduling of EES owner in the introduced bi-level framework. The introduced bi-level approach is applied to the IEEE RTS 24-bus network in order to assess the performance of the model.

INDEX TERMS Bi-level programming, energy storage unit, operation cost minimization, power system operator, profit maximization, power generation unit.

Indices: i, j
Bus indices g Thermal generation unit index s Energy storage unit index w Wind turbine index t Time interval l ij Line connecting node i and j Parameters: P min g,i /P max  Degradation cost of the s th EES located at the i th bus P r,i Rated power of the wind turbine located at the i th bus VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ s ci,i /s co,i /s r,i Cut-in/cut-out/rated speeds of the wind turbine located at the i th bus Variables: P g,i (t)/Q g,i (t) Active/reactive power generation of the g th thermal plant located at the i th bus in the t th time (MW/MVAR) P ij (t)/Q ij (t) Active/ reactive power transfer between the i th bus and the j th bus in the t th time (MW/ MVAR) P w,i (t) Active power generation of the w th wind turbine located at the i th bus in the t th time (MW) Voltage value/angle of the i th bus in the t th time P c s.i (t)/P disc s,i (t) Power charge/discharge of the s th EES located at the i th bus in the t th time (MW) E s (t) Energy storage of the s th EES located at the i th bus in the t th time (MWh) b c s,i (t)/b disc s,i (t) Binary variable for charging/ discharging operation of the s th EES located at the i th bus in the t th time s i (t) Wind speed at the i th bus in the t th time λ i (t) Marginal cost of the system at the i th bus in the t th time ($/MWh) α(i, t)/β (i, t) Dual variable for active/ reactive power balance constraint of the system at bus i in time t δ(l ij , t)/δ(l ij , t) Dual variable for minimum/maximum active power flow constraint of the line between buses i and j at time t φ(l ij , t)/φ(l ij , t) Dual variable for minimum/maximum reactive power flow constraint of the line between buses i and j at time t ϕ(g, t)/φ(g, t) Dual variable for ramp up/down constraint of the generation plant g at time t γ (g, t)/γ (g, t) Dual variable for minimum/maximum active generation capacity constraint of the unit g at time t π(g, t)/π(g, t) Dual variable for minimum/maximum reactive generation capacity constraint of the generation unit g at time t ρ(i, t)/ρ(i, t) Dual variable for minimum/maximum voltage value constraint of bus i at time t ω(i, t)/ω(i, t) Dual variable for minimum/maximum voltage angle constraint of bus i at time t

I. INTRODUCTION
The electrical energy storage (EES) systems have encountered substantial challenges in recent years considering the increased electricity demand and restrictions of such systems with a limited investment in network transmission expansion.
Moreover, high penetration of renewable sources, including wind and solar and the associated power production uncertainty have introduced operational issues for power systems [1]. EES systems are introduced as practical solutions for stabilizing the power supply to overcome such challenges and minimize peak demand cost. Such systems are effective in restraining power fluctuations that appeared due to the stochastic and intermittent nature of renewable sources [2], [3]. In addition, such systems are effective in reducing system imbalances [4], load shifting and reserves [5], and reducing optimal operation cost of the system [6]. Accordingly, the EES technology has been introduced as a practical solution for attaining power system stability by the US Department of Energy (DOE), which has been planned to be developed through the EES program (DOE OE/ESSP) [7]. Significant growth is predicted within the EES technology considering the above-mentioned advantages of such systems, for which remarkable efforts have been made in recent researches. Different researches have been conducted on the operation and planning of EESs in electrical energy systems, including optimal allocation of plant systems in distribution networks [8], [9], optimal energy storage management in an integrated system [10], [11], different energy market bidding strategies of storage systems [12], [13], and investigation of different energy storage technologies [14]- [16]. As an instance, the authors have investigated the role of Cryogenic energy storage on the operation of wind-based energy systems in [17].
EES systems are allowed to be coordinated in energy and ancillary services markets according to orders offered by the Federal Energy Regulatory Commission (FERC) [18]. Different strategies have been introduced for the cooperation of energy storage systems in the operation of energy networks and participating in electricity markets. A stochastic strategy is proposed in [19] for optimal operation of energy storage units owned by independent private investors. An optimal bidding framework is also proposed for the participation of EES in energy and reserve markets in day-ahead and hour-ahead markets, respectively. The proposed participation model for storage units in this reference does not treat the independent storage systems differently from other energy and reserve sources. The authors of [20] propose an optimal scheduling and planning model for EES units where long-term planning of storage units is considered to decide for investment planning for EES units. The charging/discharging schedule of the storage units is studied for the long-term period in order to achieve technical and economic benefits. In the same research, the optimal location, size, and power rating of storage systems are provided taking investment, operation and reliability cost into account. In [21], optimal operation control of EES in a grid-connected micro-grid has been investigated considering costs of energy usage, battery operation, and utility-oriented aims for peak load management. The authors have introduced a mathematical model in [22] for the operation of storage units with optimal market prices in prefect competitive environments considering special incentive pricing for the storage unit. The main objective of the study is maximizing the social welfare, where the effect of energy storage size and location on electricity market prices and the mentioned objective have been investigated. The authors have proposed a distributed home energy management scheme with EES units in [23], where a Stackelberg game framework is developed to deal with a bi-level problem with maximizing the microgrid's owner profit as well as optimizing the energy consumption of consumers. A bi-level model for obtaining optimal bidding framework of a merchant energy storage operator is proposed in [24], where both day-ahead and following days are considered. The proposed model in this reference aims to obtain the maximum profit of EES units using the market-clearing results provided by the lower level, where the objective of network operator is the maximization of social welfare. Likewise, a bi-level model is introduced in [18] for coordination of a storage owner in the energy and reserve market considering a price-maker EES system. The bidding framework of the storage unit is investigated in both the energy-only market and the joint energy and reserve markets. The authors have studied a risk-constrained bidding/offering framework for a merchant compressed air energy storage system in [25], where a bi-level information gap decision theory-based strategy is proposed for considering the uncertainty of forecasted price.
The participation of EES systems in real-time markets and pricing issues in the market for EES, as well as the role of EES on power system operation, are also investigated in recent studies. The authors in [26] have studied the role of EES systems in reducing the power system peak condition and congestion based on a robust model, where the uncertain parameters of state-of-charge, demand and renewable power have been considered. In [6], an operation and bidding optimization model has been proposed for virtual power plants quipped with EES systems and renewable resources, where the effect of demand response programs is studied in maximizing the benefit. A new stochastic methodology is proposed for optimizing the bidding and offering process of the pumped storage unit in the pay as bid markets in [27]. The participation of EES units in reserve and energy markets based on a detailed bid/offer process is investigated in [28] with the inclusion of degradation cost. An intelligent energy management system for obtaining optimal set points of a hybrid energy system containing renewable unit, EES and the grid is presented in [29], where various types of EES are evaluated. A market participation model for pumped hydro storage systems in the day-ahead regulation market is introduced in [30], where the optimal offering and bidding of the storage units are determined for maximizing its profit. A participation model of a price-maker EES is proposed in [31], where a method is applied for evaluating the role of EES on electricity prices in the New York state market. In this reference, hourly production and load price quota curves are utilized for modeling the price effect of EES operation in the market participation scheme of a price-maker EES.
Considering the aforementioned literature survey, it is observed that sufficient research studies have been undertaken to investigate the optimal operation of energy storage units in power networks or micro-grids. However, simultaneous optimal operation of energy storage units in power systems from the viewpoint of energy storage owner as well as optimal scheduling of power system from the viewpoint of system operator has not been addressed in recent literature to minimize the cost of the system in line with maximizing the benefit of storage systems' owner. Accordingly, in the present study, the authors consider that the power system operator can supply the power demand of the network using different energy sources, including privately-owned energy storage units. The system operator should analyze the potential benefits of thermal generation units and energy storage units in order to supply power demand with minimum operation cost. On the other hand, the owner of storage should find the optimal charge/discharge scheduling for maximizing his profit considering the cost of power charging from the network, the benefit of power discharging to the network and degradation cost. In other words, the proposed model is a bilevel framework, where the power system operator and the storage owner are two different agents of the upper and lower levels. The minimization of the system operation cost to meet daily power demand will be done at the lower level, and the maximization of the benefit of storage owners will be performed at the upper level. After defining the charge/discharge price of the units as the market-clearing price obtained by the lower level of the proposed bi-level model, the participation of storage units in providing system load is done. Figure 1 demonstrates the interaction of the two agents, which is considered as a bi-level programming problem (BLPP) in this study. The BLPP is modeled as a bi-level optimization process, including two optimization levels, where an objective function and a set of constraints define each level. Each level of the BLPP, known as the upper and lower level, is associated with an agent. This research considers two decision-making agents, including the storage unit owner and the system operator as the BLPP leader and follower, respectively. The principal contributions of this study are highlighted as follows: • A bi-level strategy is proposed for optimal operation of energy storage units in power systems, which minimizes the cost of the system satisfying daily power load and maximizes the benefit of storage units' owner • An optimal operation strategy for the energy storage owner is provided in the presented bi-level framework considering explicit decision making of power system operator in optimal charge/discharge scheduling of the energy storage owner • An equivalent single-level programming model is provided for the proposed bi-level model considering power flow equations of the system, ramp rate limitations of thermal generation plants, load profile of the system and electrical constraints of the energy storage unit The remainder of this study is categorized as: Section II provides the proposed bi-level model for the optimal operation of EES units in power systems. Section III provides the implementation and simulation results of the proposed bi-level model on a test system. Finally, the paper is concluded in Section IV.

II. PROBLEM FORMULATION
The proposed bi-level problem is provided in (1)- (23). The upper-level optimization problem, which is associated with the energy storage unit, aims at maximizing the profits obtained by the storage owner. The objective function of the storage unit is provided as profit maximization in (1), which can be stated as the revenues provided by selling the energy to the system minus the cost of buying the energy from the system and the degradation cost of the storage unit in the scheduling time horizon. The equality and inequality constraints of the EES units are provided in (2)-(6). The energy balance of the energy storage units is represented in (2), and the minimum and maximum operating limitations on charging/discharging power are presented in (3)-(4). The operation of the EES unit in charge/discharge/idle mode is considered in (5), and related constraints to the energy capacity of the storage units are provided in (6).
The lower-level optimization problem, which is associated with the power system operator, aims at minimizing the cost of supplying power demand of the system by thermal units and the storage unit owned by the independent section (i.e., owner of power storage systems). So, the objective function of the lower-level problem contains the summation of the operation cost associated with thermal units and power purchase cost from the storage systems as well as benefits from selling power to storage units. The cost minimization objective function of the power system operator, as shown in (7), is divided into two parts. The first part is related to power generated by the thermal units, and the second part is related to power transferred between the system and the storage unit. The objective function of the lower level can be calculated as the summation of the cost of power production of generation units, the cost of buying energy from storage unit minus the revenues provided by selling the energy. Equations (8) and (9) present active and reactive power balance of the system, respectively [32]. The equality constraints of active and reactive power flow limitations through the transmission lines between the system buses are represented in (10)-(13), respectively. Equations (14)-(23) represent the ramp down/up limitations, minimum and maximum operating limitations of active and reactive power produced by thermal units, power flow through the transmission lines of the system, voltage magnitude and phase angle of the buses, respectively. Dual variables of the equations related to the lower level are defined to characterize the single-level problem in the following. For instance, α(i, t) and β(i, t) are utilized to set the associated dual variables with (8) and (9), respectively.
Wind power generation is a function of the wind speed at each time is divided to three parts including speed between cut-in and rated speeds, speed between rated and cut-out speeds and speed lower than cut-in speed or upper than cut-out speed, which can be formulated as [5]: The equivalent single-level problem of the introduced bi-level model can be provided by replacing the lower-level problem given by (7)-(23), given by Subject to (2)- (6) and (8)- (23) and The first set of constraints defined by (2)-(6) and (8)- (23) proposes the primal feasibility conditions of the bi-level problem. The stationary conditions of the Lagrangian problem are given by (26)- (30), and complementarity conditions of the multipliers are represented by (31)-(40).

III. IMPLEMENTATION AND SIMULATION RESULTS
The presented bi-level framework for the optimal scheduling of energy storage systems in power systems is evaluated VOLUME 8, 2020 utilizing the IEEE RTS 24-bus system, which is depicted in Fig. 2. The system data has been adapted from [33]. The characteristics of power generation units of the studied network are inspired by [34]. The line data of the studied IEEE RTS 24-bus network are adopted from [33]. The proposed bi-level model needs the power system operator to make decision on the charge/discharge of the EES in terms of charge/discharge energy quantity at each time of the scheduling horizon. Three storage systems are assumed in the studied system with a capacity of 100, 200 and 200 MWh at buses 21, 8 and 19, respectively. The degradation cost of storage units are considered as 2 $/MWh [11]. Also, because the proposed model is a short-term scheduling for one day, and the amount of self-discharging rate for one day is very low, it is not considered for this reason The minimum stateof-charge (SOC) of the EES units has been considered as 0 MWh [18]. The charge/discharge cost of the EES units are short run marginal costs of the system. The characteristics for EES units are reported in Table 1. It should be mentioned that the optimal allocation of storage units is not addressed in the introduced framework; instead, it is considered that the storage planner has located the units according to recent studies. Three wind farms are considered in the studied network at buses 8, 19 and 21. The number of wind turbines with characteristic of Table 2, are 50, 75 and 100 located at buses 21, 19 and 8, respectively.
The proposed bi-level model is applied to the studied network to provide the optimal charge-discharge of the EES unit and optimal production scheduling of the power generators of a 24-hours daily time interval. The results for the proposed bi-level model have been carried out utilizing   the SBB that is a powerful solver provided in the General Algebraic Modeling System (GAMS) software [35]. SBB is a powerful tool for dealing with mixed-integer nonlinear optimization problems that is an integration of the standard Branch and Bound approach and some standard nonlinear programming (NLP) solvers. This solver can utilize NLP solvers available in the GAMS environment as sub-solvers. Also, the SBB obtains the minimum path instead of obtaining the minimum successor so no repetition is done during the process. Generally, the SBB solver inspects fewer subproblems during the optimization process and hence it is effective in reducing the optimization computational time. The power generation data of wind farms at buses 21, 19 and 8, is shown in Fig. 3.
The participation of storage units in supplying the demand of the system is accomplished considering the charge/discharge price of the units as the market-clearing price obtained by the lower level of the proposed bilevel model. Accordingly, the cost/benefit of energy charge/discharge is obtained by calculating the locational marginal prices of energy as the market-clearing price using the duality theorem for active power load balance equation (8). The daily energy charge/discharge prices for the EES located buses, including buses 8, 19, and 21, are  depicted in Fig. 4. As observed in this figure, the maximum charge/discharge price for all three buses is related to t = 9 h. Thus, it is expected that the EES units will discharge energy at this time interval to obtain more profit. In addition, the minimum energy price for all three buses is related to t = 24 h, which will result in charging energy by EES units at this time interval. The power energy production of the network generators during a 24-hour time interval has been demonstrated in Fig. 5. As seen in this figure, g1 participated more than other generation units in supplying the load demand of the system. The sum of power generation by thermal units as well as EES units and wind turbines satisfies the active power load demand of the system and power transmission losses between system nodes. The rampup and ramp-down limitations of the generation units have been considered, as can be seen in this figure.
The hourly energy dispatch, including charge/discharge energy and SOC of EES at buses 8, 19 and 21, are demonstrated in Figs. 6-8, respectively. As mentioned in Table 1, the EES units have 40, 40, and 20 MWh initial energy at buses 8, 19 and 21, respectively. The hourly energy dispatch of EES at bus 8 has been shown in these figures, which presents that the EES is fully charged at t = 6 h. It is obvious that all the EES units have been charged during t = 1 h to t = 7 h due to lower energy prices with respect to other time intervals. The energy charge/discharge of the EES units are in agreement with the low/high price of the energy in the studied network. Finally, the EES units have been charged at the end of the scheduling time horizon to receive the initial SOC for the next day.    The cost/benefit analysis of the studied system for three EES units has been demonstrated in Figs. 9-11, which includes the charging cost and discharging benefit of the EES units in the studied time horizon. In addition, degradation cost and total revenue of the EES units from participating in the load demand supply of the system have been investigated. The total revenue of the EES units has been   obtained considering the charge and degradation costs and profit obtained by energy discharge to the system. As seen in Fig. 9, the EES at bus 8 has been charged from t = 1 h to t = 7 h, where the energy cost is 0 $/MWs at t = 4 and 5 h, and it is -0.59078 $/MWh at t = 6 h. Accordingly, by charging the ESS, total revenue has become negative during such time interval. By increasing the energy price between t = 8 h to t = 10 h and between t = 12 h to t = 14 h, the EES unit has discharged the energy to the system and the revenue has increased during this time interval. The charge/discharge process of the EES unit continues until t = 24 h when the SOC reaches the initial value.
The charge/benefit analysis of the storage units is reported in Table 3 in terms of energy charge cost, energy discharge profit and degradation cost. It is obvious that the last line is obtained as the sum over the preceding ones for the owner of the EES units. Accordingly, the total profit of the EES units has been obtained, which is equal to $9681.06 for a 24-hours scheduling time horizon. In addition, the total operation cost of the studied network is obtained as $558556.778 for the 24-hour scheduling time horizon. The power balance of the studied IEEE-bus system is demonstrated in Fig. 12. It can result from this figure that the EES owner has maximized his own profit considering the participation in managing the load demand of the studied network. As it is obvious from this figure, the EES units have been charged in off-peak load demands, when the price of energy is lower than other time intervals. In addition, the EES units have participated in supplying on-peak load demands when the price of energy is higher than other time intervals.

IV. CONCLUSION
This study introduced a bi-level optimization model for obtaining optimal energy management of electrical energy networks, where the power system operator can select to meet the load demand by generators or from energy storage units. The proposed model includes the minimization of operation cost of the power network incurred by the system operator to satisfy daily load demand and the maximization of the benefit of storage units' owner in a single optimization problem. The optimal operation strategy for the energy storage owner is considered as the outer level, and the optimal operation for the power system operator is studied as the inner level of the proposed bi-level framework. The presented framework has been tested on the IEEE 24-bus network taking into account limitations of transmission lines capacity, ramp rate limits of thermal generators and load profile of the system. The obtained results satisfied the proposed bi-level model, which obtained a daily profit of $9681.06 for the storage owner and a daily operation cost of $558556.778 for the system operator. The operation charge/discharge decisions of the EES units have been adjusted based on existing circumstances of the power balance of the studied system. The provided results verify the application of the proposed bi-level model in providing optimal set points of the EES units and network components with the maximum profit of the EES owner and minimum operation cost of the network operator. The future works can be focused on the analysis of detailed systems of batteries such as chargers and integration of household energy storages as well as studying multi-objective models such as economic emission analysis of power system operation in the presence of EES.  BEHNAM MOHAMMADI-IVATLOO (Senior Member, IEEE) received the M.Sc. and Ph.D. degrees in electrical engineering from the Sharif University of Technology, Tehran, Iran. He is currently a Professor with the University of Tabriz, Tabriz, Iran, where he is also the Head of the Smart Energy Systems Laboratory. He is also working as a Senior Researcher with Aalborg University, Aalborg, Denmark. Before joining the University of Tabriz, he was a Research Associate with the Institute for Sustainable Energy, Environment, and Economy, University of Calgary, Canada. He has a mix of high-level experience in research, teaching, administration, and voluntary jobs at the national and international levels. He was a PI or a CO-PI in 20 externally funded research projects. He was a Supervisor or an Advisor for more than 20 Ph.D. thesis and 45 M.Sc. theses. He has been a Member of the Governing Board of Iran Energy Association since 2013, where he was elected as the President in 2019. He is also serving as an Editor or an Associate Editor for different journals, such as IEEE ACCESS, IET Smart Grid, and Sustainability. He is included in the 2018 and 2019 Thomson Reuters' list of the top 1% most cited researchers. His main research interests include integrated energy systems, renewable energies, microgrid systems, and smart grids.