Energy Scheduling Framework of Micro-grids Considering Battery Lifetime

The stochastic and intermittent nature of renewable energy poses grave challenges to the energy scheduling of micro-grids. The exploitation of batteries has been considered as an effective way to cope with such challenges. However, fluctuation in renewable process causes batteries to charge/discharge frequently, reducing the lifetime and increasing the maintenance cost. This paper presents a hierarchical two-layer energy management, a combination of battery and supercapacitor, to lessen daily energy costs. The upper layer is equipped with a battery to minimize the economic cost, optimizing the energy usage by employing a mixed-integer nonlinear programming (MINLP) model; the lower layer is equipped with a supercapacitor to treat the uncertain nature of renewable energy. The optimization process includes load shifting and battery degradation costs. The upper layer determines the optimal schedule of interruptible appliances and the profile for energy storage for the next 24 hours. The schedule results are then passed to the lower layer, which deals with the forecast uncertainties with a supercapacitor with a 15 minutes interval. The effectiveness of the proposed algorithm is examined by a single-layer scheduling system without forecast errors and a two-layer scheduling system with forecast errors. The obtained results show the capability and effectiveness of the proposed scheduling system to reduce operating costs and forecast errors.


INDEX TERMS
T HE sustained growth of energy demand and gas emissions has become a crucial issue of world concern. One of the promising ways to solve their related problems is to exploit renewable energy resources. Under this circumstance, the concept of micro-grid has aroused lots of interest. A micro-grid is composed of renewable energy resources, energy storage systems, and some controllable loads. Such advanced technology can enhance energy efficiency and security by balancing the energy supply and energy demand.
Designing a productive energy scheduling strategy is a critical part of the micro-grid system. The investigations on energy scheduling methods have stimulated great interest [1]; nevertheless, uncertainty such as forecasting error is inevitable for energy scheduling, which increases the difficulty of energy control. The battery storage system, which acts as bidirectional mediators to improve energy efficiency with the utility grid, is often integrated into scheduling to deal with uncertainty [2]- [8]. Thus, optimization approaches including robust optimization [2], chance-constrained optimization [3], and scenario-based stochastic optimization [4], [5] have been implemented for renewable energy optimization problems considering uncertainty. However, the literature mentioned above narrowly considered the economic effects of the realtime battery system operation under different resources, loads, and environmental conditions. In fact, the real-time battery system operation has a significant impact on the battery lifetime in the long term. For instance, the battery life would be seriously shortened by the frequent charging and discharging. On the other hand, to eliminate the uncertain factors, the energy management system mainly relies on the real-time battery system operation.
In order to reduce the influence of the real-time operation on battery life, one feasible tactic is to combine some high-power energy systems with long life and fast response characteristics to construct a hybrid energy storage system (HESS). Supercapacitors have very high cycle lives and safe application properties. Therefore, one of the hot topics of current research has been application of hybrid batteries and supercapacitors as an effective solution to the HESS [9,10]. The battery-supercapacitor-based HESS can reduce the effect of errors and extend battery life through a reasonably coordinated control strategy. The general control strategy is that the battery supplies the long time part of the power, while the supercapacitor supplies its short time part. This control strategy significantly lessens the charge and discharge times of the battery to extend its lifetime. This paper presents a hierarchical two-layer micro-grid energy scheduling framework that combines an optimization layer and a real-time control layer to deal with the differences between an ideal scheme and a real operation. The micro-grid is built with the following sorts of equipment: interruptible load, must-run loads, PV, wind turbines, batteries, and supercapacitors. The hierarchical micro-grid energy scheduling framework minimizes micro-grid energy costs while also considering the battery degradation costs, uncertainty, and opportunity for load shifting. The hierarchical framework consists of two main layers: (a) the upper layer (long-time scheduling without uncertainty), and (b) the lower layer (realtime controller layer). The upper layer optimizes micro-grid energy usage by employing a mixed-integer nonlinear programming (MINLP) optimization. The lower layer consists of a real-time controller layer that controls the supercapacitor while minimizing the generated cost by the upper layer. The main contributions of the present work are summarized as follows: 1) A two-layer energy scheduling system including a hybrid battery and supercapacitor for micro-gird is created. An effective energy scheduling is planned to minimize the long-term costs of the upper layer; further, the prediction uncertainty and energy fluctuations are eliminated in the lower layer.
2) A non-convex MINLP model, which considers the battery lifetime and interruptible appliances, is built within the upper layer. Additionally, an algorithm is proposed to seek a near-optimal solution for the model's problem.
3) A lower layer strategy is developed to accommodate the imbalanced power mainly due to the uncertainty of the renewable energy and loads, while extending the lifetime of the battery.
Compared with other micro-grid energy system publications considering layered and multi-layer, this paper proposes an active error eliminate method to evaluate the impact of uncertainty timely and provides cost-effective real-time optimal measures after occurring uncertainty. It is basically different from the double-layer structure proposed in the literature [12], [13]. In these studies, uncertainty was handled by underlying real-time optimization. Although their method could effectively deal with the uncertainty, they ignored the impact of this operation on battery life. In contrast, supercapacitors are exploited to make up for this deficiency.
The approaches associated with the micro-grid energy system [11] commonly propose a two-layer energy management system for the micro-grid with hybrid batterysupercapacitors considering degradation costs. The main novelty of the proposed micro-grid energy system is that the structure contains a large number of interruptible equipment, which is more in line with the real situations. Additionally, the optimization model has been modified from the original mixed-integer linear program (MILP) to the MINLP.
Until now, derivative-based methodologies such as nonlinear branch and bound approaches as well as outer approximation methods [14] and derivative-free ones such as evolutionary algorithms [15]- [18] have been established for solving MINLP problems. A hybrid optimization method integrating derivative-free and derivative-based (branch-andbound algorithm) is proposed to solve the problems associated with the MINLP. The proposed hybrid optimization method makes full use of the flexibility of the derivativefree in handling the non-convexity and the efficiency of the interior-point algorithm for large NLP.
The remaining of the paper is organized as follows. The second section presents the smart buildings that integrate different production and consumption systems. In Section three, mathematical formulas with all the constraints associated with each equipment are developed. The fourth section discusses optimization procedures that can be utilized to optimize the demand response programming, and a proposed hybrid optimization algorithm that deals with the MINLPbased problems. In section five, simulation results are discussed, and finally, the main obtained results are briefly explained in the sixth section.

A. SYSTEM ARCHITECTURE
The proposed micro-grid model has been depicted in FI-FURE 1. The micro-grid system mainly consists of electrical interruptible loads, PV, wind turbine, battery storage system, supercapacitor, electric grid, and must-run load. The leading purpose of energy scheduling is to control the load for minimizing the financial costs by considering the physical characteristics of appliances.

B. MODELS 1) Modeling of the battery degradation cost
The battery is employed to avoid energy mismatch between the demand and power generation; however, improper cycling can drastically deteriorate the charging and discharging performance of the battery [19]. The battery lifetime is mainly affected by several cycling conditions, such as weather, charging and discharging rates, and depth of discharge.
Actually, the depth of discharge essentially impacts the battery lifetime compared to other cycling conditions. The cost function of the battery can be formulated as [11], [19]: (4) where a and b represent the curve-fitting coefficients, DOD batt (depth of discharging) denotes the energy in one charging or discharging event to the full capacity, I c (DOD batt ) is defined as the lifetime of the battery, which is relative with DOD batt .

2) Modeling of the supercapacitor degradation cost
In general, the life of supercapacitors is expected to last until the estimated lifetime under normal operation conditions, which is not limited by cycling stress [11], [20]. Despite the DOD batt of each charge/discharge event, the degradation cost of the supercapacitor is time-dependent. Therefore, the supercapacitor degradation cost for any time interval can be formulated as follows: As can be seen from Eq. (5), the supercapacitor cost is proportionally related to the time interval.

3) Constraints a: Constraints of battery
The battery charging and discharging energies can be formulated by Eqs. (6)-(10) where the maximum amount of power charged and discharged by the battery storage during the time t are represented by p max and p min , respectively. The charging and discharging rates of the battery are given by Eqs. (6)- (7) while Eq. (8) is utilized to calculate the amount of energy transferred to the battery. Based on Eq. (9), the SOC of the battery at t + 1 is a function of the SOC at time t m, and the inequality relation in Eq. (10) is exploited to limit the range of the SOC. Interruptible appliances were assumed to only operate with either an 'on' or 'off' status at a fixed energy level, which was characterized as follows: Eq. (11) involves operation within a preferred time range [α a , β b ], while Eqs. (12) and (13) describe an interruptible appliance that completes slots during a cycle.

c: Constraints of supercapacitor
Supercapacitors can eliminate instantaneous power imbalance through frequent charging/discharging. Eq. (14) introduces that the power generated from the supercapacitor must be limited by the upper and lower bounds. The given constraints in Eqs. (15) and (16) present the supercapacitor from being overcharged and over-discharged.

III. ENERGY SCHEDULING STRUCTURE
The proposed two-layer structure of the DR programming is depicted in FIGURE. 2. The factors T u and T l in order denote the length of the prediction horizon in the upper and lower layers. The upper and lower layers are the predictive model controller with the time interval ∆T u and ∆T l , respectively. The upper layer obtains the optimal value at the current time t, and then the lower layer reuses the reference values provided by the upper layer for its own optimization. After ∆T u time ∆T u , the upper layer starts for the next ∆T u with the updated state variables, which are sent back by the lower layer.

A. OPTIMIZATION STRUCTURE OF THE UPPER LAYER
The main objective of the upper layer is to minimize the total operational cost by optimizing t decision variables [P grid (t u ) , P batt (t u ) , P def e (t u )] Tu tu=1 . The optimization problem in the upper layer can be formulated as follows: The electricity cost consists of grid cost and battery degradation cost. As it can be seen, the upper layer represents an MINLP. This is an NP-hard problem that can be solved by the proposed hybrid algorithm. FIGURE 2: The structure of two-layer energy scheduling.

B. OPTIMIZATION STRUCTURE OF THE LOWER LAYER
The objective of the lower layer is to eliminate fluctuations induced by uncertainty in the EMS. The optimization of the second layer can be formulated as: Eq.(1) − Eq.(10). Eq.(14) − Eq.(16). variables: [P batt (t l ) , P sc (t l )] where σ b denote weighting coefficients. The values P batt (t u ), optimized by the first layer are exploited as the reference values of the second layer. In order to reduce the deviation from the reference of the battery, the penalty term C B (t) is formulated and added as a quadratic function:

IV. OPTIMIZATION ALGORITHMS A. PROPOSED ALGORITHM FOR THE UPPER LAYER
The upper layer is a mixed-integer nonlinear problem with constraints. Both gradient-free and gradient-based algorithms are employed to solve such an optimization problem. The gradient-based optimization method is more suitable for examining the problems with smooth design space and convex near the optimal value. Therefore, it is unsuitable for the MINLP-based problems with non-convex design space and integer variables. The gradient-free optimization method does not require the design space to be continuous or smooth, making it easier to solve the MINLP problem. However, due to the lack of mathematical optimality conditions, the efficiency of the gradient-free optimization method is relatively low. For the upper layer here, we propose a hybrid algorithm that takes advantage of both gradient and gradient-free. The gradient-free algorithm is carried out to determine the initial variables. The obtained results from the gradients-based algorithm can be used as the starting point of the gradientbased algorithm. The gradient-free optimizer minimizes the chance of finding a local minimum in the nonlinear design space, and the gradient-based optimizer exactly pinpoints the optimum location.
1) The gradient-free algorithm We choose the particle swarm optimization method as the gradient-free algorithm. The particle swarm optimization (PSO) algorithm has been successfully applied to various fields of study to solve different optimization problems. The population of the PSO is designated as the swarm, with each particle in the swarm representing a solution to an optimization problem. Each particle moves within the solution space to solve an optimum solution, with the movement of a particle affected by the exchange of information between individual particles. The velocity of the ith particle is defined by Eq. (20), where gbest t i is the global best of the swarm, while pbest t i is the so-called personal best of the particle, w is a weighting function that controls how much the velocity of a particle v t i affects the velocity of the next particle. The acceleration constants c 1 and c 2 , which are learning factors and usually are set as c 1 = c 2 = 2 [23]. r 1 and r 2 represent two diagonal matrices of random numbers generated from a uniform distribution between 0 and 1. The position vector x t+1 i of each particle can then be updated using Eq. (21): Kennedy and Eberhart [25] proposed a discrete binary version of the PSO to solve binary variables. The major difference between the binary PSO with the continuous version is that the speed of a particle coordinate is mapped to a probability using a sigmoid function by the following relation: and the new position of the particle is obtained using the equation below: where r is a uniform number within the range of 0 to 1.

2) The gradient-based algorithm
The MINLP can be expressed as follows where x = [x inte ] represent the binary variables and y = [y grid , y batt ] denote the continuous variables. When the initial values are specified by the PSO, then the function becomes a typical constrained nonlinear program (NLP).
We choose the interior-point gradient-based algorithm. This effective one can solve a sequence of nonlinear programming, which restricts the constraints into the objective function by creating a barrier function as follows: in which µ denotes a small positive scalar, the so-called barrier factor. The interior-point algorithm is a gradientbased type from Lagrange multipliers [25]. In this paper, the interior-point algorithm is implemented by the fmincon function. In the process of implementation, relative parameters such as the Hessian function, nonlinearly constrained tolerances, and minimum disturbances are involved.
3) The proposed algorithm 1) The interior-point algorithm is a fast way to obtain the optimal solution; however, its results rely on the chosen initial value. The PSO algorithm is an evolutionary one, which can generate a large number of individuals and can VOLUME 4, 2016 5 be analyzed simultaneously. It improves the probability of obtaining the global value. For optimization of the MINLP issue, the PSO is utilized to provide both discrete variables ([x inte ]) and continuous variables ([y grid , y batt ]), with a chart for the proposed strategy presented in FIGURE. 3. Initialize parameters and stopping criteria.
2) The PSO is initially applied to a set of continuous and discrete variables within a feasible region, sending values [x inte , y grid , y batt ] to the interior-point algorithm as an initial value for processing.
3) The interior-point algorithm then solves the continuous variables to obtain a new value x inte , y grid , y batt that is sent back to the PSO algorithm.
4) The PSO algorithm then deals with the discrete and continuous values obtained by the interior-point algorithm.
5) The fitness of each particle individual is then calculated to guarantee that the pbest and gbest particles are appropriately identified.
7) The iterative algorithm is continued until a satisfactory stop condition is achieved.

B. PROPOSED ALGORITHM FOR THE LOWER LAYER
The mathematical model of the lower layer is a nonlinear programming problem, which is solved using the solver IPOPT [22]. The IPOPT (interior-point optimizer, pronounced eyepea-opt) is a software package for large-scale nonlinear optimization. The IPOPT has been released as an open-source code under the Eclipse public license (EPL).

V. SIMULATION RESULTS
In this section, different scenarios configured to analyze the algorithms fitness for our purposes are examined. Firstly, a micro-grid structure was constructed, integrating PV panels, wind turbines, battery, supercapacitior, and interruptible appliances. The day-ahead price of electricity is adopted by weighting data from May 2013 to April 2014 in Energy Market Company of Singapore [11], [21]. The exact power of must run load, PV, and wind turbine is demonstrated in FIGURE. 4 [11]. The used parameters in the case studies are provided in TABLES 1 and 2. The parameters of the controllable appliance are assumed to be rand generated and shown in TABLE 1.

A. UPPER LAYER ALGORITHM ANALYSIS (SINGLE-LAYER SCHEDULING SYSTEM WITHOUT FORECAST ERRORS)
Further experiments were carried out to evaluate the performance of the proposed algorithm.
In order to analyze the influence of the initial value on the non-convex MINLP problem, the initial particle number of this experiment was tested from 1 to 30. The experimental results are presented in TABLE 3.
It can be seen from TABLE 3 that the fitness values decrease as the number of particles grows. The possibility    of obtaining the optimal solution increases with the growth of the number of particles. Therefore, the performance of the interior-point algorithm problem is more dependent on the initial value. Further experiments are carried out to assess the performance of the proposed strategy. The convergence characteristics of the proposed strategy with five particles are presented 6 VOLUME 4, 2016 in TABLE 4. It can be observed that with the development of the algorithm, its performance has been improved, and the solution has become superior. Therefore, the PSO-based algorithm can help the proposed algorithm to realize convergence better. Further performance analysis of the proposed algorithm with other algorithms has been published in the previous study [26].

B. SCHEDULING RESULT OF TWO-LAYER SCHEDULING STRUCTURE
The results of the scheduling schemes based on the time-ofuse pricing are presented in FIGURE. 5.
Additionally, the battery plays a critical role in saving financial costs. Batteries can offer assistance to reduce costs by charging at off-peak hours and discharging at peak hours. It can be seen that the battery discharges quickly within the peak hour periods of 17, 36, and 38. When the grid price is relatively lower at hours 25-30, charge the battery to guarantee that enough energy is discharged in the following time. Further, excessive energy is sold back to the main grid to reduce costs during peak price since the grid price is high.

C. PERFORMANCE ANALYSIS OF TWO-LAYER SCHEDULING STRUCTURE FOR FORECAST ERRORS
Due to the uncertain nature of the PV and wind turbine, a two-layer optimization model is proposed in the micro-grid program. For this purpose, the upper layer minimizes the financial cost, and the lower layer removes the forecast errors. In order to decrease the computational costs, only an inter-ruptible within the time interval (3:00-11:00) is considered in the upper layer. The forecast horizons of the upper and lower layers in order are 12 hours and 1 hour. The optimization results with increasing forecast errors from 10% to 40% are demonstrated in FIGURES. 7-9.
As shown in FIGURE. 8, the change of battery operation is more stable with the increase of the prediction error, whereas the output of the supercapacitor is enormously influenced by the forecasted error, as displayed in FIGURE. 9. This may be due to the forecast errors in the PV and wind turbines, which lead to the dramatic fluctuation of the supercapacitor. Since one of the main destinations of the lower layer is to smooth the output of batteries, it can be reasonably explained that increasing the power mismatch will result in more volatile variations in the output of supercapacitors, which is consistent with the obtained results in Ref. [11].

VI. CONCLUSIONS
A two-layer structure of the micro-grid energy scheduling accounting for the cost of battery degradation is proposed to eliminate the forecast errors. The primary duty of the upper layer is to minimize the operating cost, while the lower layer is exploited to eliminate the fluctuations caused by the prediction errors. In order to solve the two-layer energy scheduling structure for MINLP-based problems, an upper mixed integer nonlinear programming, hybrid gradient-free and gradient-based optimization algorithms are proposed. The analysis results indicate that the proposed hybrid algorithm can optimally solve the non-convex MINLP. Further research works have successfully demonstrated the effectiveness of the proposed two-layer structure in eliminating prediction errors.
YANTAI HUANG received the Ph.D. degree from Tongji University, China, in 2016. He is currently a Lecturer with the School of Automation and Electrical Engineering, Zhejiang University of Science and Technology. His current research interests include smart scheduling, home energy management system, heuristic algorithm, dynamic programming, intelligent automation.
QIANGQIANG ZHANG was born in Nanjing, Jiangsu, China, in 1997. He received the B.E. degrees from the Suqian College, in 2016 .He is currently studying for a master's degree in engineering at Zhejiang University of Science and Technology. His research interest includes Optimal energy scheduling systems.
MIN KANG received the Ph.D. degree in electrical engineering from Zhejiang University, China, in 2008. He is currently a Professor with the School of Automation and Electrical Engineering, Zhejiang University of Science and Technology. His current research interests include electrical control and power electronics related fields. VOLUME 4, 2016