A Transactive-Based Control Scheme for Minimizing Real-Time Energy Imbalance in a Multiagent Microgrid: A CVaR-Based Model

Power systems are undergoing significant transformations due to the introduction of microgrids (MGs) and distributed energy resources (DERs). In this regard, MGs as new entities in the system facilitate the integration of independently operated DERs to the power system. In this structure, DERs would be operated by independent agents, while the microgrid's control unit (MCU) would coordinate agents’ resource scheduling to maximize social welfare. In this structure, the uncertainty of resources operated by agents could lead to real-time energy imbalance in the MG resulting in agents’ loss of profits. Consequently, a novel transactive-based-scheme is developed in this article to facilitate flexibility service exchange between the agents to minimize the real-time energy imbalance in the MG. In this context, MCU provides independent agents with bonuses as transactive signals to exploit their operational scheduling while addressing privacy concerns. The proposed framework increases the overall social welfare as well as minimizes the dependence of the MG on the upper-level system to ensure the demand–supply balance in the MG. Finally, this scheme is implemented on an MG with a multiagent structure to investigate its effectiveness in minimizing the real-time energy imbalance in the system.

A State of charge of EVs in dayahead scheduling for agent k at t ' in scenario st.

I. INTRODUCTION
L ATTERLY, by unprecedented integration of distributed energy resources (DERs), including renewable energy sources (RESs) and local flexible resources [i.e., conventional distributed generations (CDGs), flexible load demands, and electric vehicles (EVs) along with energy storage systems (ESSs)] in power systems, significant challenges have been raised in distribution networks. Moreover, due to the power system privatization in recent years, DERs are typically operated by independent agents in local systems [1]. In this regard, the traditional power designs need to be restructured to facilitate the installation of the local resources operated by independent agents. As a result, microgrids (MGs) are introduced as entities responsible for the efficient management of independently operated DERs in order to facilitate their integration into power systems [2]. In this respect, as a new and practical concept in modern grids, MGs play a significant role to modify the existing systems to future smart grids that are bounded in a localized area in the distribution level [2], [3].
Considering these facts, to effectively enable the operation of an MG composed of independently operated agents, a novel approach should be developed in which each agent pursues its own objectives (i.e., agents' total cost minimization). In this structure, the MG's control unit (MCU) would also strive to efficiently coordinate agents operational scheduling in order to maximize the overall social welfare. Consequently, in terms of the utility's as well as independent agents' perspectives, MCU can facilitate the efficient integration of independently operated local resources into the power system. In this new structure, agents schedule their local resources in day-ahead and real-time markets to maximize their profits, whereas MCU enables flexibility service exchanges between agents. It should be noted that due to uncertainties associated with local resources (i.e., RESs and load demands), energy imbalance could exceedingly be imposed on the MG scheduling during the real-time optimization [4]; therefore, agents could face real-time high market prices for ensuring demand-supply balance. That is why, a novel control management framework seems to be required to enable flexibility exchange among agents in the real-time operational management. In this vein, new-developed control management concepts, such as transactive energy (TE) [5], [6], could be employed to incentivize rescheduling of independently operated flexible resources in order to decrease the real-time energy imbalance in the MG.
Liu et al. [7] present a hybrid stochastic/robust bidding strategy in scheduling of MGs considering uncertainties made by intermittent DERs, load variation, and market prices to minimize total costs in mixed-integer linear programming (MILP). In this context, forecasted scenarios are presented to efficiently handle uncertainties associated with decision parameters in the dayahead operational optimization. To minimize the electricity and dis-satisfaction costs, a multiagent home energy management is presented in [8] by considering electricity price and solar photovoltaic (PV) uncertainties. In [2] and [3], a hierarchical multiagent energy management strategy (EMS) and a distributed robust EMS are developed for the operational management of MG systems. In this regard, the provided scheme in [2] strives to maximize the usage of RESs in the system, while Liu et al. [3] aim to minimize the total MGs' cost in the real-time energy market by considering uncertainties of RESs and load demands, respectively. Violante et al. [9] present a novel MILP EMS model based on the predictive control (MPC) for the operational management of an isolated MG with the aim of considering uncertainties of thermal units while minimizing fuel costs. To coordinate the MG's net-load and mitigate the net-load ramping, a flexibility-oriented MG optimal scheduling model is presented in [10]. Furthermore, a coordinated energy dispatch based on MPC is presented in [11] to enable exchanging energy between the distribution network operator and MGs as well as improving RESs utilization. Moreover, to minimize the electricity purchase costs, the conditional value at risk (CVaR) optimization model is deployed in [12], which show the application of CVaR concept in modeling the operational risks associated with the uncertainty of decision parameters in energy systems. Papadaskalopoulos et al. [13], [14] have studied the participation of flexible demands in day-ahead market in a decentralized manner using Lagrangian relaxation principles, which show the importance of flexible demands in peak demand alleviation and operation of the system. However, this approach has not studied the energy imbalance minimization in real-time operation of a multiagent MG. Moreover, the application of TE as well as the stochastic programming, CVaR, and MPC technique are not investigated in the developed approaches in [13] and [14].
In [15], a two-stage robust stochastic programming model is proposed to maximize expected profits and minimize the imbalance cost in the real-time balancing market considering optimal MGs scheduling coupled with RESs' uncertainties. Li et al. [16] present a novel bilateral TE trading framework to handle technical as well as economic issues in energy trading by considering detailed designs for PV penetration. To minimize the aggregated MGs' operational costs, a multiperiod distributed TE scheme with the aim of preserving information privacy is proposed in [17]. In [18], a real-time EVs charging optimization in a TE scheme is suggested to handle the uncertainties of RESs and maximize the profits. This article merely focuses on activating EVs' flexibility by providing the upper control unit with the changes in their power requests in case of receiving a TE signal. As other TE research articles, in order to maximize profits for the residential buildings, a multiagent TE scheme is presented in [19] to preserve consumers' privacy with full decision-making authority. Finally, Tavakkoli et al. [20] present a bonus-based model to activate the participation of heat demands to provide flexibility services for wind power aggregators. In this article, the proposed framework is dependent on the exchanging information of operational characteristics associated with the heat demands to the aggregator by the purpose of modeling the Stackelberg game and determining the bonuses associated with the contribution of each agent.
Based on the previous studies in efficient energy management of an MG, coordination of multiagent systems, and real-time energy imbalance optimization in power systems, efficient operational coordination of independent agents to minimize the real-time energy imbalance in an MG has not yet been investigated in previous research works. It is noteworthy that the coordination of independent agents in this structure is conducted by MCU, which is a nonprofit entity and just strives to facilitate exchanging flexibility services among the agents to minimize energy imbalance in the MG. Therefore, to well address the above-mentioned research gap, a novel approach is presented in this article to minimize the energy imbalance in an MG composed of multiagent components utilizing the TE concept. In this regard, the developed framework leads to limit the information exchange between MCU and flexible resources operated by independent agents to address the increasing privacy concerns while improving the system flexibility.
In the developed scheme, each agent independently operates its local resources, while the MCU strives to coordinate their scheduling by the purpose of maximizing social welfare. In this regard, TE signals are devised to be offered to independent agents operating flexible resources to minimize the energy imbalance costs in the real-time operation of the multiagent MG. The cost of balancing real-time supply and demand will be minimized under an iterative TE-based approach in which MCU provides bonuses to flexible resources to incentivize their contribution to energy imbalance minimization. Each agent would maximize its profits (i.e., minimize its operational costs) while improving the system flexibility. In this regard, the real-time scheduling of local resources in each agent is adjusted by taking into account the MPC concept to consider the operational conditions of the agent as well as the system in future time periods. Moreover, the stochastic optimization and CVaR concept are taken into account to address the uncertainty of optimization parameters and model the perspective of each agent toward its respective risk in real-time operational management.
Note that the proposed algorithm would also enable the MCU to coordinate the agents' operation in the case that the upper-level network could not provide flexibility service to the agents of the MG for balancing supplies and demands during the real-time optimization. According to the above discussions, the proposed framework for energy imbalance minimization in real-time management would enable the MG to operate independently as well as privately by decreasing its dependence on the upper-level network for ensuring the balance between supplies and demands during the real-time operation. Furthermore, the proposed framework could be employed by MCU to incentivize the contribution of flexible resources in the provision of the flexibility service to the utility of the main grid. Besides, different kinds of flexible resources are taken into account in this article to discuss their contribution to providing flexibility service during the real-time operation; while, sensitivity analysis is employed to study the effects of agents' viewpoints toward the risk on the operational scheduling, flexibility service exchanges, and received bonuses.
The rest of this article is organized as follows. In Section II, the structure of the multiagent MG and the TE-based control framework to facilitate flexibility service exchange between the agents are presented. Then, Section III describes the proposed mathematical formulations of defining TE signals by the MCU and the real-time operational scheduling optimization conducted by each agent. The results of implementing the proposed algorithm on a multiagent MG to minimize the energy imbalance in real-time operation are presented in Section IV. Finally, Section V concludes this article.

A. Multiagent Structure
By emerging the new resources in modern power grids, MGs are developed to manage the independent agents that operate their respective resources. Therefore, MCUs coordinate the management of the agents to maximize the social welfare, whereas each agent aims to maximize its respective profits. In this regard, Fig. 1 presents a simplified TE-based control model of an MG with a multiagent structure. In this framework, MCU plays a vital role in coordinating the agents, which finally results in facilitating the exchange of flexibility services, improving the flexibility of the main grid, and engaging TE control signals to maintain the privacy of independent agents. It is noteworthy that without the loss of generality, in this article, it is considered that each agent manages one type of flexible resources, i.e., CDGs, ESSs, EVs, RESs, and flexible load demands.

B. Proposed TE-Based Framework
Based upon the control structure for an MG with a multiagent structure, transactive control concept could be deployed by defining bonuses to exploit the previous scheduling of flexible resources conducted by each agent. In this vein, MCU could allocate bonuses to minimize real-time energy imbalance and facilitate the flexibility exchange among the agents. In detail, the general objective is to minimize the overall operational costs by considering maximizing the profits of independent agents during the real-time operational management. In this regard, in this article, bonuses as TE signals are employed by MCU to incentivize the independent agents to reschedule their respective flexible resources. This scheme facilitates flexibility exchange between agents and benefits both entities; i.e., agents responsible for real-time energy imbalance and the agents operating flexible units. Consequently, the developed framework by minimizing the real-time energy imbalance, while considering the profit maximization of independent agents, would result in the improvement of the overall social welfare.
According to Fig. 2, the announced bonuses are iteratively revised by the MCU to minimize the energy imbalance in the real-time optimization for the current interval. In addition, each agent takes into account the MPC concept to model future time periods while optimizing the scheduling of their resources in the current time interval. In this scheme, stochastic optimization is employed by each agent to consider uncertainties associated with parameters, such as real-time prices, in future time periods. Furthermore, the risk associated with the uncertainty of operational parameters in the resource scheduling optimization of each agent is modeled utilizing the CVaR concept. Finally, the information exchange between each agent and the MCU is limited to the accumulated power request by each agent and TE signals, which is a great opportunity to keep the agents' privacy.

A. TE Signals Definition
Regarding the operational management of local resources, one of the primary reasons for real-time energy imbalance occurrences would be the uncertainties associated with nondispatchable local resources, i.e., RESs and load demands. On the other hand, based on the independent operation of each agent, agents would have to balance the supply and demand based upon real-time prices, i.e., price of selling/purchasing power from the main grid. As a result, the framework strives to enable activating flexibility service from the flexible resources to minimize the real-time energy imbalance in the MG. Note that the offered bonuses would finally be compensated by the agents responsible for energy imbalance at each time dispatch. Furthermore, TE signals would be updated in an iterative manner by MCU to minimize the energy imbalance in the MG.
The mathematical formulation of determining bonuses in the proposed scheme is presented in (1). It can be seen in the case that the MG has to purchase power from the main grid to maintain the power balance; positive bonuses (i.e., b Pos t ) are defined to incentivize agents in order to increase their generation or decrease their consumption. Opposite to the above condition, the negative bonuses (i.e.,b Neg t ) would be announced by the MCU to persuade agents in order to decrease their power generation or increase their consumptions. It is noteworthy that when the main grid could provide the overall real-time energy imbalance in the MG, b Pos t /b Neg t would be increased up to the differences between the real-time purchasing and selling prices, which would also assure the convergence of the proposed approach in finite steps. In other words, in this condition, flexibility service exchange between the agents would benefit both parties, i.e., agents responsible for energy imbalance and the agents providing flexibility service. However, in case that the energy provided by the main grid to address the real-time energy imbalance in the MG is limited, the proposed scheme could be continued until the step that assures the supply-demand balance in the MG. See (1a)-(1d), shown at the bottom of this page.
In case the real-time energy imbalance becomes zero due to the contribution of flexible resources, by increasing the offered bonuses, the system would confront with energy imbalance in a different direction. In other words, by increasing the bonuses, the increase/decrease in the power generation/consumption of the resources could again result in the energy imbalance in the system. In this regard, as the proposed framework is based on updating bonuses in an iterative discontinuous way, in some cases, the increase in bonuses could cause real-time energy imbalance in a different direction. In this condition, in order to ensure the energy imbalance of the system would be equal to zero, the permissible change in selling/purchasing power of agents would be limited as follows: where ΔP i,Allowable t shows the permissible amount of change in the power scheduling of agent i. Note that as the offered bonus would be equal to the iteration n, the contribution of the agents would be limited proportionally to ensure the real-time energy imbalance at time period t i.e., i∈Ω Agent ΔP i,Allowable t becomes zero. In the following sections, the optimization models conducted by the agents of the MG are presented. In this regard, it is noteworthy that the change in the day-ahead operational scheduling of the flexible resources would be considered in the operational scheduling models developed for real-time optimization of each agent.

B. Scheduling of Flexible Demands
Flexible load demands could revise their scheduling in order to provide the flexibility service based upon the received bonuses. In this regard, the optimization model associated with the scheduling of flexible demands in the current time dispatch utilizing the MPC and CVaR concepts to maximize their respective profits is presented as follows: where i∈Ω Agent The objective function of the optimization model is presented in (3a) and (3g), while (3b)-(3e) model the expected operational profits associated with the scheduling of flexible load demands in the current time dispatch (i.e., t) and the future T time intervals. Moreover, (3f) shows CVaR, which is taken into account to model the risk of operational scheduling of the flexible load demands. In this context, α D is a confidence level that indicates the right tail probability of density function [19]. Moreover, β D is a risk parameter that defines the perspective of the flexible load agent toward risk. Similar to α D , β D is bounded 0 to 1 and the risk significance will be increased when its amount is closer to 1 [21]. Additionally, (3h) and (3i) are modeled to provide a linear formulation for CVaR term. Furthermore, the limitation of flexible demands. variations in each time interval and each scenario is defined in (3j); while (3k) imposes the energy that should be provided to the demand in the respective time period. Finally, (3l) and (3m) define the limitations for load shedding in each time interval, and (3n) presents that the optimization results for the real-time interval t are here-and-now decisions.

C. Scheduling of Storage Units
Storage units would play a vital role in MGs to provide flexibility services to the utilities as well as independently operated agents. In this regard, the stochastic optimization model employed by ESS agents to schedule their charging/discharging in the current time dispatch (i.e., t) while considering the T future time intervals is as follows:  The objective function of the optimization model associated with scheduling of ESSs is presented in (4a) and (4g), while (4b)-(4e) evaluate the expected operational profits associated with the scheduling of ESSs in the current time dispatch (i.e., t) and the future T time intervals. Furthermore, (4f)-(4i) model the CVaR index, which is taken into account to model the operational risk associated with the scheduling of ESSs. Specifically, the uncertainty associated with the energy price in the future time intervals could be modeled by a number of scenarios and so CVaR addresses the operational risk associated with the developed operational scenario. Moreover, the constraints over the increase/decrease in charging/discharging of the ESSs are considered in (4j). The change in the state of charge of the ESSs in each time interval as well as their respective limitations is defined in (4k) and (4l), respectively. Furthermore, (4m)-(4o) are modeled to impede the simultaneous charging/discharging of the storage units at each time interval. Note that P DA,Dis,ESS

D. Scheduling of CDG Units
CDGs are responsive resources that could be rescheduled corresponding to the received bonuses. In this regard, the MPCbased stochastic scheduling of the CDG units is developed as follows: Subject to: The objective function of the CDGs scheduling optimization model is presented in (5a) and (5g), while (5b)-(5e) determine the expected operational profits of scheduling CDG units at the current time dispatch (i.e., t) and the future T time intervals. Furthermore, (5f)-(5i) present the CVaR index. Finally, the limitations over the increase/decrease in power generation by CDG units are considered in (5j), and the (5k) presents the results of the optimization model at the current time interval as here-and-now decisions.

E. Scheduling of EVs
EVs would become potential sources of flexibility in MGs by the ongoing trend of increasing their role in transportations. As a result, EV agents would optimize their scheduling in the current time dispatch based upon the energy prices and bonuses offered by the MCU. In this regard, the optimization model associated with EVs' agents is developed as follows: Subject to:  (6r) The objective function of the optimization model utilized by EVs' agents to schedule their resources is presented in (6a) and (6g), while (6b)-(6e) model the expected operational profits associated with the scheduling of EVs at the current time dispatch (i.e., t) and the future T time intervals. Note that T V2G presents the time periods that the EV unit is connected to the grid and could be charged/discharged. Equation (6f) represents CVaR, which is deployed to model the risk of operational scheduling of the EVs. Additionally, (6h) and (6i) are modeled to provide a linear formulation for CVaR term. Moreover, the limitations over the change in the charging/discharging of the EVs are imposed by (6j). Furthermore, the change in the state of the charge of the batteries of EVs in each time interval as well as their respective limitations are defined in (6k)-(6n), respectively. In addition, (6o)-(6q) are considered to impede the simultaneous charging/discharging of the EVs at each time interval. Note that P DA,Dis,EV

IV. RESULTS
The developed scheme is applied on an MG composed of PV units, wind power units, ESSs, EVs, flexible load demands, and CDGs, which are operated by independent agents in order to study the effectiveness of the framework in decreasing the real-time energy imbalance in the system. Note that the operational data of the system are adapted from [22]- [24] and presented in [25]. Moreover, it is considered that, similar to the conventional systems, the main grid could ensure the supplydemand balance in the MG. Consequently, based on the previous discussions, the offered bonuses at each time interval would be limited to the difference between the power selling/purchasing prices of power from the main grid. The proposed framework would facilitate the flexibility service exchange among the MG's agents in order to decrease their real-time energy imbalance in a cost-effective way. Respectively, MCU is the responsible entity for providing transactive signals, while the agents causing the energy imbalance would finally compensate the offered bonuses.  It is noteworthy that the framework aims to optimize the energy imbalance in the MG at each hour of the real-time operation of the system. Based on the developed formulation, agents employ stochastic optimization and CVaR index to model the uncertainty of optimization parameters (i.e., energy price at future time periods) and their respective risks. In this case study, it is considered that the agents would model 20 scenarios associated with the energy price while participating in the proposed real-time energy imbalance minimization. Moreover, it is considered that the developed scenarios would have a similar probability of occurrences.
In the first case study, it is considered that the real-time operation of the MG over 24 h is studied, where the agents consider β as the risk factor to be 0.5 in their resource scheduling optimization. In this regard, the energy imbalance in the MG before/after implementation of the proposed scheme is presented in Fig. 3, which shows the ability of the proposed framework to cost-effectively minimize the real-time energy imbalance in the system. Note that this would finally result in decreasing the dependence of the system on the main grid for addressing real-time energy imbalance; therefore, the framework would improve the system's flexibility and reliability. Furthermore, the offered bonuses to the agents for exploiting the scheduling of their flexible resources are shown in Fig. 4. It is noticeable that, at some hours, the offered bonuses are smaller than the maximum possible amounts, which means that, at these time intervals, the energy imbalance of the system is merely alleviated by the flexibility service provided by local responsive resources. In other words, at these time intervals, the system would not rely on   the upper-level power network to ensure the supply-demand balance. It should be noted that the information exchange between the agents and the MCU is limited to the offered bonuses and the accumulated change in power request of agents compared with the day-ahead scheduling, which addresses the privacy concerns in multiagent systems. In this regard, the MCU would update the offered bonuses, while agents independently optimize the scheduling of their flexible resources during the real-time operation. The announced transactive signals and the energy imbalance per iteration of running the algorithm at hours 3, 21, and 22 are presented in Figs. 5-7. According to (1), the negative/positive value of the energy imbalance at each time interval would determine that the positive/negative bonuses would announce to the agents. In other words, based on the operational condition of the system, the MCU would merely announce b Pos t or b Neg t , which is presented as the announced transactive control   signal. Moreover, the MCU could increase the penalty factor ρ in (1a) to increase its convergence speed based on the responses of the agents. The change in day-ahead power scheduling of flexible resources (i.e., provided flexibility service) is presented in Fig. 8, while the share of each type of flexible resources in supplying the real-time energy imbalance at each hour is demonstrated in Fig. 9. It is conceivable that the flexible resources play a significant role in addressing the real-time energy imbalance engendered by RESs' uncertainties. Finally, flexible resources receive the bonus (i.e., ΔP · b), as shown in Fig. 10, for their contribution to minimizing the real-time energy imbalance at each time period.
In the second case study, the operational condition of the system in case that the agents employ different amounts of the risk factor (i.e., β) in their real-time resource scheduling optimization  is analyzed. In this respect, the total amount of bonuses received by each kind of flexible resources over the 24 h for collaboration in minimizing the real-time energy imbalance is presented in Fig. 11. Furthermore, the benefit of the system (i.e., the decrease in operational cost of providing real-time energy imbalance from the main grid) over 24 h and its proportion to the total amount of received bonuses by flexible resources for contribution in minimizing the real-time energy imbalance are demonstrated in Fig. 12. Note that, at time periods, the offered bonuses are less than the maximum range of bonuses, as presented in Fig. 4; the real-time energy imbalance of the system is alleviated without exchanging power with the main grid and so the operational cost of the system is decreased. In addition, the ratio of the benefit to the total amount of bonuses received by the agents is decreased as the β increases, which means as the agents become more risk-averse, their benefit would be decreased. This is based on the fact that as the agents become more risk-averse, they would be less incentivized to reschedule their resources due to the uncertainties of decision parameters. Fig. 13 shows the real-time energy imbalance in the system after the implementation of the proposed framework. Similar to the previous discussion, the total amount of energy imbalance of the system over the 24 h (i.e., T i∈Ω Agent |ΔP i t |) is increased as the agents become more risk-averse. In this regard, the total amount of energy imbalance over 24 hours from 111 MWh before implementing the proposed framework is, respectively, decreased to about 61, 62, 72, and 73 MWh for case studies that β equals to 0, 0.2, 0.6, and 1.

V. CONCLUSION
This article proposes an efficient decentralized mechanism in order to manage the real-time energy imbalance in an MG composed of independently operated agents. In this regard, transactive control signals are designated to incentivize agents operating flexible resources to reschedule their resources to minimize the energy imbalance in the MG. Respectively, the provided scheme facilitates flexibility service exchange among agents while addressing their privacy concerns. Moreover, the proposed approach benefits the MG by maximizing the social welfare of the system as well as decreasing its dependence on the main grid to ensure supply-demand balance. The interaction between MCU and agents is limited to accumulated power requests and transactive signals in order to address the privacy concerns.
The proposed mechanism is applied on an MG consisting of independent agents scheduling CDGs, RESs, EVs, ESSs, and flexible demands, which demonstrates the effectiveness and significance of the developed approach in minimizing the real-time energy imbalance in the system. Moreover, the impacts of riskaverse/risk-seeker preferences of agents on their collaboration in minimizing the real-time energy imbalance and so the system benefits are thoroughly investigated. Finally, the presented results indicate that the proposed approach would minimize the real-time energy imbalance in the MG in a cost-effective way.