Operation Mechanism and Strategies for Transactive Electricity Market With Multi-Microgrid in Grid-Connected Mode

Under the policy support of electricity market reform and the promotion of microgrids in grid-connected operation mode construction in China, the sales side of electricity market is gradually liberalized. Microgrids are developing rapidly through participating in the electricity market competition. In order to find the multi-player game relationship of real-time electricity market with multi-microgrid in grid-connected operation mode and provide an incentive for electricity trading among microgrids, we propose a transactive real-time market trading mechanism based on the principle of proportional distribution. Through the analysis of multi-player game in real-time market, we build cost-benefit models of microgrids with power surplus and those with power shortage. Based on the game theory, non-cooperative game models with multi-buyer and multi-seller of microgrids with power surplus and those with power shortage are established. The existence of Nash equilibrium for the game models of microgrids with power surplus and those with power shortage has been proved and the Nash equilibrium is solved to obtain their optimal bid and selling strategies. The feasibility and validity of the proposed mechanism and models are proved through case study. The result demonstrates that all microgrids can obtain the best profits with their optimal strategies.


I. INTRODUCTION
As small-scale power distribution systems, microgrids (MGs) integrate renewable energy, load, energy storing device and energy management system organically [1]. MGs can improve the efficiency and reliability of the energy system, as well as reduce transmission losses [2]. They are also special power sources that deliver the surplus power to distribution network (DN) on the premise of satisfying the electricity demand of their own users. Therefore, MGs manage to make full use of the advantages of distributed generation (DG) by dealing with the problem of intermittent and volatility [3]- [6]. As an important means to promote the interaction between DN and users, the reasonable construction and operation of MGs can adapt to the new normal of energy The associate editor coordinating the review of this manuscript and approving it for publication was Siqi Bu . development such as multi-energy complement and energy interconnection and create new trading modes for the market. Configured by ''source-grid-load-storage'' integration, MGs will participate in the market competition as sellers in the future power grid with the development of energy systems [7]- [12].
Based on the technical and economic advantages of gridconnected MGs, there is an urgent need for applicable operation mechanism of the transactive electricity market and appropriate trading strategies of MGs. These issues have become the glare of global attention and greatly affected the development of MGs. Scholars all over the world are focusing on the policy and reform to promote the development of MGs in grid-connected mode. For example, China has issued Several Opinions on Further Deepening the Reform of the Power System [13] and Trial Measures for Promoting Grid-Connected Microgrids Construction [14], which provides policy conditions for the construction, development and market participation of MGs.
The production and use of electricity are simultaneous. Power generation is usually dominated by renewable energy in MGs [15]. Renewable energy is greatly affected by environmental factors with strong intermittent and large output fluctuation. Thus, the market with multi-microgrid in grid-connected mode is more suitable for real-time trading.
Reference [16] proposed a smart trading energy framework for home MGs and explored the feasibility of cooperation alliance with multi-microgrid. The price matching mechanism considering the demand-side energy management was raised and the validity of the model was verified by simulation studies. Reference [17] designed a market incentive mechanism to encourage MGs participating in market trading and transformed the bargaining problem into two related issues of minimizing social costs and benefits sharing. Besides, a disperse calculation method which could make minimum expenditure via information interchange was proposed and the model provided theoretical guidance for MGs to participate in market competition actively. Reference [18] proposed an optimizing operation method for home MGs based on the non-cooperative game theory and market operation theory and used the Nikaido-Isoda relaxation algorithm to solve the model. Reference [19] proposed a smart energy management system for MGs based on multi-agent, which took full consideration of the flexibility of MGs in active DN. Based on load participation in demand-side response, frequency-based metric priority confirmed the order of both sides of the trading by the queuing method. Reference [20] proposed a real-time market trading model with multi-photovoltaic MGs based on the cooperative game. The Shapley value method was adopted to allocate the benefits to all parties in the alliance. The effectiveness of the cooperative game method was verified by case study. Reference [21] proposed the concept of energy market of MGs based on blockchain and constructed a highefficiency energy market framework of MGs. Brooklyn MG engineering was used as a case to make market assessments. Reference [22] considered the impact of network congestion on market trading. A trading model that could make individualized design according to the trading object in a complex network was proposed to forecast the bilateral trading and find a relatively stable network structure.
Based on the current studies on the related researches, we find that it is of great significance for MGs participating in market trading to obtain their appropriate bid and selling strategies. The main contributions are listed as follows.
1) Based on the roles and respective duties of the market trading participants, an operation mechanism of the market is proposed and the real-time trading process is precisely clarified and introduced.
2) Considering the renewable energy generation subsidies, energy storage benefits, generating costs, and electricity trading services cost, a comprehensive cost-benefit model of MGs with power surplus in real-time market trading process is built and applied to the non-cooperative game models. Combined with the model of MGs with power shortage, the trading profits of all MGs based on the above operation mechanism are obtained.
3) The non-cooperative game models of MGs with power surplus and those with power shortage are presented, respectively. The method to verifying the existence of Nash equilibrium of the models and solving the optimal strategies for MGs are presented.
The market mechanism and game models proposed in this paper are proved to be feasible and effective by solving the optimal strategies and maximum profits of MGs through the case study, which can provide theoretical reference for the development and reform of the electricity market trading.

A. THE ROLES AND DUTIES OF MARKET TRADING PARTICIPANTS
The internal power generation and the load of MGs can't always be balanced. So, when connected to the DN, MGs can be divided into MGs with power surplus, MGs with power shortage and MGs with power balance. Since that the electricity generation cost is usually lower than the electricity prices of other MGs or the DN, we assume that there is no need for MGs with power balance to participate in real-time market. MGs with power surplus sell surplus electricity to the DN and the MGs with power shortage at an adequately high price through the market. Meanwhile, MGs with power shortage purchase electricity from the DN and the MGs with power surplus at appropriate prices through this market. The buying price C B and selling price C S of DN are determined according to the policies and principles of electricity pricing. If MGs with power shortage decide to purchase electricity from the MGs with power surplus at a higher price than that of DN, MGs with power surplus are willing to trade with the MGs with power shortage for more profits. When several MGs with power shortage participate in real-time market trading, the higher bids they offer, their demands are more likely to be satisfied. In addition, the task of power transmission during the market trading is undertaken by DN. So, it will charge electricity trading service fees c Ser from MGs participating in the market for the maintenance of distribution line. Fig. 1 illustrates the real-time market operation mechanism.

B. CONSTRUCTION OF MARKET TRADING CASES
We usually divide one day into several time slots. All MGs submit their power generation plans and load forecasting results of the next time slot to DN, which in turn calculates their net electricity. DN proposes the constant selling and buying price to all MGs. The numbers of MGs with power surplus and those with power shortage in time slot T are I , J , respectively. Based on the numbers of MGs with power surplus and those with power shortage in the real-time market, there are five cases shown as follow.
1) Case 1: I = 0. All the MGs with power shortage directly purchase electricity from the DN at C S .
2) Case 2: I = 1. There is only one MG with power surplus in this case, leading to a seller's market. The MG with power surplus put the surplus electricity into the market trading at C S − c Ser for more benefits.
3) Case 3: J = 0. All the MGs with power surplus directly sell electricity to DN at C B . 4) Case 4: J = 1. The market turns to a buyer's market in this case. The MG with power shortage can choose to purchase electricity from the MGs with power surplus at C B + c Ser to save cost. 5) Case 5: I > 1, J > 1.There are multi-buyer and multiseller in the real-time market. MGs will formulate optimal strategies for maximum profits.

C. MARKET TRADING PROCESS
Only Case 5 has multi-player game, whose market trading model is established based on the relationship of market participants [23]. For MG i with power surplus, E i is defined as its surplus electricity. Let θ i be the proportion of E i that MG i plans to sell to the market. The electricity that MG i sells to real-time market can be expressed as The total electricity of the MGs with power surplus can be expressed as For MG j with power shortage, we define E j as the electricity purchased from the MGs with power surplus. So, the total expenditure charged from the MGs with power surplus can be expressed as where c j denotes the unit price bid submitted by MG j with power shortage. Based on the principle of proportional distribution, the MGs with power surplus will allocate the electricity to the MGs with power shortage according to the proportion of bids submitted by MGs with power shortage. The MGs with power surplus will share benefits according to their contributions. Thus, E j can be expressed as Let E D,j be the electricity deficiency of MG j. If E j < E D,j , the MG j participates in the second round of trading or directly purchases electricity from the DN. On the other hand, if E j > E D,j , MG j puts the surplus electricity into the second round trading or directly sells to DN. Following the trading process illustrated above, the distribution system reaches a state with power balance after several rounds of market trading. Finally, the DN publishes the electricity price information and the market trading results to MGs.
On account that renewable energy generation is greatly affected by weather conditions and the forecasting method is not quite precise, the planned electricity interaction is usually different from the real-time. So, having accomplished the market trading in time slot T, each MG pays for the operating penalty cost to the DN.
where c Pun denotes the unit penalty price. E T ,k and E T −1,k denote the actual and planned trading electricity of MG k, respectively. The real-time market operation process of the distribution system with multi-microgrid is shown in Fig. 2.
The MGs focus on how to maximize their own profits through the market operation mechanism above. In section III, we will introduce the cost-benefit models of MGs with power surplus and those with power shortage to analyze their profits.

III. OPTIMAL COST-BENEFIT MODEL OF MICROGRIDS
The sunk cost such as the construction cost, operation cost and maintenance cost is not included in the overall cost during the real-time market trading. We only consider the cost related to the real-time market trading.
The cost-benefit model of MGs can be expressed as where B MG denotes the benefits of MGs. C MG denotes the cost of MGs. 79596 VOLUME 8, 2020 The power generated by the MGs with power shortage is used to satisfy their own load. Their power generation costs are irrelevant to their trading objects and the market. The cost of MGs with power shortage related to market trading is the electricity trading service cost.
The profits of MGs with power shortage can be written as

B. OPTIMAL COST-BENEFIT MODEL OF MICROGRIDS WITH POWER SURPLUS
The benefits and cost of MGs with power surplus can be expressed as where B Sel,i B Sub,i and B ES,i denote the benefits of the electricity trading, the generation subsidies of renewable energy and the electricity storage, respectively. C Gen,i and C Ser,i denote the generation cost and the electricity trading service fees, respectively. 1) Electricity trading benefits: According to the principle of proportional distribution mentioned in section II, the total expenditure of MGs with power shortage in market trading is allocated to each MG with power surplus based on the proportion of electricity sold. So, the trading benefits of MG i in time slot T is written as 2) Generation subsidies of renewable energy: Many countries have set targets to facilitate the development of renewable energy sources especially wind power. [24] In this paper, we don't consider the difference of subsidies among specific types of renewable energy generator sets. Meanwhile, on account that the generation subsidies inside the MGs have no effect on the benefits of market trading, we only consider the renewable energy generating capacity that put into the market competition. We assume that the proportion of the electricity generated by renewable energy is the same as that inside the MGs. The generation subsidies of renewable energy can be expressed as where b Ren denotes the unit subsidies for electricity generated by renewable energy. α i denotes the proportion of electricity generated by renewable energy.
3) Electricity storage benefits: The installation of the energy storage system contributes to the market trading between MGs and DN through peak shaving and valley filling. There is another part of cost called penalty cost during the market operation mentioned in section II. The electricity storage benefits are transformed into cost saving for decreasing the gap between the planned electricity interaction and real-time electricity interaction.
where b Unp denotes the expected benefits for storing unit electricity, whose value is related to the storage capacity and the penalty cost of electricity interaction.
where β denotes the proportion of storage capacity in the storage system to the total electricity storage capacity. 4) Power generation cost: The overall power generation cost of MGs consists of marginal power generation cost of renewable energy, gas turbine and diesel units. Generally, the power generation cost of gas turbine and diesel units are described by the quadratic function, while the marginal power generation cost of renewable energy is considered as zero.
where a 1 , a 2 , a 3 denote comprehensive cost parameters which are positive numbers. 5) Electricity trading service cost: MGs with power surplus are supposed to pay electricity trading service fees to DN for electricity transmission, which can be written as The profits of MGs with power surplus can be written as

IV. NON-COOPERATIVE GAME MODEL OF MICROGRIDS A. NON-COOPERATIVE GAME AND NASH EQUILIBRIUM THEORY
The game can be divided into cooperative game and non-cooperative game according to whether the players can reach a binding agreement. In the transactive electricity market with profit-maximizing MGs, the relationship between MGs is competitive. So, the trading between MGs can be regarded as non-cooperative game. Non-cooperative game theory includes five elements: game player, strategy, information, utility function and equilibrium. Nash equilibrium is the core concept of non-cooperative game theory, which can be defined as a state where no game player can gain more benefits by a unilateral change of strategy if the strategies of the others remain unchanged.
As for a strategic game, there is a strategy set a = {a 1 , a 2 , · · · , a n }. For all i ∈ N and a * i = a i , if the utility function always satisfies u i ≥ u * i , a is called Nash equilibrium. The existence theorem of Nash equilibrium is shown as follow [25] For a strategic game model with multi-player, if the strategy set is a finite, closed and non-empty convex set and the utility function is continuous and quasi-concave, there must exist pure strategic Nash equilibrium for this model.
Based on the market operation mechanism and the cost-benefit models mentioned in Section II and III, we propose non-cooperative game models for MGs with power surplus and those with power shortage respectively. Then we prove the existence for the Nash equilibrium of the game models and finally obtain the Nash equilibrium, that is, the optimal strategy sets of MGs with power surplus and those with power shortage. Where c j and c −j denote the strategies of MG j with power shortage and other MGs with power shortage except MG j, respectively.
When the strategies of the other MGs with power shortage except MG j are determined, the Nash equilibrium of the non-cooperative game model of the MG j with power shortage is written as where θ denotes the strategy set of MGs with power surplus, that is, the proportion of the electricity put into real-time market.
Since the strategy set of MGs with power shortage is a non-empty convex set, there is the necessity for a continuous and quasi-concave utility function in the closed interval to prove the existence of Nash equilibrium. Based on (4) and (9), the utility function of MG j with power shortage is written as (20) The first and the second derivatives of utility function with respect to c j are calculated.
Based on the method in [23], the optimal strategies of all the MGs with power shortage can be expressed as From (23), we find that the strategies of MGs with power surplus only affect the profits of those with power shortage, but not affect their optimal strategies.

C. EXISTENCE OF NASH EQUILIBRIUM FOR THE GAME MODEL OF MICROGRIDS WITH POWER SURPLUS
The non-cooperative game model of the MGs with power surplus is shown as follow: 1) Game players: all the MGs with power surplus and those with power shortage in each time slot.

5) Equilibrium: the optimal strategy sets of all MGs with power surplus
Where θ i and θ −i , denote the strategies of MG i with power surplus and other MGs with power surplus except MG i, respectively. c denotes the strategies for unit price bid of MGs with power shortage. When θ −i and c are determined, the Nash equilibrium of the non-cooperative game model of the MG i with power surplus is written as The first and the second derivatives of utility function with respect to θ i have been calculated and we obtain The utility function of MG i with power surplus is strictly quasi-concave on θ i because the value of (27) is negative. There is only one Nash equilibrium for the non-cooperative game model, which means the optimal strategy of the MG i with power surplus is existed. The optimal strategy of MG i with power surplus can be solved when (26) is equal to 0.
Based on (23) and (28), we obtain It can be seen from (29) that θ * i are determined by c * j and their generation cost and surplus electricity. From (23), we find that c * j is independent of θ * i . So, during the market trading, the optimal strategies of MGs with power shortage are specified first and then the optimal strategies of MGs with power surplus are determined.

V. CASE STUDY A. BOUNDARY CONDITIONS
There is a distribution system with 6 MGs in grid-connected mode numbered MG1 to MG6. We assume that each MG is close enough to ignore the power loss caused by power transmission. All MGs who have no energy storage in initial time slot can obtain the same information about the market simultaneously.
The renewable energy subsidies b Ren = 0.40 yuan/(kW·h). The value of C S and C B are 1.4 yuan/(kW·h) and 0.40 yuan/(kW·h), respectively. The electricity trading service fee c Ser = 0.1yuan/(kW·h). The power capacity configuration of the MGs are shown in Table 1. In this paper, we define each time slot as an hour. The load forecasting method is mentioned in [26]. The load forecasting results, renewable energy outputs, and the net power of MGs are illustrated in Fig.3-5, respectively.

B. THE BID AND SELLING STRATEGIES OF THE MARKET PARTICIPANTS
There are multiple buyers and sellers in time slot 3, 4 and 17, which belong to case 5. Other time slots belong to case1case 4, where all MGs can trade with each other or DN at specific prices and there is no need for MGs to find optimal strategies. So, in this section we will focus on case 5 and solve VOLUME 8, 2020  the optimal strategies of MGs with power shortage and those with power surplus in these time slots.
In case 5, electricity market with multi-microgrids reaches Nash equilibrium based on the operation mechanism and game model during the trading process. In part B, we will discuss the bid and selling strategies of MGs.
From (23) and (29), we can find that the optimal strategies of MGs with power surplus and those with power shortage are related to the numbers of MGs with power shortage of the market trading. Fig. 6-8 are presented to find the relationship between the optimal strategies and the numbers of MGs with power shortage. Fig.6 illustrates the relationship between the strategies and numbers of MGs with power shortage. The optimal bid price of MGs with power shortage increase with the increase of their number and tend to be constant gradually. The optimal strategies of MGs with power shortage range from 0.36 yuan/kWh to 0.65 yuan/kWh. As is shown in section IV, c j ∈[C B + c Ser , C S − c Ser ]. For this specific case study, c j is supposed to range from 0.5 yuan/kWh to 1.3 yuan/kWh. So c j should range from 0.5 yuan/kWh to 0.65 yuan/kWh.
There are two MGs with power surplus in time slot 3 and three MGs with power shortage in time slot 4. Fig.7 and 8 illustrate the relationship between the optimal strategies of   The variation tendency of the curve is similar to Fig.6. With the increase of the numbers of MGs with power shortage, the market is gradually transformed into seller's market. The more proportion of trading electricity that MGs with power surplus decide to put into the market, the more profits they will obtain through the market. That is the reason why the optimal selling strategies of MGs with power surplus may exceed the allowable range. Under this circumstance, we should let θ be 1, which means there is no need to store energy for the difference between the predicted electricity and real-time electricity. When the number of MGs with power shortage reaches adequately high, its influence on the selling strategies becomes small, that is, the variation tendency tends to be constant.

C. THE MARKET TRADING RESULTS AND PROFITS OF MICROGRIDS
When there are two MGs with power shortage in time slot 17, the value of c j is 0.433yuan/kWh. MGs with power shortage should purchase electricity from MGs with power surplus at 0.5 yuan /kWh. Fig.9 illustrates the profits of MGS in time slot 17. There are four MGs with power shortage in time slot 3 and three MGs with power shortage in time slot 4. Their optimal strategies are 0.52 yuan/kWh and 0.557 yuan /kWh respectively, which are within the reasonable range.  The system achieves power balance after the first-round trading in time slot 17. The first round of market trading results in time slot 3 and 4 are shown in Table 2.
There is still power shortage after the first-round trading in time slot 3. Therefore, MGs with power shortage have to purchase electricity from DN. Similarly, there is power surplus after market trading in time slot 4. MGs with power surplus need to make a second-round trading or sell surplus electricity to DN. Table 3 illustrates the second round of market trading results in time slot 4. After the second-round trading, the distribution system meets electricity balance. Fig. 10 illustrates the benefits of purchasing and selling electricity of each MG after two rounds of market trading in    Fig. 11 illustrates the profits for participating in market trading of each MG in time slot 3 and 4. This demonstrates the effectiveness of our proposed mechanism. All MGs participating in real-time market can get more profits through their optimal strategies.

VI. CONCLUSION
Under the background of electricity market reform and the policy for promoting microgrids in grid-connected mode in China, we have proposed the transactive real-time market trading mechanism and clarified the roles and duties of the participants. The comprehensive cost-benefit model for MGs with power surplus has been proposed. Based on the game theory, we have built the non-cooperative game models of the MGs with power surplus and those with power shortage and proved the existence of Nash equilibrium for the game models respectively. The results have demonstrated that all MGs in market trading can obtain their optimal strategies and gain maximum profits in our proposed mechanism. The proposed mechanism and models can effectively promote the initiative of MGs to participate in the market and improve the further development of the distribution system. In the future work, we will study the multi-player game and cooperative game model of MGs to achieve the electricity sales side reform in the future power grid.
JIANYU GAO (Student Member, IEEE) was born in Langfang, China, in June 1996. He received the B.Sc. degree in electrical engineering and automation from North China Electric Power University, in 2018, where he is currently pursuing the M.Sc. degree.
His research interests include operational mechanism of electricity market with multimicrogrid and comprehensive assessment of integrated energy systems. Tai Her research interests include electricity market and intelligent patrol of transmission lines. VOLUME 8, 2020