A Novel Multi-Hierarchical Bidding Strategy for Peer-to-Peer Energy Trading Among Communities

Recently, several market types and regulations have been developed in an attempt to handle the increased carbon effect. End-users can also actively participate in the existing distribution system thanks to Peer-to-Peer (P2P) energy trading, which is one of the new emerging market types. In this paper, a novel dual bidding strategy for multi-hierarchical P2P energy trading that includes both intra community and inter communities is proposed considering uncertainties in solar irradiance and temperature. While the lower-level problem consists of both optimal bids of the households to the own Local Market Operators (LMO) for intra community trades and optimal bids of the LMOs to the Central Market Operator (CMO) for inter community trades, profit of both the LMOs and CMO is maximized by clearing the market prices at the upper-level problem. To prove the validity of the devised model, a set of case studies are created. Moreover, the results suggest that the proposed bi-level model is robust, and a remarkable amount of cost savings could be provided by integrating the model.

INDEX TERMS Bi-level optimization, local market, market clearing, optimal bidding strategy, peer-to-peer energy trading.

NOMENCLATURE
The nomenclature for the proposed model is introduced in a detailed way in this section.  Lagrange multipliers for the equality constraints of community i.

A. MOTIVATION AND BACKGROUND
Recently, several innovative solutions have been proposed in order to use the energy in an efficient manner. End-users have been becoming a part of the distribution system along with Peer-to-Peer (P2P) energy trading, which is one of these solutions. Therefore, physical supports to the grid and economic savings for the participants are provided [1]. Carbon emissions can be reduced by increasing self-consumption of renewable energy resources especially photovoltaic (PV) by means of such a new emerging market type [2].
Another point in the P2P trading is to encourage all the participated households in terms of economy. In this context, there are available several regulations and market clearing strategies [3]. Furthermore, new energy management strategies could be introduced to the grid with the aim of mitigating some challenges such as voltage drops, increment in power VOLUME 10, 2022 losses, and peak loading [4]- [7]. However, selling excess energy between communities has been playing a key role to ensure the participation of any community which has no generation unit.

B. LITERATURE OVERVIEW
Many studies considering energy scheduling and market clearing strategies for P2P energy trading are available in the literature, and it can be categorized into hierarchical approaches and inter-community trades for this paper in order to clarify literature gaps.
In terms of hierarchical market types and inter-community based research, Liu et al. [8] proposed a bi-level distributed optimization based energy trading system between PV and schedulable loads equipped participants. Cui et al. [9] propounded both real time operation for intra-community P2P energy trading and day ahead scheduling for inter-community P2P energy trading under two stage approach which includes sharing and market clearing. Nezamabadiet al. [10] suggested arbitrage strategy based non-cooperative game considering risk constraint and uncertainties related to renewable generation and real time market prices. Cooperative game for intra-community and non-cooperative game for inter-community were also considered in the paper. Chen et al. [11] presented intra-region and inter-region P2P trading considering multiple energy sharing regions. Game theory based trading system was designed in the paper also considering dynamic pricing of sharing regions, transmission and distribution usage fees and demand response.
Profit of market operator and participants was increased via game theory in [12]. It was achieved that both self-consumption of PV and encouragement to the P2P were improved by being shifted the responsive demands to the time intervals in which high PV generation is available. Liu et al. [13] supported the P2P energy trading with community energy storage in the proposed bi-level Stackelberg game based model. While daily profit of households were maximized in the lower level problem, profit of market operator was maximized by determining clearing prices in the upper level problem. Total electricity bill under P2P trading and distribution usage fee were minimized by Alternating Direction Method of Multipliers (ADMM) based distributed optimization in [14]. Although hierarchical managements were realized in [9]- [11], optimal local prices were not determined considering both intra-community and inter-community trading bids under multi-hierarchical manner. In this regard, even if high amount of energy is traded between via intercommunity P2P structure, energy prices will not reach fair level according to the economic manner between supply and demand.
In terms of the works related to various market pricing schemes and strategies, Paudel et al. [15] presented a game theory based model for real time P2P energy trading under game based market clearing. Khorasany et al. [16] achieved to reduce computational challenges in the communication considering both community and decentralized based P2P trading. Impacts of the schedulable and deferrable loads on P2P energy trading were investigated in the genetic algorithm based optimization model in [17]. Supply Demand Ratio (SDR), Mid-market Rate and Bill Sharing pricing mechanisms were also examined and it was observed that SDR method provides by 27%-68% bill reduction. In [18], day ahead scheduling and real time operation were presented as to include two stage. While optimal energy sharing profile was carried out via ADMM in the first stage, market prices were cleared based on non-cooperative game theory in the second stage. A novel decentralized optimization model without any controller system operator was developed in [19]. Line capacities were also considered in the bilateral contract pricing based study.
Khorasany et al. [20] developed a novel intraday market in which flexibility and indepented pricing strategy are implemented besides minimization of the daily cost in day ahead scheduling. It was observed that higher profit can be gained in the case of integrated market model compared to the case in which only day ahead market is considered. In [21], clearing of both traded energy and reserve energy was carried out for the first time in the model which aims the participation of renewable energy producers. Cui et al. [22], reduced energy cost thanks to the controllable air conditioners and reduced dependency on the grid. Li et al. [3] compared the proposed model with the existing pricing mechanisms in the literature, and achieved increment in the participation ratio as well as economic efficiency. Impact of demand response on P2P trading considering penalty system in the case of deviation from scheduled demand was examined in [23]. It was observed that cost reduction in non-optimization based model is greater than optimization based model which includes day ahead and real time operation.
Among the literature studies mentioned above, although P2P energy trading was carried out in [15]- [17] and in [3], [22], [23], hierarchical approach, inter-community energy trading and uncertainty were ignored. While hierarchical trading was implemented in [8], [12], and [13], intercommunity energy trading was not included. Conversely, inter-community trading was contained in [14]. However, hierarchical system was not comprised. Finally, although hierarchical test system, inter-community energy trading and uncertainty were considered in [9], [10], and [11], bidding strategy for multi-hierarchical energy trading system was not investigated. In this regard, there is a still gap in the literature in terms of evaluation of fair market clearing and bidding strategy which simultaneously consider multi-layer hierarchical bidding strategy, and unpredictable behavior of renewable generation.

C. CONTRIBUTIONS AND ORGANIZATION
In this study, a novel hierarchical bidding strategy for both intra-community and inter-community P2P energy trading is proposed in the mixed-integer linear programming (MILP) based mathematical model. To reveal the effectiveness of the proposed model, a bunch of case studies considering different PV installation powers and ESS capacities are carried out. The main contributions are stated as follows: 1) A novel optimal bidding strategy for multi-hierarchical approach considering both intra-community and intercommunity P2P energy trading system is proposed for the first time in the literature.

2) A bi-level MILP algorithm is developed, in which
while the lower-level aims to minimize the daily electricity bill of households and total bill of communities, upper-level aims to maximize the profit of Local Market Operators (LMO) and Central Market Operator (CMO).
3) The unpredictable profiles of solar irradiation and temperature are considered by generating the scenarios via the historical data.
The remainder of the paper is organized as follows: the proposed trade structure and mathematical model are detailed in Section II. Test results and comments are presented in Section III. Finally, conclusions are evaluated in Section IV.

II. METHODOLOGY A. OVERVIEW OF THE MULTI COMMUNITIES STRUCTURE
The proposed multi community P2P energy trading structure is shown in Fig. 1. LMO for intra-community energy trading and CMO for inter-community energy trading, which manage all the local transactions, are available. Furthermore, CMO and Distribution System Operator (DSO) are in communication and coordination in order to manage physical trades and clear market prices. First of all, all participants send their quantity and price bids to the own LMO for intra-community P2P energy trading. After receiving the local bids, all LMOs send excess or deficit energy quantity, and price bids to the CMO as to cumulative after intra-community energy trading. Also, optimal local trading prices are determined by the CMO according to the all energy bids received from LMOs and grid price signals received from DSO. Finally, both participants and LMOs schedule daily consumption profiles along with the determined prices so as to maximize the profits in the hierarchical market structure.

B. MATHEMATICAL FORMULATION 1) LOWER LEVEL PROBLEM
Objective of the lower level problem is to minimize total cost for both intra-community and inter-community energy trading. In (1) Equations (2)-(5) state the bidding constraints of the intra-community and inter-community P2P energy tradings. If a household is a consumer, it can buy the deficit energy via intra-community P2P after own load and ESS are fed by own PV. Similarly, if a household is a prosumer, it can sell the excess energy after own load is fed by own PV and ESS. Similar to the intra-community trading, maximum total energy that can be bought from the P2P by a community is total deficit energy remained after all the load and ESS charge demand are met by all the PV generation and intra-community trading. Furthermore, a community can sell the total excess energy to the another community via inter-community P2P trading after all the load and intracommunity P2P demand are met by all the PV generation and ESS.
Equations (6) and (7) define the net energies which consist the difference between load and PV. Also, parameters X and Y represent excess consumption and excess generation, respectively. Finally, cumulative power equations in the bidding constraints are presented in (8)- (15). In (21) and (22), while general energy balances of a household and a community are shown, PV, load, and power bought from and to the grid which take part in the balance of community are stated in (23) Constraints of the optimal local trading prices are stated in (34). Both buying and selling prices should be between grid prices in order to encourage the participants. Furthermore, since the local buying price is the buying price of the households while it is the selling price of the market operator, local selling price should be lower than local buying price in order to avoid any loss in the profit of operators. Karush-Kuhn-Tucker optimality conditions as in [24] and Big-M linearization method as in [25] are used in an attempt to solve the bi-level model, and are shown as simplified form in (36) and (37), respectively. Furthermore, strong duality theorem as in [26] is adopted to linearize bilinear terms in the objective function of the upper level problem, and is obtained as demonstrated in (38).
Here, f (x), g(x) and h(x) are the objective function, the inequality constraints, and the equality constraints of lower level problem, respectively. γ x i,m,s,t , β x i,s,t and λ x i,m,s,t state the Lagrange multipliers related to the constraints.
Here, µ is mean value of each time interval, σ is standard deviation, φ is probability density function of the standard normal distribution and is the cumulative distribution function. Furthermore, truncation ranges (a, b) are implemented to the function under the doubly truncated case since infinity boundaries in normal distribution cause deviation in the desired data generation. Thus, probability generation for each time interval can be determined between the minimum and maximum values of the input data. In addition, 30 household profiles in total for three community are generated probabilistically by referencing consumption profile of a smart home belong to the 30 days of September in 2007-2010 [29].

A. INPUT DATA
Consumption profiles of 30 households located in 3 communities are randomly generated by referencing daily and yearly consumption dataset obtained from [29]. All households in the Community-1 are consumers which have no PV and ESS. While 5 consumers and 5 prosumers are situated in the Community-2, all the 10 households are prosumers in the Community-3. It is assumed that all the PV and ESS capacities are 4 kW and 10 kWh, respectively for Case-1 and Case-2. Furthermore, capacities of PV and ESS for Case-3 are randomly generated between the boundaries [3,6] kW and [5,15] kWh, respectively. The probabilistically generated PV power scenarios are depicted in Fig.2.

B. SIMULATION AND RESULTS
The proposed model is implemented in GAMS [30] v.24.1.3 and is solved using the commercial solver CPLEX v.12. Furthermore, three different case studies are examined to prove the effectiveness of the model as follows: • Case-1: Intra-community P2P without inter-community P2P.
• Case-2: Intra-community and inter-community P2P with same capacities of PVs and ESSs.
• Case-3: Intra-community and inter-community P2P with various capacities of PVs and ESSs. Only intra-community P2P trading under the management of LMOs is taken into account in the Case-1. In the Case-2, both intra-community and inter-community P2P are examined assuming that all the participants have same PV and ESS capacities. Furthermore, both intra-community and intercommunity P2P with various capacities of PVs and ESSs is considered in Case-3 in order to investigate the impact of different investments on the market conditions. In Case-1, it should be noted that LMO-1 equalizes the local prices to the grid buy price to avoid loss of the profit since no energy trading in Community-1 is available. Local clearing prices of Community-2 and Community-3 for Scenario-5 in Case-1 are depicted in Fig. 3 and 4, respectively. While prices in Community-2 are high because of the equal number of prosumers and consumers, prices in Community-3 which  consists entirely of prosumers are determined lower because of the excessive energy. Furthermore, while P2P trading is provided with the green PV generation at the noon time intervals, it is carried out by the ESSs which are external power suppliers at the peak time intervals as can be seen from the power balance of Community-3 in Fig 5. Thus, market prices at the peak time intervals are reduced lower compared to the noon time intervals. The profit of LMO-1 is zero due to the no transaction in Community-1, and CMO cannot get any profit since does not include in the hierarchical bidding system in the Case-1. Also, LMO-2 and LMO-3 get profit as 3.041 TL and 3.873 TL, respectively. Power balance of the Community-3 for Scenario-9 in Case-2 is demonstrated in Fig. 6. Apart from the Case-1, ESSs drawn more power from the grid at 3 am and 5 am which are the lowest price intervals after 2 am in order to participate inter-community P2P trading. Thus, since PV generation at the noon is used for both intra-community and inter-community instead of storing, grid is supported by increasing the capacity of P2P trading with ESSs at the peak time intervals. Optimal buying and selling prices for Scenario-9 in Case-2 are shown in Fig. 7. Furthermore, since all the communities are considered together in the multi-hierarchical system in this case, while Community-3 reduces the local prices, Community-1 and Community-2 rise the prices at the same time. It can be seen that jointly announced local prices to all the communities are determined between the prices of two different communities in Case-1. On the other hand, it is worthy to state that Community-2 has  loss in the profit because joint selling price in Case-2 is lower than specific selling price in Case-1. The profits of LMO-1, LMO-2 and LMO-3 are 0.529 TL, 12.153 TL and 6.715 TL, respectively. CMO can also get profit as 4.534 TL from the inter-community P2P via arbitrage strategy.
The power balance of the Community-3 for Scenario-9 in Case-3 is presented in Fig. 8. It can be seen from the figure that ESSs trend to charge more as a result of randomly PV and ESS capacity generation. Furthermore, while intra-community P2P trading for Community-2 and Community-3 decreases due to the increased self sufficiency ratio, inter-community P2P trading also rises. Thus, it is observed that decrement in the total cost of all the communities is available. On the other hand, cost reduction of Community-2 and Community-3 causes loss in the profit of LMO-2 and LMO-3 as expected. However, profit of the LMO-1 and CMO rises in each case since play active role in only inter-community trade. Local clearing prices for Scenario-9 in Case-3 are depicted in Fig. 9. It can be seen that local selling prices in Case-3 are slightly lower than Case-2's because of the increased capacities.
For instance, while selling prices at 8 am and 12 pm for Case-2 are 0.652 TL and 0.612 TL, respectively, they are determined as 0.566 TL and 0.588 TL for Case-3, respectively. The profits of LMO-1, LMO-2, LMO-3 and CMO are 0.922 TL, 10.796 TL, 4.999 TL and 8.549 TL in Case-3, respectively. It is observed that total bill cost    for Case-1, Case-2 and Case-3 is obtained as 348.716 TL, 333.759 TL and 295.706 TL, respectively. According to the results obtained from the bi-level MILP based model, cost saving by %15 in Case-3 compared to Case-1 is achieved.
Finally, amount of traded energy in best and worst scenario, and cost of communities and profit of LMOs and CMO for each case are shared with Table 1 and Table 2, respectively. Traded energy in best scenarios for both inter-community based cases increases compared to the worst scenarios. However, total trade volume in Case-3 is more than Case-2 regardless of the scenarios because of the increment in PV and ESS capacities. Furthermore, total energy for all the scenarios in Case-2 and Case-3 is greater than Case-1 since ESS units trend to drawn more power from the grid at the low price time intervals in order to participate inter-community energy trading. On the other hand, different capacity investments and uncertainties in PV generation influence daily bill cost and market conditions as can be seen from the results. Thus, the profit of the operators changes depending on the amount of traded energy and the market clearing prices. Another important point is that Community-2 has loss in the profit unexpectedly because joint selling price in Case-2 is lower than specific selling price in Case-1.

IV. CONCLUSION
In this study, a novel multi-hierarchical bidding strategy for P2P energy trading considering both intra-community and inter-community energy trading was proposed. According to the results obtained from the bi-level MILP based model, while total cost saving of communities was achieved by 15%, Community-1, Community-2, and Community-3 individually got decrement in the cost by 6.31%, 1.01%, and 95.64%, respectively.
The results in terms of both economic and energy showed that inter-community P2P trading provides some advantages to the households and power system since trading with the grid during the off peak periods and P2P trading during the on peak periods are significantly increased. Furthermore, it was observed that optimal buying and selling prices are influenced by the number of participants and installed capacities of PV and ESS units. Thus, households gained more profit in case of high investment capacities compared to both intra-community based case and low investment based case. However, while local operators had loss in the profits, CMO increased the benefit as a result of hierarchical opposite situation as expected. As future extension of this work, impact of P2P energy trading on the coordination between Transmission System Operator and DSO will be investigated.