Market Impact of Wind-Energy Storage Alliance Strategic Bidding under Uncertainty (October 2021)

The output of wind turbine is volatile and difficult to predict. Energy storage can help wind turbine offset the deviation between forecast and actual output. Based on the concept of sharing economy, there will be more alliance for wind turbines and energy storage in the electricity market. However, an open question is how the wind-energy storage alliance’s participation affects market clearing and the profits of market participants. Therefore, a stochastic bi-level optimization model is proposed to describe the bidding behavior of wind-energy storage alliances in energy and frequency regulation markets. At the same time, a new quantitative index of bidding behavior is defined—regulation participation ratio. Considering the uncertainty of wind turbine output, the profits of wind-energy storage alliance are maximized in the upper level. The lower level minimizes the power purchase cost of distribution system operator (DSO) for the joint market clearing. The bi-level model is transformed into a mixed integer linear programming (MILP) model by Karush-Kuhn-Tucker (KKT) conditions, strong duality theory and large M method. Regulation participation ratio is set to different values in the case analysis, so as to analyze the influence of the alliance’s bidding behavior on market. Moreover, the economic impact of alliance on wind turbine and energy storage is compared.


A. MOTIVATION
With the change of the climate, every country is undergoing a difficult energy transition. China made a plan to peak carbon emissions by 2030 and become carbon neutral by 2060. Despite the global economic is slowing down by the COVID-19 pandemic, the renewable energy is growing faster than expected. The era of clean energy is approaching [1].
However, renewable energy has the characteristics of intermittent and low predictability. The high volatility of wind energy destroys the stability of power grid. Wind farms are punished as real-time delivery deviates from forward commitment. The fluctuation of wind energy is reduced by energy storage, which increases the profits of wind farm. Energy storage provides an important opportunity to increase the penetration of wind power [2]. However, the cost of investing in energy storage is high. With the development of sharing economy [3], an alliance is composed of wind energy and energy storage. On the one hand, energy storage helps wind turbine stabilize fluctuations in output. On the other hand, due to the fast and flexible regulation performance of energy storage, the wind-energy storage alliance also has such performance. The wind-energy storage alliance not only improve its own profits, but also relieve the regulation pressure of system. Therefore, it is an inevitable trend that wind-energy storage alliance participates in joint optimized dispatch of the energy and frequency regulation markets. However, the questions are how the bidding behavior of alliance affect the clearing result and how much the alliance contributes to profits.

B. LITERATURE REVIEW
Over the past few years, probability scenario method is used to deal with the uncertainty of power market. In order to reduce the complexity of calculation and consider the uncertainty of parameters, reference [4] decouples scenario reduction from bidding process. Reference [5] adopts scenario-based stochastic method to model the uncertainty in demand. Reference [6] uses scenario reduction technique to deal with the uncertainty of renewable energy output in demand-response exchange market. Reference [7] uses Beta distribution functions and forward reduction algorithm to minimize scenes, maximizing the profits of power producers. Reference [8] uses scenario tree to describe the uncertainty of market. A two-stage stochastic optimization model is set to minimize the operating costs of energy hub operators. Reference [9] uses the Kantorovich method to reduce scenarios. A random bidding strategy for virtual power plants is proposed. Reference [10] proposes the integrated dispatch optimization model based on kernel density estimation. Reference [11] proposes a two-stage stochastic programming approach for electric vehicle integrators. Potential uncertainties are modeled as stochastic processes represented by different sets of scenarios. Reference [12] proposes virtual bidding to improve the market power of retailers, in which the demand uncertainty of strategic retailers is represented by scenarios.
To safely increase the penetration of fluctuating renewable energy, another promising measure is the incorporation of energy storage. In order to reduce the impact of electric vehicles rapid charging stations on the power grid, reference [13] proposes the bidding strategy of rapid charging stations with energy storage systems in energy and reserve markets. Reference [14] establishes energy management models for residential buildings. A peer-to-peer trading platform is developed to implement demand response plans. In [15], the co-investment of photovoltaic and energy storage is discussed. The five sources of post-investment profits are summarized in the day-ahead and intraday markets. Considering electric vehicle cluster as a storage system, reference [16] proposes an optimal bidding framework for regional energy internet in day-ahead markets. In [17], a data-driven real-time probabilistic characterization is proposed to complete the day-ahead scheduling problem of energy storage system owner. In [18], a hybrid interval robust optimization method is proposed and the optimal SoC interval is obtained. In order to improve the profitability of photovoltaic and energy storage system investors, reference [19] uses the Proximal Policy Optimization agent to allocate system capacity. Reference [20] proposes capacity optimization that takes into account the degradation of the battery. In order to analyze whether market clearing order affects participants, reference [21] defines two distributionlevel flexibility markets with different clearing time. The above literature shows that energy storage improves the penetration of renewable energy. However, considering the high cost of investment in energy storage, this paper makes the alliance between wind energy and existing energy storage. The wind-energy storage alliance reduces additional investment costs for power generation companies. A way for existing energy storage to increase profits is provided.
In [22], the stochastic problem of two-stage risk constrained scheduling for virtual power plants is presented in day-ahead, real-time and spinning reserve markets. Reference [23] examines the impact of strategic and nonstrategic participation of pumped storage facilities in dayahead energy and regulation markets. In [24], the capacity of energy storage to provide grid services by participating in energy and reserve markets is explored. Reference [25] proposes a minimization model of cost for electric vehicle parking lot aggregator, which take into account user comfort and market uncertainty. Based on multi-stage stochastic programming, reference [26] proposes the bidding strategies for virtual power plants in day-ahead and intraday markets. When participating in energy and frequency regulation markets at the same time, the above literatures focus on how to maximize the profits of power producers. To the best of our knowledge, there are few researches on the influence of wind-energy storage alliance's decision-making. Bidding strategies of market participants affect market clearing results. Similarly, market clearing results affect the bidding strategies of market participants. To maximize the profits of market participants, it is necessary to define a quantitative index for the bidding behavior to study the market impact brought by the decision.

C. CONTRIBUTIONS
Contributions include the following: 1) Energy system and probability scenario method are used to deal with the uncertainty of wind power output. Probability scenario method is adopted to reduce the redundancy of energy storage. Energy storage is used to deal with the error brought by probability distribution function of uncertain variables.
2) The wind-energy storage alliance is proposed. Wind power and energy storage are modeled as independent market players to maximize common benefits. Previous studies have mostly assumed that these energy sources operate jointly or that they are independent participants without interaction. Meanwhile, the alliance eliminates the investment in building energy storage.
3) Regulation participation ratio is defined. In addition to the research on which bidding strategy maximize alliance profits, the influence of bidding behavior on clearing results and profits is also studied. 4) A day-ahead stochastic bi-level optimization model in energy and frequency regulation markets is proposed. The proposed bi-level optimization model is transformed into a MILP model by KKT condition, large M method and strong duality theorem.

D. PAPER ORGANIZATION
The remainder of this paper is organized as follows. In section II, electricity trading framework is introduced in energy and frequency regulation markets. Section III introduces the mathematical formula of regulation participation ratio and bi-level model. The process of solving the model is described in section IV. Section V makes analysis through the case. In section VI, conclusions and future work are presented.

II. ELECTRICITY TRADING FRAMEWORK
Considering that most of the spot market electricity bidding takes place in the day-ahead market, this paper takes the dayahead market as the research background. The commodity attributes of electricity include quantity of electricity and quality of electricity, so the varieties of electricity market transactions are divided into two categories: energy services and regulation services. Electricity quality is affected by accident recovery ability, voltage stability and frequency stability. To make up for the shortage of contract volume and the difference in short-term load demand, market participants bid for energy services in energy market. Market participants bid for frequency regulation services in frequency regulation market to ensure power quality [27]. When electricity transactions in energy and regulation markets are delivered at the same time, bidding problems in different markets are interdependent. Therefore, it is necessary to establish a joint bidding and clearing model.  Figure 1 shows the trading framework of the energy and frequency regulation markets. The trading methods of this paper are as following: 1) The market adopts the joint clearing of energy and frequency regulation markets. Market participants include wind-energy storage alliances.
2) In order to maximize their own profits, market participants bid strategically based on the power generation capacity and the quotation limits. Then, the bidding plans are submitted to DSO.
3) Based on the dispatching constraints, DSO carries out market clearing. Location marginal price (LMP) is used for pricing. Clearing power and clearing price are fed back to market participants. 4) Although energy storage and wind turbines are independent market participants, the bidding strategy aims to benefit both parties. 5) To form an incentive mechanism in frequency regulation market, this paper refers to two-part tariff in Pennsylvania-New Jersey-Maryland (PJM) market. In twopart tariff, the comprehensive clearing price is divided into capacity price and mileage price. Mileage price is priced based on regulation performance [28].

A. REGULATION PARTICIPATION RATIO
The bidding profits in energy and frequency regulation markets are interdependent. The profits of wind-energy storage alliance are changed by bidding strategy. When wind-energy storage alliance bids more power in frequency regulation market, the bidding power in energy market is reduced, resulting in low profits in energy market and high profits in frequency regulation market. In order to conduct a detailed study on the influence of the market behavior, this paper defines the concept of regulation participation ratio. , is the bidding power of the unit  in the time period t . In energy market, unit  represent wind turbine unit i or energy storage unit es . , capoffer t P  is the regulation bidding power of unit  in the time period t .
,t D  is the regulation participation ratio of unit  in the time period t .

B. THE BI-LEVEL MODEL
In this paper, the spot joint market optimization is a bi-level optimization model. As shown in Fig. 2, the upper-level is the profit maximization model of the strategic bidder. The optimal bidding strategy is transferred to the lower-level. The lower-level is the objective function of DSO, whose aim is to achieve market clearing by minimizing the cost of power purchase. After market clearing is completed, the clearing price and clearing power are returned to the upper-level.
Considering the uncertainty of wind turbine output, multiscenarios in the spot joint market is adopted to simulate the deviation between the predicted output and the actual output.

1) UPPER LEVEL
The source of the alliance's profits is energy storage profits and wind turbine profits. The energy storage and the wind turbine bid in the energy and frequency regulation markets. Multi-scenarios w is used to describe the uncertainty of wind turbine output. In the upper-level, the profits of the wind- energy storage alliance are maximized in energy and frequency markets. The objective function of the upper-level is: where w  represents the probability of scenario w. The   respectively indicate the upper limit of the capacity and mileage bidding price of wind turbine i in the frequency regulation market. The bidding price for energy storage or wind turbines is limited by (6)- (12).

2) LOWER LEVEL
In the day-ahead energy market and frequency regulation market, the clearing mechanism is composed of sequential clearing and joint clearing. Although the sequential market clearing is easy to operate, the problem of network congestion is not solved and the global resources are not optimally distributed. The calculation of joint clearing is large, but it effectively overcome the disadvantages of sequential clearing. Therefore, this paper adopts the joint clearing mechanism.
In the lower-level, the power purchase cost of DSO is minimized in energy and frequency regulation joint markets. The objective function of the lower-level is: ,, ,  (18) and (19). The state of stored energy is calculated by (20).

D mt
P is the load of node m during period t. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. , The clearing power is limited by the bidding power of the unit in (23) - (27).
Upper and lower limits of line transmission power min max , , , , where mn F is the upper limit of transmission capacity of line n-m. The line power is limited by (28).
Upper and lower constraints of voltage phase angle: The voltage phase angle is limited by (29

IV. SOLUTION METHOD
Transforming the bi-level model to the single-level model is one of the effective solutions [29]. The decision variables of the upper-level are constant in the lower-level, so the lowerlevel is a linear model. The lower-level is replaced by equivalent KKT conditions [30]. The bi-level optimization model is transformed into the single-level mathematical planning problem with equilibrium constraints (MPEC). Then, MPEC is transformed into MILP model by strong duality theory and large M method. The process of transformation is shown in Fig. 3.

A. THE KKT CONDITIONS OF THE LOWER LEVEL
The KKT conditions of the lower-level model consists of equality constraints (14) - (22) and formulas (30) , , Re max min , , ,

B. CONVERT MPEC TO MILP
For the wind-energy storage alliance, MPEC is composed of (2) -(12), (14) - (22) and (30) The linear term is obtained by (32), (40)- (41) and (50): After the linearization of MPEC, MPEC is transformed into a MILP problem. The commercial software of Gurobi or CPLEX is used to solve MILP. Diagonalization algorithm is adopted to solve the transformed single-level game model. The flow chart of diagonalization algorithm is shown in the Fig. 4. Firstly, the bidding strategy of each wind-energy storage alliance is calculated. Then, the bidding strategy is updated. Finally, the calculation is stopped until the alliance's bidding strategy is converged or the maximum value of iterations is reached.

V. CASE ANALYSIS
In order to verify the effectiveness of the bi-level bidding model, IEEE 33 node are taken as an example to simulate without considering transmission fault. The parameters of alliances and corresponding nodes are shown in Table 1. The alliances are named as wind-energy storage alliance 1, 2 and 3. The SOC of energy storage is limited in 10%-90%. The SOC of the initial period and the end period are both 10%. Charge and discharge efficiency are 92%. It is assumed that the actual charging and discharging behavior of energy storage is same as the clearing result. The system regulation capacity requirement is 1% of the load. The regulation mileage requirement is obtained by multiplying the regulation capacity requirement by the system historical mileage-capacity ratio (take 10). Regulation performance indexes of each unit are shown in Table 2. Four scenarios are set up to simulate power output. Based on historical wind power data from Elia Energy, the typical curve and probability of wind power output are obtained by K-means clustering algorithm in multiple scenarios. The output of wind power and load are shown in Fig. 5 and Fig. 6 respectively, and the output uses per-unit value.

A. MARKET IMPART ANALYSIS ON REGULATION PARTICIPATION RATIO
Considering that the power demand of the system in frequency regulation market is less than the power demand in energy market and the restriction of biding capacity in frequency regulation market, 0.1, 0.2 and 0.3 are taken as low, medium and high regulation participation ratio respectively. The influences on DSO operation cost, clearing price and clearing power are studied successively. Different market players have different characteristics.
Wind power output is instability. Energy storage has fast ramp capability and excellent regulation performance. To study whether the market influence brought by different market players is the same, wind power and energy storage are set as following: Scheme 1: The regulation participation ratio of energy storage is changed from 0.1 to 0.3, and the regulation participation ratio of wind turbine remains unchanged at 0.2.
Scheme 2: The regulation participation ratio of wind turbine is changed from 0.1 to 0.3, and the regulation participation ratio of energy storage remains unchanged at 0.2.
1) The influence of regulation participation ratio on clearing price As shown in Fig. 7, the clearing price in energy market and comprehensive clearing price in frequency regulation market are analyzed.
In Scheme 1 of Fig. 7, with the increasing participation ratio of energy storage, the clearing price of energy market increases significantly. The bidding power of energy storage and the power purchased by DSO from energy storage are decreased in energy market as the increasing participation ratio of energy storage. Energy storage charges when the power is sufficient and discharges when the power is insufficient. Therefore, the clearing price at load peak is reduced and the clearing price at load valley is raised by energy storage. Due to the instability of wind turbine output, DSO tends to buy power from energy storage during peak hours. Therefore, the effect of energy storage at load peak is greater than that at load valley. For the above reasons, in energy market, the reduced bidding power of energy storage increases the clearing price.
In Scheme 1 of Fig. 7, the increase of regulation participation ratio reduces the comprehensive clearing price in frequency regulation market. Increased regulation participation ratio raises bidding power of energy storage in frequency regulation market. The energy storage with low bidding price and excellent regulation performance is more likely to win in frequency regulation market. As excellent regulation performance of energy storage increases the actual regulation mileage, the cost of capacity is reduced and the cost of mileage is increased in frequency regulation market. The reduction of capacity cost is more than the increase of mileage cost. So, the clearing price in frequency regulation market decreases. In Scheme 2 of Fig. 7, when the participation ratio of wind power is increased, the clearing price in energy market is decreased significantly and the comprehensive clearing price in frequency regulation market does not change obviously. The increased regulation participation ratio of wind turbines reduces the amount of bidding power in energy market. Therefore, energy storage wins the bid and the clearing price of energy market is effectively reduced. Although the bidding power of wind turbine in regulation market is increased, the bidding power of energy storage is not changed. In frequency regulation market, DSO prefers to buy regulation power from energy storage. The clearing power changes little in frequency regulation market, resulting in the change of comprehensive clearing price is not obvious.
2) The influence of regulation participation ratio on clearing power Figure 8 reflects the changes of energy storage and wind turbine total clearing power percentages with regulation participation ratio changes.
As shown in Fig. 8(a), with the increase of regulation participation ratio, the proportion of energy storage clearing power is decreased by the drop of bidding power in energy market. Compared with the decrease of the power proportion in Fig. 8(c), it is relatively mild. Because DSO tends to purchase power from energy storage during peak power hours, which slows the reduction in energy storage's percentage of clearing power. Figure 8(b) reflects the percentage of clearing power in frequency regulation market in Scheme 1. When D=0.1, the clearing power of energy storage is at an advantage compared with wind turbine in frequency regulation market. Energy storage with excellent regulation performance is easier to win the bid in frequency regulation market. The proportion of energy storage's clearing power is increased significantly in regulation market. Because the increase of the bidding power of energy storage in regulation market and the tendency of DSO to buy regulation power from energy storage.
It is seen from Fig. 8 (c) and (d) of scheme 2 that the percentage of clearing power is changed with the increase of regulation participation ratio in energy and regulation market. With the increase of regulation participation ratio, the percentage of wind turbines' clearing power increases obviously in energy market. The bidding power of wind turbines is reduced by the increased regulation participation ratio in energy market. Therefore, clearing power of wind turbines is reduced and clearing power of energy storage is increased in energy market. For wind and energy storage, the percentage of winning power is similar in frequency regulation market. The bidding power of energy storage is not changed. Although the bidding power of wind turbine is increased in frequency regulation market, DSO tends to buy the regulation power from energy storage.  3) The influence of regulation participation ratio on profits Capacity profits are settled by day-ahead clearing results. Mileage profits are settled according to actual regulation mileage. Therefore, wind-energy storage alliance 1 is selected as examples to discuss energy profits and capacity profits. In scheme 1 and 2, it is seen from Fig. 9 that the profits of energy storage and wind powers are changed with the change of regulation participation ratio in energy and regulation markets. As shown in Fig. 9(a), with the increase of regulation participation ratio, the profits of energy storage increase firstly and then decrease in the energy market. The clearing price is increased by increasing the bidding power of energy storage in energy market. However, with the further increase of regulation participation ratio, the clearing power of energy storage is further decreased in energy market. Although the clearing price is increased, the profits of energy storage decline. The reasons for the steady growth of wind turbine profits are the continuous increase of market clearing price and clearing power. As shown in Fig. 9(b), the increase of regulation participation ratio makes energy storage's profits increase continuously in frequency regulation market. With low comprehensive regulation bidding price and excellent regulation performance of energy storage, the clearing power of energy storage are increased in frequency regulation market, then its profits effectively increase in regulation market. However, the profits of wind turbine fall sharply. Influenced by the low bidding price of energy storage, the clearing price in regulation market is decreased and the clearing power of wind power is reduced.
As the regulation participation ratio of wind turbines increases, in Fig. 9(c) of scheme 2, the profits of energy storage in energy market increase and then decrease slightly. However, the profits of wind turbine declines. With the increase of wind turbine's regulation participation ratio, the clearing power of energy storage is increased in energy market. The clearing price of energy market is effectively reduced. Wind turbine clearing power is decreased resulting in the decline in the profits of wind turbine. At the same time, the profits of energy storage keep rising. Although the market clearing price is decreased, the clearing power of energy storage in energy market is increased significantly. As shown in Fig. 9(d), the profits of wind turbine and energy storage have no significant change. Although wind turbine improves the bidding power in frequency regulation market, DSO still tends to purchase regulation power from energy storage. The clearing price of frequency regulation market also has no obvious change. Therefore, the profits of energy storage and wind turbine are not affected.

B. ALLIANCE PROFITS ANALYSIS
In order to verify the economy of the alliance between wind power and energy storage, table 3 shows the profit comparison between the alliance and non-alliance considering the deviation penalty [31]. The profits of alliance are higher than those of non-alliance in energy and frequency markets. The bidding price of allied wind turbine is lower than that of unallied wind turbine. When the output deviation occurs, the deviation penalty cost of allied wind power is smaller than that of unallied wind power. Therefore, the total profits of wind turbine and energy storage is effectively increased.

VI. CONCLUSION
In this paper, a bi-level bidding optimization model of windenergy storage alliance is established by introducing regulation participation ratio. Market clearing and economic dispatch are the lower-level optimization objectives. Under the condition of uncertain wind output, the upper optimization objective is to maximize the profits of the windenergy storage alliance. In order to solve the model quickly and efficiently, the bi-level model is transformed into a MILP model by KKT conditions, strong duality theory and large M method. Finally, simulation verification is carried out. The results show that the comprehensive clearing price of the system is reduced by energy storage participation in frequency market. However, the mileage cost of the system is increased by the excessive participation of energy storage. Therefore, it is necessary to make a reasonable choice of regulation participation ratio in the bidding strategy of the wind-energy storage alliance. In this paper, the research of bidding mode provides reference for the bidding strategy of day-ahead market transaction. However, due to the complexity of the electricity market transaction, this paper is simple to deal with the electricity market transaction. The influence of network loss, transmission line congestion and how to distribute the common benefit are ignored. These problems are the focus of future research.