A Grid-Generator-Electrolytic Aluminum Multi-Agent Cooperative Game Model Based on Nash Negotiation Theory

In order to realize the local consumption of surplus electric energy, this paper proposes a multi-agent cooperative game model of power grid-generator-electrolytic aluminum based on Nash negotiation theory. The operation characteristics of the power grid, generator, and electrolytic aluminum are considered in the modeling of each agent. Then, a cooperative game framework considering electricity trading is further constructed based on Nash negotiation theory. Finally, a series of linearization methods are used to transform the complex problem into the mixed integer linear programming (MILP), which can be effectively solved by commercial solver. In the case studies, based on the actual situation of Yunnan power grids, a comparative analysis of the impact of cooperative and non-cooperative frameworks on multi-agents proves the feasibility of the cooperative game framework. Finally, the cooperative game model is applied to different provinces to verify its universality.


I. INTRODUCTION
As an important industrial electricity load, the high-energy electrolytic aluminum industry is widely distributed in Shanxi and Shandong, rich in coal resources, Southwest China, rich in hydropower resources, and the northwestern region, rich in renewable energy [1]. Compared with commercial and residential demand-side response resources, high-energy power user such as electrolytic aluminum has the advantages of large power consumption, easy adjustment, and less impact on production load [2], [3]. Considering that electricity cost can account for more than 30% of the cost of aluminum production, combined with China's increasingly stringent emission reduction policies, high-energy electrolytic aluminum companies have a high degree of enthusiasm in participating in grid market and auxiliary services. With the participation in the grid market and auxiliary services, high-energy The associate editor coordinating the review of this manuscript and approving it for publication was Yang Tang . electrolytic aluminum can contribute to local renewable consumption [4], [5].
On the other hand, renewable power generation such as wind and solar has developed rapidly in recent years. It is estimated that from 2019 to 2024, the global installed capacity of renewable energy will increase by 50% on the existing basis, and China will account for 40% of the global renewable energy capacity [6]. During the forecast period, new photovoltaics accounted for 60% of all new renewable energy sources, and distributed photovoltaics will usher in a rapid development stage. The total configured capacity will double to an installed scale of more than 500GW. With the gradual increase in the penetration ratio of renewable energy such as wind and solar, the intermittent and uncertainty brought by renewable energy itself will bring a huge challenge to the operation and control of the power system [7]. Therefore, how to improve the level of renewable energy consumption in the power system is an urgent problem to be solved [8], [9].
On October 31, 2017, the National Development and Reform Commission and the National Energy Administration VOLUME 9, 2021 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ issued the ''Notice on Launching Pilot Distributed Power Generation Market Trading'', which attracted wide attention from the industry [10]. The notice clearly stated that distributed power generation projects conduct direct power transactions with power users and pay ''grid-traffic fees'' to power grid companies. In addition, in order to promote the high-quality development of renewable energy and improve the market competitiveness of wind and solar power generation, China is gradually reducing subsidies for wind and solar power generation, and actively promotes the level-price grid of wind and solar power generation [11]. Therefore, through the bridge connection of the power grid, direct transactions between distributed generators and high-energy electrolytic aluminum users are realized, which is beneficial to improve the economics of distributed generators and high-energy electrolytic aluminum, and can also alleviate the transmission pressure of higher voltage grids. At present, there have been some related studies on the participation of high-power industrial users in the power system. For example, the authors of [12] constructed a joint operation model of electrolytic aluminum enterprises and wind farms, and studied the feasibility and economic analysis of electrolytic aluminum enterprises' participation in wind abandonment. The authors of [13] designed a grid demand response mechanism based on the energy efficiency of large users. The authors of [14] discussed the development trend of Henan's electrolytic aluminum industry from multiple perspectives such as market demand, resource regulation, and technological innovation. Reference [15] proposed a market research on the participation of high-energy electrolytic aluminum users in frequency modulation auxiliary services based on a decentralized cross-chain transaction model. Reference [16] studied the control strategy of the power supply system of the electrolytic aluminum plant under the isolated grid operation, and realized the safe production of electrolytic aluminum. The above research shows that the electrolytic aluminum load can effectively participate in the consumption of new energy such as wind and wind in the power system. Consider the national policy to guide the power grid to play the role of an intermediary, and use high-energy electric load to realize the consumption of renewable energy. However, there has not been any research on the above power grid-generatorelectrolytic aluminum multi-agent structure to carry out relevant work.
To improve the consumption of distributed renewable energy, peer to peer energy trading problems have been extensively studied. For instance, to adequately incentivize prosumers to participate in energy trading, two efficient mechanisms to construct a stable grand coalition of prosumers were proposed in [17], based on cooperative gametheoretic principles. Considering that using telecommunication systems can group energy prosumers constrained by physical structure into virtual microgrids, prosumer benefits can then be optimised by modelling the energy trading interactions among producers and consumers in a virtual microgrid as a Stackelberg game [18] In [19], an optimization model with a blockchain-based structure was developed to manage the operation of crowdsourced energy systems to address the rapidly transformed power grid. A cooperative energy market model consisting of three types of participants, the DN operator, buyers, and sellers, was formulated using the Generalized Nash Bargaining theory [20] Based on the Nash negotiation theory, a cooperative operation of the wind-lighthydrogen multi-agent energy system was established in [21], which considers the energy trading between the entities.
Compared with other works, the main contribution of this paper is to explore the potential of the cooperative game of power grid-generator-electrolytic aluminum based on the needs of national policies. Also, we apply this cooperative game model in several provinces with different electricity prices and solar energy. The main contributions are listed as follows: 1) According to national policy guidance needs in China, we propose a multi-agent cooperative game model of power grid-generator-electrolytic aluminum based on Nash negotiation theory. 2) In the model solution, the complex non-linearity problem is transformed into the mixed integer linear programming (MILP) with a series of linearization methods. Through the implementation of electricity prices and natural characteristics in different provinces, case studies prove the practicality of this cooperation model. The chapters of the article are arranged as follows: Section II gives the modeling of each subject, Section III describes the framework of cooperative games, and Section IV gives the model solution process. Section V shows the case analysis. Section VI gives the conclusion.  Fig.1 shows a typical Nash negotiation game model including multiple entities of power grid-generator-electrolytic aluminum. In practice, power grids, generators, and electrolytic aluminum companies belong to different stakeholders. Under the traditional non-cooperative framework, power generators directly sell electricity to the power grids at an on-grid price, while the electrolytic aluminum companies purchase electricity from the power grids at the industrial electricity price for industrial production. The article assumes that in the cooperative operation framework, generators can sign an agreement with the electrolytic aluminum companies, negotiate to determine the trading power and price, and directly sell electricity to the electrolytic aluminum companies through power grids. At the same time, the grid company will charge the corresponding cross-grid for the electric trading energy between electrolytic aluminum companies and generators. As a result, the grid-generator-electrolytic aluminum constitutes a cooperative game framework to optimize the grid fee price charged by the grid and the electricity price and electricity sold by the generator to the electrolytic aluminum user. Fig. 1 shows the structure of grid-generator-electrolytic aluminum. The modeling of them is given as follows in sequence. Fig.2 shows the structure diagram of the electrolytic aluminum production process. The production of the modern electrolytic aluminum industry generally uses cryolite-alumina molten salt electrolysis. First, the carbon material is used as the two stages of cathode and anode, and inserted into the electrolyte with molten salt cryolite and alumina as solvent and solute, respectively. Through direct current electrolysis, gases such as electrolytic aluminum liquid and carbon dioxide are respectively generated at the anode and cathode. As an important industrial load, electrolytic aluminum can be used as an important demand-side resource to achieve peak-shaving and valley-filling of power loads, and improve the reliability and economy of the power system. It is assumed that the high-energy electrolytic aluminum load includes non-transferable load and transferable load. The non-transferable load must be met in a scheduling period, and the transferable load can be transferred in and out in a scheduling period based on actual demand. The specific constraints are as follows:

A. MODELING OF HIGH-ENERGY ELECTROLYTIC ALUMINUM WITH DEMAND RESPONSE
Formula (1) indicates that the electrolytic aluminum load P E t at each time t is composed of the non-transferable load P EF t , the transfer-in load P EI t , and the transfer-out load P EO t ; formula (2) represents the electrolytic aluminum load at each time t comes from the sale of electricity by generators P S t and the sale of electricity from grid company P G+ t ; constraints (3)-(4) limit the maximum transferable load at each time t, where ϕ is the maximum transferable load coefficient, with a value ranging from 0 to 1; constraints (5) limit the total load transferred out in one scheduling period to the total load transferred in, where T is equal to 24h.
The goal of electrolytic aluminum load is to purchase electricity from outside at the lowest possible price, so its objective function is as shown in equation (6): where, c G+ t and c S t are the unit purchase price of electrolytic aluminum load from power grids and generators.

B. MODELING OF DISTRIBUTED GENERATORS
The goal of distributed generators is to sell their own power generation at a higher electricity price as much as possible, but they need to pay a certain grid fee to the grid. Therefore, the objective function is as shown in formula (7): where P G− t is the power sold by the generators to the grid company, and c G− t and c fee are the unit price of electricity sold by the generators to the grid company and the unit grid fee charged by the grid, respectively.
Generators include conventional thermal power plants and renewable energy generators. Here, we use distributed photovoltaic power generation as an example. Considering the randomness of distributed photovoltaic output, we assume that the distributed photovoltaic output is within a certain confidence level η [22]: where p PVF t is the predicted value of photovoltaic output, ϕ −1 a (1 − η) is the inverse cumulative distribution function of the standard normal distribution N (0,1), and σ PVF t is the variance value of the predicted value of photovoltaic output and the actual value, which is set as 5% in this article. Equation (9) indicates that the actual output of distributed photovoltaics per hour is equal to the sum of the power sold to the grid company and electrolytic aluminum users. VOLUME 9, 2021

C. MODELING OF POWER GRIDS
The operation constraints of the power grid need to be met in practice. To calculate the power flow of the power grid, we can adopt an AC power flow model, a DC power flow model, or a linearized AC power flow model. Among them, the AC power flow model will introduce nonlinear terms, while the DC power flow model is too simplified. Compared with the DC power flow model, the linearized AC power flow model allows the voltage deviation in the networks, which can not only ensure the convexity of the model, but also take into account a certain degree of accuracy. Therefore, the linearized AC power flow model [23] is used for modeling power grids.
where Eqs. (10) and (11) limit the maximum interactive electric power between power generators and electrolytic aluminum users. P G max is the maximum transmission power of the tie line. The relevant constraints of the linearized AC power flow model, with the principle of using the octagon in to approximate the circular constraint depicted in Fig. 3, are given in Appendix A.
By charging a certain grid fee, the grid profit is as follows:

III. COOPERATIVE GAME FRAMEWORK
Based on the Nash negotiation theory, the objective function of the Nash game model shown in formula (13) is constructed, which is composed of the opportunity item of the profit increase of each stakeholder. The objective function can realize the fair distribution of the profits of all stakeholders. In addition, formulas (14)- (16) ensure that all stakeholders can increase their benefits through cooperation. Among them, profit G 0 , profit L 0 , and profit P 0 are the benefits of grid companies, electrolytic aluminum users, and generators before cooperation, which is also known as the breakdown point of the negotiation.

IV. MODEL LINEARIZATION AND SOLUTION
The above model has obvious non-linear characteristics, including the product term of price and power in formulas (6), (7) and (12) and the product term of the objective function (13). In order to solve the model, the following linearization process is performed. Regarding (6), (7) and (12), the product term form of price and power can be discretely processed by the 2 K linearization method [24]. For example, c fee P S t be expressed by the following formula: In formula (17), the grid fee is discretized into k∈K 2 k−1 z fee k levels, where c fee is the price granularity and calculated by formula (18). The smaller the value, the more accurate. c fee l and c fee m are the minimum and maximum values of the grid fee, respectively. z fee k is the optimize variable for the integer grid fee. Formula (19) is a discretized product term, but there is still a product term of the integer variable and the connected variable. Therefore, we introduce the auxiliary variables v fee k = z fee k P S t , which need to meet the following constraints: where M is a big value. Similarly, c S P S t can be linearized in the same way.
where c S is the granularity of electricity sale price, which is calculated by formula (26). v S k is an auxiliary variable, and its value is equal to z S k P S t , c S l and c S m are the minimum and maximum value of the unit price of electricity sold by the generator to the electrolytic aluminum user, and z S k is the integer price optimization variable.
Regarding the product term (13) in the objective function, it can be characterized as the following logarithmic differential form, which can eliminate the nonlinear term.
The piecewise linear function of (µ iq = ln(profit G i − profit G i,0 )) with Q grid points shown in formula (28) is used to obtain the logarithmic differential of different stakeholders of electrolytic aluminum load, generators, and grid company. λ G q , λ L q , λ P q represent a set of special sequence optimization variables for grid company, electrolytic aluminum load and generators. (29) and (30) respectively define it to be positive, and its sum is 1. Formulas (31)-(33) limit the linearized objective function value of the grid company, generator and electrolytic aluminum load to be equal to the original target value. The YALMIP toolbox in MATLAB and CPLEX are used to solve the above-linearized problem. All numerical simulations are performed on a 64-bit PC with a 3.40-GHz CPU and 8-GB RAM.
Taking Yunnan Power Grid to conduct research, consider the operating pressure on Yunnan Power Grid caused by the asynchronous interconnection operation of Yunnan Power Grid and China Southern Power Grid in 2016 [25], [26]. Fig.4 further shows the peak-to-valley time-of-use electricity prices of large industrial users in Yunnan Power Grid. The unit photovoltaic output of distributed photovoltaic power generation companies is shown in Fig.5. Fig.6 shows an example structure of an IEEE-39 transmission network node, in which distributed generators and electrolytic aluminum loads are connected to node 35 and node 22, respectively. Based on the above data and example model, the validity of the method proposed in this paper is verified.

A. ECONOMIC COMPARISON
We set the following four scenarios: 1) conventional transaction framework S1 that does not consider cooperation, that is, VOLUME 9, 2021 distributed generators sell electricity to grid companies at the price of grid electricity, and grid companies sell electricity to electrolytic aluminum users at the sales price of industrial electricity; 2 ) Framework S2 considers the demand-side response capability of electrolytic aluminum load on the basis of 1); 3) Frameworks S3 and S4 construct cooperative game models on the basis of S1 and S2, respectively. Table 1 shows the economic comparison of the four frameworks. It can be seen from the table that in S1, the power grid does not charge a grid fee, the power generation company gains 29,400 by selling electricity to the grid, while the electrolytic aluminum user purchases 129,300 by selling electricity to the grid company. It should be noted that the objective function of the power grid is the grid fee rather than its own electricity sales profit fee, so its value is 0 . In S2, after the demand-side response, the electricity purchase cost of electrolytic aluminum users was reduced to 112,300 . Compared with S1 and S3, after considering the cooperative game, the power grid can obtain 7,800 by charging the grid fee. At the same time, the electricity sales cost of the generators has increased to 35,900 , and the electricity purchase cost of electrolytic aluminum users has been reduced to 121,800 . Compared with S3, S4 takes into account the demand response and the grid benefit is slightly reduced, while the cost of electrolytic aluminum users is the smallest in the four frameworks.
In terms of optimizing prices, the price of electricity sold to electrolytic aluminum users by generators in S3 is 0.56 /kWh, of which the grid charges 0.12 /kWh. After considering the demand response in S4, the price of electricity sales becomes 0.5 /kWh, and the grid crossing fee is reduced to 0.08 /kWh. Considering that the fixed on-grid electricity price is 0.34 /kWh, the cooperative mechanism can significantly increase the electricity sales price of generators. In addition, from the perspective of energy consumption, the electricity price of 0.50-0.56 /kWh is also lower than the peak and normal sales price. Fig.7 shows the comparison of the power purchase strategy of electrolytic aluminum load under S1 and S3, and Fig.8 shows the comparison of the power purchase strategy of electrolytic aluminum load under S2 and S4. It can be seen from the figures that all the electrical energy of the electrolytic aluminum load under S1 comes from the upper-level power grid, while the electrolytic aluminum load under S3 can purchase electricity from the upper-level power grid and the  power generation company at the same time. Among them, during the period of 8:00-17:00 photovoltaic output, more electricity is purchased from generators. In other periods, because the generators cannot provide electricity, the electrolytic aluminum load can only purchase electricity from the upper-level grid. After considering the demand response of the electrolytic aluminum load, it can be found that the electrolytic aluminum load is purchased from the power grid during the trough period from 11:00 to 7:00 the next day, and the electricity purchased from the power generation company during the period of 8:00-17:00 photovoltaic output. That is to say, the electrolytic aluminum load can purchase electricity from distributed generators and the power grid during low electricity prices to the greatest extent, so its energy cost is further reduced. The above results show that the electrolytic aluminum load can benefit by forming a cooperative game with the power grid and power generators, and its own certain  demand response capability can further reduce its own energy cost. Fig.9 shows the comparison of the electricity sales strategies of distributed generators under S1 and S3, and Fig. 10 shows the comparison of the electricity sales strategies of distributed generators under S2 and S4. It can be seen from the figures that under the non-cooperative framework, distributed generators can only sell electric energy to the grid. Under the framework of cooperation, generators can sell most of the electric energy to electrolytic aluminum users on-site, and only sell part of the electric energy to the grid when the electrolytic aluminum load cannot be absorbed during 11:00-15:00. Similarly, the electrolytic aluminum load with a certain demand-side response can also benefit power generators. Because the electrolytic aluminum load can absorb the electric energy of distributed generators to a greater extent, it is increased from 9MW in S3 to 13.5MW in S4. The above results show that this cooperation framework can also  benefit distributed power producers and is more conducive to promoting the consumption of renewable energy.

B. ANALYSIS OF TYPICAL DAILY OPERATING SCENARIOS
In addition, from the perspective of the power grid, after the cooperation, electrolytic aluminum users purchase electricity from the upper-level grid less, and it is easier to purchase electricity from the upper-level grid during the low load period, and distributed generators also reduce the electricity sales to the upper-level grid. Therefore, the transaction volume between the source and the load with the power grid is reduced, which is beneficial to alleviate the transmission pressure of the upper-level power grid.

C. ANALYSIS OF OTHER REPRESENTATIVE PROVINCES
In order to study the applicability of the cooperative game model in other provinces, we further give the performance of Shandong, Sichuan, Shanxi, Xinjiang and Jilin and other representative provinces. The unit photovoltaic output is calculated by the PV calculator developed by the National VOLUME 9, 2021  Renewable Energy Laboratory of the United States, and the industrial sales prices of different provinces come from the provincial power grid companies, as shown in Fig. 11 and Fig. 12, respectively. Based on the above cooperative game framework of S4, Table 2 presents the profit/cost of power grids, power generators and electrolytic aluminum. Among them, Xinjiang Power Grid charges the highest grid fee, followed by Shanxi, and Jilin charges the lowest grid fee. From the perspective of power generation companies, Xinjiang's distributed power generation companies have the highest revenue, at 51,800 . This is mainly due to the abundant light resources in Xinjiang. From the perspective of energy consumption, the same electrolytic aluminum users in Xinjiang have the lowest electricity purchase costs. In addition to benefiting from Xinjiang's abundant light resources, Xinjiang's electricity sales costs are also the lowest. In sharp contrast with Xinjiang, Jilin, where the power grid and generators have earned 11100 and 42,100 , respectively, are the lowest among all provinces, while the electrolytic aluminum purchase cost is the highest, which is 10,0700 . Table 3 further shows the optimized electricity selling price and the grid fee charged by the power grid from electrolytic aluminum users. It can be seen from the table that the highest electricity selling price in Shanxi is 0.52 /kWh, and the lowest electricity selling price in Xinjiang and Jilin is 0.44 /kWh. The highest grid fee in Xinjiang is 0.24 /kWh, and the lowest in Shandong is 0.13 /kWh. In combination with Table 2, it can be found that Xinjiang benefits from abundant light radiation intensity, and the same power generators can obtain the highest revenue under the condition of the lowest optimized electricity selling price.

VI. CONCLUSION
In order to study the bidding strategy of high-energyconsuming electrolytic aluminum loads in the power market, this paper establishes a grid-generator-electrolytic aluminum multi-agent cooperative game model based on the Nash negotiation theory. Then, we propose a solution algorithm for the cooperative model. Case studies prove the advantages of the cooperative game framework compared with the traditional framework. The results show that the cooperative game model can increase the revenue of power generators, reduce the energy cost of electrolytic aluminum, and alleviate the transportation pressure on the upper grids.
This work mainly discusses the possibility of cooperation among the grid-generator-electrolytic aluminum multi-agent. In the future, more distributed renewable generators and electrolytic aluminum users will participate in this trading mechanism. Considering the privacy of each agent's information, it is necessary to use the distributed algorithms to achieve energy trading based on actual operating conditions. where Eqs. (A1)-(A4) limit network voltage constraints, node net injection active power and reactive power. Eqs. (A5) and (A6) give the voltage constraints and line power constraints in the power network, respectively. VS n t and VSn t are the voltage square terms of node n and respectively, R n−n and X n−n are the resistance and reactance of the line, P n−n and Q n−n are the active power and reactive power of the line n−n, C n−n is the transmission capacity of the line n −n,V n2 and V n2 are the upper and lower limits of the squared voltage term. V 2 0 are the squared voltage of the reference node n.