Application of Game Theory in Integrated Energy System Systems: A Review

With the rapid development of society, global energy is in short supply. China has put forward an integrated energy system focusing on interconnecting energy resources and harnessing their complementary advantages to address the energy crisis, thus no longer pursuing the production, transportation and supply of a single source of energy. In the continuous development of integrated energy systems, the elements of participation and interaction are becoming more complex. Game theory, which can effectively solve the problems arising from multi-agent trading, is naturally introduced to the integrated energy system. This paper provides a comprehensive overview of the introduction of game theory into the integrated energy system. First, the development of integrated energy is briefly introduced, game scenarios in integrated energy systems is proposed, and game scenarios considering the energy supply side, distribution network, demand side and common planning and dispatching problems in integrated energy systems are summarized. Secondly, main game theory models in the integrated energy system are summarized, such as cooperative game theory model, the non-cooperative game theory model and the Stackelberg game theory model. Finally, the future prospect and challenge for the application of game theory in integrated energy systems is proposed. The new game models are introduced to the integrated energy system, and a mixed game is considered to solve related problems. It is hoped that this work can serve as a reference for the researchers in this field.


I. INTRODUCTION
With the development of society, humans need more energy support. In the past, a series of single energy sources, such as electricity, gas and heat, became unable to meet the current needs of society, so integrated energy systems (IESs) emerged [1]. In this context, many researchers in China and abroad have studied IESs. In view of the cost effectiveness of urban IESs, a strategy for solving this problem was presented abroad, and a method based on reliable scenarios was simultaneously proposed. A comprehensive IES model was proposed for the rural environment considering the cost factors and the impact of energy consumption factors [2]. China has also attached great importance to IES and introduced many policies to support it. Researchers in China have proposed strategic thinking based on the mutual integration of energy, The associate editor coordinating the review of this manuscript and approving it for publication was Amjad Anvari-Moghaddam . the economy and the environment and corresponding solutions for energy challenges and difficulties in IES. In [3], IESs were studied macroscopically. In [4] proposed community integrated energy systems (CIES) and investigated how to achieve the optimal strategy given CIES dispatching problems. There have also been some studies in China considering energy conversion and storage issues [5].
The application of and research into IES are increasing in depth, and the number of problems requiring urgent solutions in IES is growing (such as coupling between energy sources and interests between different subjects). Therefore, game theory, which has been widely applied to single power systems, can be used for reference. Game theory has been well integrated into the power market. Game theory started as a concept in the field of economics and was introduced into various fields to solve related problems, including the power market, power system dispatching, and power system control [6]. Historically, the power industry was monopolistic, but with the reform of the power market, competition intensified. In [7] introduced the concept of game theory as a way to balance the market and to maximize the interests of each subject. A new mechanism was proposed to show that market supervision was wise to the market. In the same way, game theory was introduced to solve problems with the smart grid. In [8], two situations were considered that focused on the problem of shared storage in smart grids: having a single storage device to achieve the minimum power consumption cost and cooperating with consumers. Cooperative game theory was used for modeling. In [9], electric vehicles were added to the framework of the smart grid, so the factors considered increased accordingly, and the framework of a three-party game was used to solve the problem of energy management. This study also shows that when more elements are involved in a system, the interaction becomes more complex. Introducing game theory to solve the problem achieves twice the result with half the effort. Game theory can also be used when electric vehicles are added to the framework of the distribution network. In [10] observes that plug-in electric vehicles had a considerable influence on the distribution network. To avoid affecting the security of the distribution network, a non-cooperative game method was proposed to design a model and used for price driven.
Most of the applications were on the supply and distribution sides of the power system, but it should be noted that game theory is more widely applied to the demand side of the power system. Reference [11], summarized relevant literature on the application of game theory to distributed energy users, high-energy users and medium-low energy users. Reference [12] provided a comprehensive overview of the main methods in game theory and added game theory methods and strategies to the increasingly open power market and management system. The above two references fully illustrate the application of game theory to the demand side of power systems. Increasingly, game theory and power systems are becoming tightly bound because of the variety of methods and effective strategies used in game theory to solve various problems in power systems.
IESs are more complex than traditional power systems. The coupling between forms of energy is more diverse, and the aspects to be considered and the problems to be solved are also complex. First, the different main forms of energy should be considered in IES. Traditional energy forms include electricity, natural gas, cold and hot energy. These individual sources of energy are coupled together to achieve the goal of optimizing energy networks.
Given the complexity and uncertainty around energy networks, [13] proposed a framework for the robust optimization of hybrid energy devices and energy networks. In this framework, staged iterations were used to improve iteration accuracy. However, in addition to solving problems between energy players, the energy market must also be considered. As seen in the electricity market, market issues are particularly complicated, and this is truer in regard to IES, the players in the energy market and their relationship network. The main players in energy trading and commercial operations in IES include IES service providers, different types of IES users, and manufacturers of energy storage conversion equipment. There are many players, and their starting points, interests and needs are all different, yet a solution is needed that improves the various systems in this huge energy market and maximizes the interests of all parties. Reference [14], proposed a daily leading dispatching strategy based on an auxiliary service market and combined energy based on uncertain factors, such as energy market price, auxiliary service market price, wind power and photovoltaic power. Finally, the connection between the network and external actors also needs to be considered in an urban multi-energy system market, such as the connection between the market and artificial intelligence technology. Under the continuous reform of energy markets, problems such as the balance between available energy and pricing strategy must be solved using artificial intelligence technology to forecast the price and improve prediction precision. Reference [15], proposed a hybrid nonlinear regression and support vector machine model for daily electricity price prediction and used a support vector machine to predict prices during peak periods.
It is clearly necessary to apply game theory to IESs to address the complex elements and interests. At home and abroad, study has begun on this aspect of the problem. Collaborative economic dispatching in energy hubs was investigated. In [16], a cooperation model contrary to the traditional non-cooperative mode was proposed and bargaining game theory applied to achieve the optimal fairness in the problem solution and to minimize operating costs. Reference [17] proposed the concept of intelligent energy hubs, and game theory was used to model the demand side of the energy hub to address the problem of energy consumption while considering the amount of data storage. In [18], wind power capacity was reasonably planned based on a cooperative game in a low-carbon economy environment, and a cooperative game was used to model wind power capacity and solved by the grid search method. Specifically, a solar power generation and cogeneration sharing system was proposed, and a joint game model was established for the system, which was solved by using distributed operations to minimize the operating cost in the alliance. Reference [19] proposed a dynamic game-based dispatching mechanism for natural gas networks and grids aiming to minimize the loss of power and gas loads. A mixed integer linear model was established, and the effectiveness of the model was demonstrated by an example.
IESs are currently a hot topic, but game theory has rarely been applied to IESs, although its introduction is highly necessary. The principle of game theory is that each player makes a self-interested decision based on his or her own ability or information when there are interest-related conflicts between multiple decision makers. IESs are well suited to game theory due to their multi-stakeholder and conflicting nature.
This paper provides a brief overview of IESs and game theory based on the above discussion, summarizes the most recent applications of game theory to IESs, and summarizes the game scenarios, corresponding game models and algorithms for IESs. Therefore, this paper provides an overview of integrated energy in the second section, and the third section introduces game theory for the interactive subjects and the corresponding scenarios in the IES. The fourth section comprehensively introduces the game models and algorithms used in IESs. Section V summarizes and discusses the future challenges of game theory application to IESs and the key points for follow-up work.

II. OVERVIEW OF INTEGRATED ENERGY SYSTEMS A. DEFINITION OF AN INTEGRATED ENERGY SYSTEM
IESs are generally energy systems formed by combining two or more electricity, gas, and cold and hot energy sources. An IES is an integrated system of production, supply and marketing. It can produce, distribute, transmit [20] and store energy well [21] in the process of planning, construction and operation. Figure 1 shows a structure diagram of an IES, including the raw materials for the energy supply, transmission and distribution network, energy conversion, energy storage, and end users; this illustrates the development of IESs from the source-source operation mode to the sourcenetwork-load-storage integrated operation mode [22]. In [23], [24], the incorporation of renewable energy into the energy system was found to improve environmental problems with no impact on economic development. However, uncertainties exist with the renewable energy, such as with wind power [25] Although integrated energy is gradually maturing, there are still many shortcomings [26], [27]. IESs are constantly being optimized, and in addition to meeting the needs of all parts of the system, an IES must constantly improve the efficiency of energy coupling, effectively utilize all kinds of resources, and work towards sustainable development [28]. IESs mainly include the coupling of the electric-gas system and the coldhot-electric system.

B. ELECTRIC-GAS SYSTEM COUPLING
Power and natural gas are both traditional energy systems, but both have the problem of low energy utilization; combining the two single energy systems can form an electric-gas integrated energy system to improve energy efficiency. In [29], power and natural gas are both traditional energy systems, but both have the problem of low energy utilization, so the two single energy systems are combined to form an electricgas integrated energy system to improve energy efficiency. In [30] examined the coordination of interdependent electricity and gas infrastructure, proposed a basic framework for the analog coupling and dispatching of power and natural gas, and optimized the gas compressor with various optimization algorithms. However, investigation was needed into how to establish the corresponding mechanism and make an effective transition to power-gas systems in the future. The answer was given in [31], which discussed how power to gas (P2G) can provide good transition operations in future energy systems.
Additionally, various application methods were discussed, which offer effective help for future strategic energy plans and strategies.
The power-gas energy system is mainly based on gas generators and P2G technology. The gas turbine was first proposed in 1791, and it has experienced more than 200 years of development. Gas units have been widely used in the industrial field, but the optimal utilization of gas units in IES is still under study.
Reference [32], established a boiler model of a gas turbine based on the step-by-step superposition method and the piecewise cooling model; a medium-pressure cylinder exhaust method is proposed. A series of conclusions were presented, such as the linear relation between the gas turbine load and the efficiency values and the linear relation between the exhaust coefficient and the efficiency values. In [33] notes that the existing power-gas coupling system still has some under-researched problems. For example, coupling elements that only consider conventional gas turbines or electrically driven compressors cannot effectively reflect the bidirectional interaction between electricity and gas. At the same time, there will be errors between simulation modeling and practical engineering. Therefore, the anti-difference estimation method of the weighted minimum absolute value was applied to power-gas modeling to comprehensively and effectively study power-gas coupling. Compared with gas turbines, P2G technology is a hot research topic, and a large number of articles have proposed innovations for this technology. Reference [34], carried out research on the current energy system and future operations strategy of P2G technology, analyzed the economy and P2G technology, modeled the mode of operation and analyzed profits. The innovation of [35] lies in the proposed method of online Lyapunov optimization to optimize P2G technology in IESs. The innovation of [36] focuses on the contradictory relationship between a higher operating cost and wind energy consumption, which it coordinates using the weighted fuzzy programming method.
The power-gas coupling system has become increasingly perfected, and its infrastructure, operation mode, internal optimization, external optimization and many other elements have become more mature, which lays the foundation for future sustainable energy development.

C. COLD THERMAL-ELECTRIC SYSTEM COUPLING
A combined cooling, heating and power system (CCHP) realizes a cold-hot power supply based on a natural gas system. The CCHP system mainly includes the generator set, the heating unit and the refrigeration unit, of which the generator set includes the gas turbine, the internal combustion generator set and so on; the heating unit includes the gas boiler, the heat exchanger and so on; the refrigeration unit includes the electric refrigeration, and the heating to cooling group. The joint operation mode of CCHP systems has greatly improved the utilization rate and operation efficiency of energy production and is the strongest potential direction for IESs at present as well as a research hotspot. At present, there are research results for optimizing and perfecting a CCHP system. In [37], the CCHP system was optimized considering the aspects of economic operation, dynamic analysis, and environmental effects, and the operation strategy in the CCHP system was evaluated by using the weighting and fuzzy optimal selection methods combined with practice. Finally, it was shown that the CCHP system was optimal in most cases when renewable energy and a heat storage tank were added. In [38], the performance of the power load, heat load, and mixed power load operating under different strategies in the CCHP system was analyzed for office and residential groups. Under two different incentive policies (incoming tariffs and carbon emissions), it was concluded that under different strategies, the CCHP system was suitable for different scenarios. In the CCHP system presented in [39], three-level collaborative optimization was realized: the selection of the first-level system equipment, the second-level system capacity configuration, and the third-level operation parameters. Application of the particle swarm optimization algorithm showed how the whole system could achieve opti-mal energy savings, environmental protection and economic benefits.
The coupling between CCHP systems is an important part of IESs. Therefore, more innovative research is needed to determine how CCHP systems can play a greater role in IES and how to use energy reasonably and efficiently.

III. GAME THEORY AND INTEGRATED ENERGY SYSTEM GAME SCENARIOS
A. GAME THEORY Game theory is aimed at optimizing the best interests of many subjects with certain constraints under a specific scenario. In the course of the game, individuals rely on their own information and, in a sequence or simultaneously, form their own strategy once or repeatedly to determine the optimal strategy to carry out the actual game. Typically, a complete game consists of three elements: the players, the strategy, and the payoff (also called the payoff function).
A player is an organization or individual in a game who is able to choose a strategy and make a judgment.
The above expression represents the set formed by the participation of N players in a game process. Strategy is an important element in the game, and the feasibility of the strategy and the number of strategies can be selected by the players collecting information in the game process and resources.
The above formula represents a strategy set formed by a player's own policies and a strategy set of all players' respective strategy sets. Profit refers to the benefits that the players receive after the game and most of the time is the maximum benefit they will receive, but there may also be negative values.
The above expression captures the benefit of all the players in the game. After the above three elements are determined, a complete game is established. Game models are divided into classical games and evolutionary games. Classic games can be divided into cooperative games or non-cooperative games and into static games or dynamic games according to different criteria.
Cooperative games and non-cooperative games are distinguished by whether the players have reached a cooperative relationship over the course of the game. The two key elements in the process of cooperative games are the distribution of benefits in the process of cooperation and cooperation under the agreed-upon conditions [40]. The non-cooperative game process is mainly based on the concept of the Nash equilibrium.
Static games and dynamic games are mainly determined according to the order in which players choose actions. In a static game, players select strategies and act at the same time, while in a dynamic game, players determine strategies to act successively. The Stackelberg game is a dynamic game [41].

B. THE SCENARIO FOR USING A GAME IN INTEGRATED ENERGY SYSTEMS
Game theory has been fully applied to the power system. It is necessary to use games to solve the problems of IES. First, on the energy supply side, the energy subject involved is no longer a single subject. The existence of the coupling mode between different forms of energy and players participating in the coupling while maximizing their own interests make it necessary to apply game theory to achieve an optimal solution. In addition to energy supply, the distribution network contains competition in the energy market, distribution network and other elements. The demand side contains more elements, such as individual residents, industrial parks, and distributed energy. These elements need to be included in the game not only with other demand-side users but also with energy companies and sales units. Figure 2 illustrates the internal game relationship between the three groups.

1) ENERGY SUPPLY SIDE
The game relationship on the supply side is mainly formed by coupling different energy sources, so there will be different game scenarios. First, based on coupling the triple supply of cooling, heat and power, CCHP is an important mode for complementary multi-energy optimization. In [42], an optimal multi-objective energy management framework for power and natural gas based on CCHP is proposed. Under certain restrictions, this work carries on the joint consideration of this framework and establishes the management mechanism. An IES including renewable energy is studied in [43] to establish a game scenario. The main players are renewable sources of energy in the energy system, taking into account energy, economic and environmental perspectives. Under the constraints of the energy rate, total operating cost, carbon dioxide emission reduction and joint optimization, the game is conducted, the different optimization models are balanced in the process of the game, the results are analyzed and compared, and finally, the optimal results are obtained. In [44], a dynamic electricity price mechanism is considered, and the game scenario takes each garden as the player accessing the CCHP triple supply. A strategy corresponding to factors such as the power output situation in the garden and the operation cost dispatching is formulated to set up the non-cooperative game model. The payoff function aims to reduce the daily operating cost of the garden and reduce the peak and valley difference in the grid load. Reference [45] presents a problem for the optimal dispatching of the energy and reserves of microgrids in CCHP. To construct the model, consideration is given to the wind speed or light intensity for wind and photovoltaic (PV) power generation, the power of the gas turbine, the heating power, etc., to establish the relevant strategy scheme; this scheme is then used to find the optimal dispatching model to diapatch all forms of energy to minimize the operating cost. Reference [46] is also based on the CCHP system and aims to optimize the IES. In the game scenario, the factors in the game are the operating cost, the construction cost and the energy transaction cost in the system. The players are the electricity and gas regions. The strategy in the game is the choice of power level and flow level of the distribution line and gas network. The payoff function of the game is to make the CCHP more economical and obtain the best distribution lines and gas pipelines.
Another conventional form of coupling is combined heat and power (CHP), which transmits both power energy and heat energy to users in a certain way. Reference [47], determines that the CHP system should not only improve the reliability of the system but also maximize its benefits, so the use of game methods can be considered for this kind of problem. The dynamic game method is given in this paper. The main players in CHP are power generation companies, retailers and energy service providers, which influence the game by improving the method of system operation, investment schemes and other strategies, ultimately achieving the goal of maximizing the benefits. In [48] the energy management issues of energy networks composed of CHP and PV power generation. Based on Stackelberg game theory, the players in the game model are the operators of the microgrid and the PV energy suppliers, with the operators of the micro-grid being the leaders, and the PV energy suppliers and other players being the followers. The strategy set in the game is the price strategy in the micro-grid, and the PV output strategy is the optional strategy set among the players. The payoff function of the game process is to obtain the optimal price, optimize the load characteristics, and reduce the operating cost and maximum output characteristic in the micro-grid.
In addition to the game scenarios in the two conventional forms of coupling, there are also game scenarios for the interaction between other energy sources. In [49] discusses the distribution of benefits between multiple energy sources in the PV cluster and proposes multiparty energy management approaches. In the game scenario, non-cooperative game theories are selected in which energy companies make decisions on internal pricing or consumers establish demandresponsive strategies. The payoff function of the producer is to improve the overall benefit, and the payoff function of the consumer is to obtain reasonable prices and use energy efficiently. The system for circulating gas vapor based on gas turbines and steam turbines in [50] established three optimization models. For these, the strategy set for operations is mainly formed by the energy consumption factor, operating cost, investment benefit and so on. The aim is to reduce the operating cost and optimize the peak period performance. Based on these three elements, a game scenario can be formed.

2) DISTRIBUTION NETWORK
As seen in Figure 1, the distribution network contains links to energy sales in the delivery network and energy market. The distribution network is related to the economy and plays an important role in IESs. The distribution network should not only coordinate with the supply side but also trade and distribute energy with the demand side. Therefore, the distribution network very much needs the application of game theory. The following is an overview of the game scenarios that appear in the distribution network.
First, entities in the energy market compete for high market share and benefits. In [51] the optimal bidding strategy of electric vehicle aggregator in auxiliary service market is proposed, and the scene representation is carried out by Monte Carlo simulation method. The effectiveness of this method and the influence of bidding strategy on power market are illustrated. In [52], all the rights of several regional distribution companies in the natural gas market were considered to compete for more benefits. The players in the main game scenarios in the literature are producers, suppliers, retail companies, and large consumers. The government agencies act as regulators, and under the specific market competition mechanism, each player aims, through competition or cooperation, to improve his or her performance. There are two ways to play a cooperative game in this game scenario. On the one hand, by opening the cooperative valve to share the gas of the cooperative network, the cost can be effectively merged to achieve cost savings. On the other hand, to find the best route in the transmission and distribution network, the ultimate payoff function aims to reduce the cost of suppliers and meet the needs of consumers to the maximum extent. The interaction of multiple subjects in the urban gas market is similarly proposed in [53] The players in the game are basically consistent with the previous literature, including the government, the gas distributor and the end user. The functions and purposes of each subject are different. The government regulates the market, the allocator ensures the security and stability of the system, and users have different needs based on their user group (industrial, commercial, residential). The strategy is to find the best time pricing to reduce operating costs and improve social welfare.
In addition to the natural gas network, there are game scenarios for the grid or power generation companies. The resulting game scenario of large renewable energy generation for power system integration in [54] introduced an innovative space-time vector and interactive game mode for the space-time vector of the energy trading network. The direct transaction of all the energy between the micro-grid and the micro-grid is included in this game. The strategy used in this game scenario aims to ensure the operation quality in the upper distribution energy market of the system and simulate the interaction of multi-energy with the micro-grid in the lower layer of the institution. The payoff function in the game process maximizes the profit of each micro-grid.
The behavior of power generation companies in the energy market is investigated in [55]. Of the game scenario players, power generation enterprises are the leaders, and other service providers are followers. A framework of the Stackelberg game is established. The strategy is for market managers to clean up the market and set a dispatch for the transmission system and energy supply prices. The ultimate benefit of the game process is to improve the social welfare of the system and encourage market-conscience competition. In [56], accurate data cannot be obtained without real-time measurement due to fluctuations in system operation, changes in load demand and changes in generator sets. Decision-making theory based on the information gap is proposed and applied to the game scenarios. The ultimate goal of the game is to help decision makers choose the appropriate repair strategy to increase the effectiveness of the recovery process.
In addition to single energy systems, there are more game scenarios between forms of energy in the distribution network, for example, conventional natural gas and power system coupling, the vigorously development of the coupling between renewable energy and traditional energy, and so on. These represent the game scenario. In [57], energy retailers and users need a reasonable operation strategy to maximize profits. Then a game model of distributed energy storage operation with energy storage cost, market price and market competition degree is proposed, which improves the forecasting ability of retailers and provides users with more choices. Reference [58] considered that with the development of cities, the interaction between power, water and gas networks in cities becomes increasingly intense. Therefore, the power, water and gas distribution networks are used as players who collect all the effective information in the network as a strat-egy and ultimately achieve the optimal configuration between systems. These three factors are considered to design the game. Beyond conventional game methods, many scholars now study new game methods. In [59] the operation mode cost and carbon emission tax minimization of CCHP triple supply are taken as the comprehensive optimization purpose, and the strategy of co-optimizing the capacity of the CCHP system with the heat network as the regional unit is established to play the game.
Finally, the distribution network, source storage and other different energy side game scenarios are discussed. Reference [60], constructed the respective decision-making schemes for different investors and players in the source-networkload-storage mode in grid-connected integrated energy. The game is divided into three different scenarios. The first is the renewable energy service provider, micro-grid system PV and power storage equipment. Second, the micro-grid system energy service provider, CCHP, gas-fired heat pump, and electric refrigerator form a game scenario. The third scenario is the EV user and the micro-grid charge and discharge equipment.

3) DEMAND SIDE
In the traditional energy structure, demand-side users often have no chance to communicate directly with energy companies and service providers and can only passively accept their policies. However, with the reform of the energy structure, the energy market is gradually opening to the demand side, and distributed energy and industrial parks have been added, so that now the demand side has game scenarios both internally and with the supply side and the distribution network. Therefore, the game scenario on the demand side is summarized here.
On the one hand, the game scenario focuses on the demand co-supply side. In [61], modern energy management in power and natural gas is studied in the context of the demand side. The interaction between energy hubs on the demand side of integrated energy is transformed into a game scenario, and the game is played based on non-cooperative game theory. Users of load and energy hubs are players playing games by choosing strategies that operate differently. The ultimate goal of reaching users is not to change consumption patterns at peak periods to reduce electricity consumption or to shift demand to nonpeak periods to respond to electricity prices at peak times, so that users will reduce the cost. Instead, [62] aims to maximize user cost reduction and maximize the profits of power and gas suppliers. Natural gas companies, power companies and consumers are established as players; the sequential number potential game serves as the theoretical basis, and through the simulation of the actual scenario in playing the game, players choose the appropriate strategy to benefit the parties. The final result shows that the peak load demand for electricity and gas is reduced, which maximizes the benefits. In an environment characterized by the vigorous development of renewable energy, the relevant players in renewable energy on the demand side play a game. Reference [63], based on suppliers of fossil fuels and renewable energy, proposes an effective scheme to reduce user costs that is turned into a game scenario involving energy companies and users and taking into account the uncertainties of renewables; the existence of Nash equilibrium is proven and that it is unique. In this paper, a strategy is proposed to combine the choice of load dispatching and energy companies. In [64] suggested that there is a potential game in the energy Internet to meet the needs of different users. The players in the game are thus all types of users, with energy managers on the demand side. A fully distributed optimization strategy considering the permeability and demand response of renewable energy is proposed. The advantage of this strategy is that it is fully covered and dynamically open in the game process; therefore, individual optimization drives the overall optimization and ensures that each player in the game has a benefit. In [65], the problem of PV system penetration into the client base is specifically studied, and non-cooperative game theory is applied. The players in the established game scenario are consumers and PV systems. In the course of the game, factors such as installed capacity and the cell capacity of different PV systems are analyzed, and different demand response capabilities are studied for the demand side. Under the influence of these factors, the Nash equilibrium is found and proven unique. Finally, it is concluded in the game that highly reactive consumers can accept large-capacity PV power generation, which can improve the economy of the PV system with fewer batteries and meet the demand for electricity with the minimum cost, thus obtaining more benefits. In [66], a two-stage Stackelberg game is conducted for the optimization of storage capacity and PV power generation in microgrids.
On the other hand, the demand-side game with a distribution network and demand-side users is very closely related, wherein transactions between users and service providers are generated. There are also many game scenarios in this area. The competition theory model of the demand response aggregator is proposed in [67] To form a game scenario, an incomplete information game is used as the theoretical basis. The players in the game are aggregators of demand responses, and the game strategy is to obtain information from other aggregators. Studies also consider the case of two non-cooperative games with different market conditions: one is competition without any conditions, and the other is a Stackelberg game limited by transaction price and size. In [68], a two-layer game model between the energy retailer and the consumer is established for the transaction between the focal energy retailer and the consumer. The first layer first provides the retail price for the energy, and the consumer responds to the price provided and minimizes energy expenditure in the second layer. With this game approach, it is feasible to increase the profits of energy retailers and reduce the user's payment costs in energy systems, and it significantly reduces the power peak. In the course of the game, we need to ensure that some of the rules and constraints in the game scenario are fair to the players. Reference [69], based on a fair environment, studies how to realize the energy dispatching problem in the interaction between energy merchants and consumers, and a game model framework for the demand response of consumer preference is proposed. Players are resident users and energy sellers, and they use relevant information such as peak volume and cost effectiveness as a strategy in the game, and any consumer must gain profits through established strategies.
A game scenario proposed in [70] has game players composed of power generation companies, retailers, various types of users, etc. to play a game focused on demand-side regulatory angle issues. The game strategy is used to prove the uniqueness of the Nash equilibrium and the strategy of simulating price and pricing is used to determine whether the system is positive or not. Finally, the game strategy is shown to be effective.

IV. APPLICATION OF GAME THEORY IN PLANNING AND DISPATCHING OF INTEGRATED ENERGY SYSTEM
There are many energy linkages and interactions in integrated energy systems. An effective interactive framework can greatly enhance resilience to natural disasters [71]. Then, in the operation of the system, it is necessary to plan and dispatch all kinds of energy, so that the future energy system technology will be more mature, the structure will be clearer, and the resilience of the system will be improved [72]. In the study of integrated energy system planning and dispatching, it also promotes the development of game theory in integrated energy system. Game theory can solve the problems in planning and dispatching. The following section provides a specific overview of game theory in integrated energy systems planning and dispatching research.

A. APPLICATION OF GAME THEORY IN INTEGRATED ENERGY PLANNING
Due to the current shortage of fossil fuels and the increasing trend of renewable energy, the earth-shaking changes are taking place in the integrated energy system, so how to plan the energy in the system is a problem worthy of consideration, and many studies on this aspect have emerged at present. Therefore, some summary of this research is made.
In [73], a planning model and an optimized output strategy are proposed for the combined system The combined system integrates the characteristics of wind power generation and the technological breakthrough of energy storage equipment, which makes it possible for large energy storage equipment to enter the grid sequence. The artificial fish swarm algorithm is used to compare the cost of each segmented system and the transmission scheme and benefits. Finally, it is proved that it is feasible to add energy storage equipment to power grid in a certain way, and it can reduce transmission investment and improve the economy. In [74], a stochastic decentralized model for coordinating natural gas and power networks is presented, incorporating wind power production, interest rates, load growth and control policies that limit the capacity of new renewable energy facilities. Finally explore the pipeline architecture and gas on the power network expansion VOLUME 8, 2020 planning. Reference [75], the prospect is presented in the energy Internet. Although the energy Internet has made a lot of progress, it still needs further study in the planning of integrated energy system. The problems existing in the integrated energy system and the network physical system are put forward, and the source-net-load integrated planning and transient process are detailed, provide some methods for future planning research. Reference [76], energy shortages and rising energy prices have led to energy efficiency problems at many factories. In order to be more competitive and profitable in the market, energy efficiency should be an important factor in early energy planning. Based on the above characteristics, a new energy planning method is proposed, which introduces energy efficiency as a key criterion into energy planning, and also considers the transportation cost and labor cost. In [77], an ontology-based multi-intelligent energy management system is proposed to monitor and optimize various links of the system for the complexity of various renewable energy and residential systems with controllable loads. The optimization process is divided into two parts, namely, the coordination of distributed efforts and the management of demand response. And use some strategies to optimize the relationship and cooperation among them. Make its system more applicable and effective. Reference [78], based on the current low-carbon environment, which requires increasing the utilization of renewable energy in power system, a theoretical framework is established to study the optimal planning of power system. The study is divided into two phases, the first of which is the optimal investment in renewable energy, and the second is the micro-grid to coordinate the supply and demand of energy to reduce operating costs. It is concluded that the solar and wind energy is the best in the renewable energy. In [79], the same is true for the lowcarbon economy, but it is mainly about energy transmission. Large-scale energy transmission and intermittent renewable energy bring uncertainty about energy delivery. Based on this situation, an energy transmission planning scheme considering the uncertainty of wind and photovoltaic is proposed to reduce the overall social cost. The advantages of the scheme are illustrated by numerical analysis.
So how to make good use of game theory in energy planning, and what role does game theory play in energy planning? This subparagraph explains. In [80], by coupling long-term and short-term energy planning with game theory, long-term energy planning investments include short-term energy management, which makes the goals of stakeholders clearer. Moreover, two game frameworks of energy planning can model different and even conflicting models, so as to solve the planning problem under multi-objective and multi-subject. Reference [81], a game model with minimum -maximum uncertainty is proposed, which is to find the minimum reserve capacity and satisfy the maximum load demand. And the algorithm is used to solve the minimummaximum game. The traditional expectation method is compared with the Monte Carlo method, and the effectiveness is proved. In [82], for the intermittent problem of clean energy, the framework model of multi-objective bidding strategy for wind-thermal photovoltaic system is proposed. The model considers two objective functions, which are related to profit maximization and thermal cell heating minimization, respectively. The proposed strategy is also developed under various modes of energy dispatching or combination. The strategy not only increases producers' expected profits but also reduces pollution. Reference [83], a new method for transmission and power generation expansion planning using multi-area energy system is proposed. In order to adapt to the market competition and make the optimal investment plan, the traditional and improved game methods are used to solve the problems of investment and operation planning, so as to promote the competition among power grids. By using fuzzy satisfy method to solve. In [84], gas-fired power plants are heavily dependent on natural gas and power grids, and the planning of gas and power systems is often not integrated. Therefore, the multi-attribute decision-making method is used to carry out common planning. Although the methods used are in conflict with each other in decision-making, the best compromise can be chosen based on different attributes, so as to find the best operation plan.

B. APPLICATION OF GAME THEORY IN INTEGRATED ENERGY SYSTEMS DISPATCH
In the integrated energy system, it is necessary not only to plan the various parts of the system, but also to dispatch the various energy sources of the system. The research on dispatch and how to apply game theory in integrated energy system are summarized below.
In [85], it breaks the traditional mode of single energy system and establishes the connection between power and thermal network. Therefore, it develops the analysis method of power and thermal network as a whole, which makes the form of dispatching optimal and integrates the calculation method of thermal power flow. Reference [86], aiming at the problem of lack of flexibility of regional heating network, the dispatching of cogeneration combined heat and power dispatch is proposed to coordinate the operation of power system and district heating system. The simulation shows that the operating economy and the utilization of wind power have potential benefits. Reference [87], the energy service is optimized by the strategy tool, which enables the user to allocate the required energy, and then the available distributed energy are dispatch to maximize the benefits. The particle swarm optimization algorithm is selected to solve the optimization problem of response. Reference [88], a method of optimizing different types of renewable distributed generation in distribution system is studied, which can minimize the loss of energy, and the technology has been applied in distribution systems with different scenarios, and the results show that energy loss is significantly reduced.
Also in the energy dispatching problem, the introduction of game theory can solve the related problems. In [89], distributed economic dispatching problem in smart grid is studied. A game method based on equilibrium state is proposed for this problem. The Nash equilibrium of stationary state is combined with the optimal economic dispatching. This method can deal with complex equations and coupling constraints of inequalities to ensure the global optimal effect. Reference [90], in order to make large-scale renewable energy use and electricity consumption two-way interaction, the optimal dispatching model of multi-interest player game are established. The optimal equilibrium strategy can improve the enthusiasm and fairness of player coordination and optimization, which can promote the consumption of renewable energy and improve the profit of consumers. In [91], in the face of the rapid growth of electricity demand, if the traditional way to expand the power generation system will cost too much and bring negative environmental impact. Demand in order to solve this contradiction, put forward the method of distributed generators in the field of economy and sustainable operation, but this method still has a shortage, then put forward a distributed generation dispatching strategy, to effectively coordinate a large number of distributed generators, to meet the demand side, and through the analysis of the dispatching policy is extensible, also is the optimal way of costs and benefits [92]. With the increasingly open power market, many power generation enterprises have some initiative in dispatching, so the competition and some benefits are particularly important. Therefore, a non-cooperative game theory is designed for wind power, water storage and thermal power, and the decision space is constructed through the characteristics of three-party dispatching output. Based on the game strategy, the dispatching of power system is more optimized, and the effect of joint optimization in different seasons is also very obvious.

V. GAME THEORY MODELS FOR INTEGRATED ENERGY SYSTEMS
Now in the comprehensive energy system, the theory of game theory has started to be applied in it, and various algorithms are needed to verify whether the game model is effective and feasible. In the current study most of the use of cooperative game, non-cooperative game and Stackelberg game model, so for each different game types and objects when they choose a suitable algorithm is used to, so can be efficient and accurate to obtain the desired results, so the following according to game theory application in the integrated energy system by using the algorithm of summary.

A. APPLICATION OF COOPERATIVE GAME ALGORITHM IN INTEGRATED ENERGY SYSTEM
This section summarizes the algorithm of cooperative game theory in integrated energy system.
When the cooperative game theory is introduced into the integrated energy system, more algorithms are used to Shapley the value allocation algorithm, Shapley the benefit of the players in the distribution process is proportional to the contribution. The advantage of the Shapley value allocation method can be reflected in the cooperative game, and the Shapley value allocation method can fairly and reasonably distribute the benefit and cost according to the marginal contribution of the alliance members in its cooperation. In [94], the cooperative game model is established for each main body in the park. The main players are power grid companies, power generation companies and power sales companies. On the basis of cooperative game, the benefits of the alliance are not less than the sum of the benefits of each player before the cooperation. Then the benefit that should cooperative game produces is called cooperative surplus. The specific cooperation surplus should correspond to the specific case analysis. Shapley value method can reasonably allocate the cooperation surplus to each player. Reference [97], for all kinds of service providers in the integrated energy system, in the spot market, the size of the integrated energy quotient varies, and there will be power deviation in the actual operation, then this part of the energy imbalance will be purchased or sold in the spot market, in which the energy market members are free to bid, free pricing, so the price in the market will fluctuate irregularly, then will face a great risk and will cause losses. A cooperative game is needed to regulate an alliance. For alliances to be formed there is a need not only to consider the issue of returns, but also to consider risks. So each player has to take his own risk, Shapley value can be used for allocation. If there are N members, A set of members is S, the number of elements S is N!. Then assume the Shapley value of the I member as follows: In the formula, ϕ i is Shapley value of i integrated energy system service provider, s j is the j element in the set S, δ i (s j ) is a change in the risk of joining the alliance in s j order. As a result, the Shapley value can be used to allocate the risk to player. Secondly is distributed alliance construction algorithm. In the cooperative game alliance, according to their own needs, cooperate with other members of the alliance to achieve the goal of win-win cooperation. Then distributed alliance construction algorithm is very suitable for solving cooperative game model. Reference [98], the alliance construction algorithm adopts two distributed rules, one is the merge rule, and the other is the split rule. In the cooperation rule, Instead, the split rule is to divide alliances into new ones. In the actual situation, it is necessary to choose the rules according to the actual situation, if the alliance is small and many, it needs to be merged, and if the larger alliance needs to choose the split rule and divide it into new alliances. There are several different schemes in splitting and merging, so it is necessary to choose the appropriate method according to the need and benefit to make the optimal scheme. VOLUME 8, 2020 In addition to the above two algorithms, the conventional iterative algorithm will also be used in the cooperative game model. Reference [99], a three-party cooperative game is used to achieve Nash equilibrium in the game, and the cooperative game can be maximized. How to make each player choose the optimal strategy and achieve the best benefit requires iterative search algorithm to achieve Nash equilibrium. By using the iterative algorithm to solve the cooperative game model, we first input the original data and parameters according to the specific situation, then set up the cooperative game model, select the initial value of the equilibrium point, then each alliance independently optimizes the decision, continuously selects the optimal strategy, simultaneously carries on the information sharing, finally judges whether to find the Nash equilibrium point, if achieves the equilibrium solution output result, if does not reach again returns the optimization. Therefore, iterative algorithm can be used to solve the optimal solution strategy.

B. APPLICATION OF NON-COOPERATIVE GAME ALGORITHM IN INTEGRATED ENERGY SYSTEM
This section summarizes the algorithm for solving the noncooperative game theory in integrated energy.
In the integrated energy system, the theory of noncooperative game is less applied, because player in the noncooperative game will only care about whether to maximize their own interests, and in the relationship of competition, the parties are a rational state. So the parties are in full confrontation. so it is not commonly used in integrated energy systems.
Nash equilibrium algorithm is often used in noncooperative game models, because in the competitive environment, all players need to find an equilibrium point to seek balance, and the Shapley value allocation mentioned in the previous section is not applicable in the non-cooperative game model. In [101], according to certain market rules and Nash equilibrium tripartite game theory to do the basic criterion, select the players in the game, calculate each player cost, transaction cost and transaction income. The equilibrium point is deduced according to a certain utility function. The advantage of Nash equilibrium algorithm lies in that it can meet the needs of all parties under the condition of competition and is the optimal method.
The other is the two-objective optimization algorithm, which often exists the competition between the two in the non-cooperative game, and establishes the utility function of the player, which includes the benefits, costs and some key elements. Reference [102], the fuzzy two-objective algorithm is used to optimize. First, the optimization calculation is carried out with objective 1, and the optimization value is obtained at this time, and the value of objective 2 is calculated at this time. In the same way, the optimization calculation is carried out with objective 2 to get the optimization value at this time, and the value of objective 1 at this time is calculated. After that, the optimal attribute of the objective function value is blurred and the mapping is established. Finally, the objective function of fuzzy double objective can be obtained by linear weighting of two objective functions.

C. APPLICATION OF STACKELBERG GAME ALGORITHM IN INTEGRATED ENERGY SYSTEM
This section summarizes the algorithm for solving the Stackelberg game theory in integrated energy.
Both the cooperative game and the non-cooperative game discussed above are based on the equal status of the players and the same amount of information obtained. But in the actual project, there will be a relationship between superior and subordinate, active access to information and passive access to information. Then there is a master-slave relationship in the authority of decision-making, and the leader can take the lead in making strategies and making decisions in the game. And the follower can only make decisions in the information released by the leader, with certain constraints.  Hence, the algorithm in Stackelberg game is introduced in detail.
In the process of game, players are often unwilling to disclose their goals and information, so the use of distributed algorithm can avoid this problem. In [104], when the players in the game need a control center to coordinate to solve the equilibrium value, the control center does not need to know the information of both sides of the game. The control center needs to initialize the parameters and communicate them to the players. The leader optimizes on this basis and then informs the control center of the optimized output result and transmits it to the follower. Then the follower optimizes its own optimal strategy based on this result. After that, summarize in the control center and iterate repeatedly until the optimal solution. Reference [106], the two-layer distributed algorithm is used to solve the model, and the revenue function of users and operators is stratified and optimized. The operator sends the compensation electricity price to the user through its own situation, each user reports each data to the operator on the basis of ensuring its own income maximization, the operator then redraws the compensation electricity price according to these data, then sends to the user, so repeatedly, until reaches the game equilibrium.
The inverse order induction method is a common method in solving mathematical problems, and can also be used in solving the unique solution of the model. Reference [108], in view of the transaction between the operator and the user in the PV user group, the operator, as the leader, should make the electricity price under the premise of ensuring its own interests, and the user as the follower ensures the maximum benefit of its own electricity consumption according to the published electricity price, but the user's demand response will affect the operator's income, so it is necessary to find the equilibrium solution and the unique solution of the game. In order to prove that the equilibrium solution of the game is unique, it can be directly proved that the electricity price set by the first operator is unique, so the inverse order induction method is adopted. When the user adjusts the power consumption, the power purchase user has the upper limit standard, and the power sale user has the lower limit standard. After the optimal electricity consumption is uniquely determined, a benefit parameter is formed, and the relevant value is substituted into the optimization to determine the optimal price of electricity. After the optimal price of electricity is determined and unique, it can be proved that the equilibrium solution of the game is also the unique solution. In [109], a Stackelberg game model between distribution network and demand response is established to solve the problem that the traditional price demand response may increase the user's cost. In order to ensure that the electricity fee paid by the participating users is not higher than the electricity fee paid by the non-participating users and improve the enthusiasm of the participating users, the dynamic response electricity price is limited. In the model, the active distribution network is the leader and the demand response is the follower. The leader publishes the current dynamic electricity price, the follower formulates the optimal response strategy according to his utility function, and the leader determines his own strategy according to the follower's strategy and his utility function, until neither party changes the strategy. According to the operation process of Stackelberg game, the backward induction method is used to calculate the optimal strategy of the demand side according to the utility function of the demand side, and the optimal strategy of the distribution network according to the optimal strategy of the demand side and the utility function of the distribution network. Using this method to solve the equilibrium solution of the model is explicit and purposeful.
In solving the Stackelberg game model, the particle swarm optimization algorithm will also appear. The algorithm of particle swarm optimization has fast search speed, high efficiency, simple algorithm, and is suitable for the processing of specific values. Reference [111] a Stackelberg game model is proposed for multiple subjects, with energy hub operators as leaders and users and energy storage operators as followers to establish a leader multi-follower model. Players set up optimization models according to their own interests, the specific models will not be repeated. The specific flow of the algorithm can be summed up as reading the relevant system parameters first, then presupposing the constraints, and then aiming at K energy hub, according to the actual energy needs and their own interests, to calculate the energy purchase constraints of each period, and so on to calculate the k+1 energy hub, and finally get the optimization results. Particle swarm optimization is easy to fall into local optimum, so in the multi-agent Stackelberg game model, hierarchical processing is carried out.
NSGA-II is a kind of genetic algorithm, an improved algorithm based on genetic algorithm. NSGA-II applied in Stackelberg game model for optimization. Reference [112] it is also a leader-multi-follower Stackelberg game model, which is a leader-three-follower Stackelberg game model with gas power plant as the follower of the leader-threefollower Stackelberg game model. The specific operation flow of the model is consistent with that described above. How to make use NSGA-II for optimization, the following is a brief introduction. First input the parameters and data of the original load, establish a leader-three-follower game model, give Nash the initial value of the equilibrium solution, below into the follower's optimization module, first wind, water and electricity, gas and electricity independent decision, if the optimal solution can be found output, can't find return. Output results into the leader's module for decision-making, after the grid decision, if know the Stackelberg equilibrium solution output, can't find the return. Output results after calculating the equalization, output the final results.

VI. THE FUTURE PROSPECT FOR THE APPLICATION OF GAME THEORY IN INTEGRATED ENERGY SYSTEMS A. CONSIDER THE NEEDS OF DIFFERENT TYPES OF USERS ON THE DEMAND SIDE
In IESs, the demand side is often in an inferior position in the game process. In reality, the demand side will trade with energy companies, energy sales companies and other subjects. Therefore, game theory is widely used in side searches. However, the demand side includes various types of users, such as residential users, commercial users, industrial parks and other high-energy users. Thus, when we apply game theory, we need to consider the needs of users.
First, [113] the proportion of residential users on the demand side is quite large, but their energy demand is not large, and the economy is a key factor for residential users. When establishing the game model, we should consider whether the price is within the range of residential users, and the ability of the residents in different regions is also different. At the same time, we should also consider the electricity consumption habits of residents in a certain area and how much electricity is consumed. When an electricity price stage is established, if the normal power consumption is exceeded, the electricity price can be increased, and some users can be restricted from wasting energy.
Second, commercial users are characterized by concentrated load use; most of them depend on equipment with large energy consumption, and so the power load will continue at a relatively high stage. Then, business users need to have a game model that adapts to them. For example, energy sales companies and commercial users need to formulate different policies with residents in terms of price, energy and environmental protection and other links to develop appropriate measures for local conditions. To achieve satisfactory results for both parties, each piece must reach an equilibrium point.
Finally, industrial parks are themselves production units but also have many energy users. Compared with the previous two types, this group of users has more initiative in the process of the game and can negotiate the transaction directly with the energy company. This type of user can interact with the energy supply unit or the energy sales unit through the cooperative game to maximize the interests of all parties. It should be noted that when users trade with energy companies, the information is often classified as trade secrets, so a game under incomplete information can also be used for this group of users.

B. EXPLORING DIFFERENT GAME MODELS AND INTRODUCING INTEGRATED ENERGY SYSTEMS
This paper mainly refers to the application of cooperative games, non-cooperative games and Stackelberg games to IESs. In the current research, these three game models mainly account for the vast majority of studies. However, there are other models in game theory, such as repeated games and evolutionary games, and these other game models have been introduced in other fields to solve problems [114]. In future research, we can consider introducing other game models to solve practical problems in the integrated energy system. Reference [115] is based on a repeated game to optimize the integrated energy system. If a group is formed in a certain area of the integrated energy system, and the players are bounded rational, we can consider introducing an evolutionary game. Therefore, to solve practical problems, there is a wider range of options. Therefore, future research should explore different game models to solve related problems.

C. CONSIDER THE WIDE APPLICATION OF MIXED GAME IN INTEGRATED ENERGY SYSTEMS
Countries are paying increasing attention to the development of IESs, and an increasing number of elements are involved in IESs. A single game model cannot solve the problem well. For integrated energy, the source-network-loadstorage mode will often produce transactions, but a certain stage of the game will also occur internally; in this case, we should consider introducing the mixed game. Transactions between different sides can be introduced in one game model, and internal transaction competition can be introduced into another game model. In [116], a hierarchical game model is proposed. To solve different levels of problems more effectively, mixed games are also introduced into other fields, but in the current integrated energy system, research using hybrid games to solve related problems is still limited.

D. CONSIDERING THE IMPACT OF CLIMATE FACTORS ON INTEGRATED ENERGY SYSTEMS
Because the whole world is facing energy problems, the whole world is studying integrated energy systems, but in the process of research, energy needs to be studied in light of the local climate. Reference [117] the characteristics of tropical climates have been studied and modeled. Combining these characteristics with the actual local situation to carry out research, [118], [119] unify the local situation to establish a corresponding integrated energy system [120] the efficiency of regional heating and cooling is reviewed. Therefore, game theory is introduced into urban multi-energy systems. Climate conditions should also be taken into account in the system.

VII. CONCLUSION
Against a background wherein the state advocates energy saving, emission reduction and the complementary energy advantages, there will be increasing research on IESs in the future, and more problems will be encountered: how to realize the complementary advantages between forms of energy, how to maximize the energy utilization rate in the distribution network, how to maximize the benefits when trading with between the demand side, the supply side, and the network, and so on. In view of this concern, this paper summarizes the current research status of game theory applied to IESs according to the game scenario, game model and algorithm. The game scenario is mainly summarized considering three perspectives-the energy supply side, the distribution network and the demand side, and planning and dispatch problems-which fully cover all the elements of the integrated energy system and some competitive environments. The game theory model is summarized considering the cooperative game theory model, non-cooperative game theory model and Stackelberg game theory model. These three game models are the most critical models in game theory and the three most widely applied game models at present. The current algorithms for solving game models are also summarized. Finally, future challenge on IESs is proposed in the hopes of providing some references for research in this field.