Variational Inequality and Distributed Learning for a Bidding Game in Electricity Supply Markets

This study uses a game-theoretic analysis of bid-based electricity supply market equilibrium. Electricity supply markets are modeled as strategic interactions of bidders that supply electric power to the market and the bidders’ pure strategies are the cost function parameters of power generation. We demonstrate that the resultant bidding game is a convex game and has a unique pure-strategy Nash equilibrium (PNE) when the bid-cost functions are parameterized by marginal costs of power generation. The PNE of the power-supply bidding game is reformulated in terms of a variational inequality and as a ﬁxed-point of a recursive mapping. We propose two distributed learning algorithms and their variations with convergence analysis to compute a PNE. Three types of measures are proposed and analyzed for quantiﬁcation of inefﬁciency due to falsiﬁed bidding actions corresponding to the marginal cost function parameters of supply-market participative generators. A numerical case study with a 26-bus power network is presented to illustrate and demonstrate our results.


I. INTRODUCTION
The smart grid infrastructure, including smart sensors and meters, and communication technology has led to many fundamental problems of power systems research, such as optimal power flow, unit commitment, and economic dispatch being revisited. In smart energy system infrastructures with information and communication networks, different independent units take the control and operational responsibility in different areas of the system [1], [2], thus optimal operation and control of electricity sectors becomes distributed, independent, and deregulated [3], [4]. A particular class of optimization problems known as economic dispatch with integration of distributed energy resources and storage systems in a smart grid infrastructure is one of the major challenges for large-scale complex power networks. It is necessary to meet the total power demand by allocating demand among many independently operated distributed generators in an efficient way with guaranteed quality of service and safety [5].
The associate editor coordinating the review of this manuscript and approving it for publication was Pierluigi Siano . Game theory has been a tool for formal analysis of economic behavior and a conceptual abstraction framework incorporated with advanced mathematical tools for studying strategic interactions of rational decision-makers. For this reason, game theory has been expected and considered to be important and heavily used in designing the future electricity markets of smart grids [6], [7]. Recently, modeling electricity markets has attracted significant research efforts and debates. The introduction of economic competition in the electricity industry has been considered to be a tool used by several countries to improve the overall efficiency of their electricity generation, transmission, and distribution, which is in turn, expected to benefit consumers, as well as producers [8], [9]. All types of competition (Cournot, Bertrand, supply function) can be utilized and have their own advantages and disadvantages for electricity markets [10]. One of the most promising approaches to modeling electricity markets is supply function competition and the associated game-theoretic analysis. Supply function competition was first examined in a seminal paper [11]. There have been many applications of the supply function equilibrium to the wholesale electricity markets, for example, [12]- [16]. VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ Motivated by the application of supply function equilibria (SFE) in the wholesale market for the supply side, we consider a retail electricity market model to match the power demand and a supply-bidding game in which private parameters of power-generation cost functions are strategically reported to the market operator. The market operator determines the electricity price through economic dispatch based on the reported parameterized supply functions and the bidders corresponding to power-generation units are price-takers. The bidders are also allocation-takers in the sense that the power-generation units fulfill the generating contracts and supply the allocated amounts of power that are determined by the electricity supply-market operator. In economic dispatch, it is assumed that all generators report their true cost (function) parameters. It is, of course, possible that a generator can take advantage of not reporting the true cost parameters but instead providing false reports. Related work is the problem of parameterized supply function equilibrium [12]- [14] and the literature of parameterized linear supply function equilibrium models in electricity market is reviewed in [17].
For a start, this paper explicitly and rigorously analyzes the supply-bidding game with one-and two-player cases. Existence of a pure-strategy Nash equilibria (PNE) is verified and the closed-form solutions of pure-strategy Nash equilibria are presented. To demonstrate the existence of a PNE, the results of a two-generator problem are expanded for the general N -generator problems. We also provide a variational inequality reformulation and the associated fixed-point representation for a PNE of the proposed supply-bidding game. Based on the variational inequality and the fixed-point condition of a PNE, two distributed algorithms based on iterative methods with fixed-point abstraction of equilibria are presented for computing a PNE. We define and analyze three different measures of inefficiency due to strategically selfish generators reporting false marginal cost function parameters. For illustrating and demonstrating our methods of computing a PNE, a numerical case study with 6 generators (or power-supply bidders) is presented.
The rest of this paper is organized as follows: Section II presents preliminary backgrounds of economic dispatch with emphasis on parametric dependence of optimal power generation allocation and shadow price. The problem formulation of a strategic bidding game in an electricity supply market is also presented in section II. In section III, rigorous mathematical analysis of one and two player games are presented. Closed-form representations of the associated pure-strategy Nash equilibria are derived. Section IV extends the results of section III for a general N player game and shows that it is a convex game. Variational inequality (VI) reformulation of a PNE is presented in section V and distributed learning algorithms exploiting the VI are proposed in section VI. In section VII, we discuss inefficiency of game-theoretic solutions with a numerical case study. Section VIII concludes the paper with a perspective on future work.

II. PRELIMINARIES AND PROBLEM FORMULATION A. ECONOMIC DISPATCH AND MARKET PRICE
Consider the simplest economic dispatch problem that is defined as where the cost functions of power generation are given by where a i [$/MW 2 ] and b i [$/MW ] are private parameters of each player(generator) i that are assumed to be controlled and reported by the strategic supply-market participants. That is, bidding the parameters a i and b i can be strategically defined by the power supply-market participants. Assumption 1: The parameters a i > 0 cannot be set by players, but they have fixed valued and are controlled by a market operator. They are assumed to be common knowledge to all the supply-market participants, which means that the only bidding parameter is b i > 0 for each generator i.
For the case with quadratic cost functions in (2), there are necessary conditions for optimality which are given as the following linear system: and Note that if a i = 0 for all i so that the system (3) is linearly independent, then we have a unique closed-form solution for (3): (5) does not necessarily satisfy the capacity constraints (4). We assume that the optimal economic dispatch profiles in (5) are strictly feasible for the inequalities (4). This is not just for mathematical convenience, but for two-sided options of each generation unit to report true or false parameter values of supply function so that the computed economic dispatch profiles with reported parameter values are feasible.

B. ELECTRICITY SUPPLY-BIDDING GAME
Throughout this paper, we consider the game = (N , where the cost function is defined as for players (or bidders) i = 1, 2, · · · , N .

III. ONE AND TWO PLAYER SUPPLY-BIDDING GAMES A. ONE-GENERATOR PROBLEM
We consider the case in which all the generators except for the generator i report their true values of {b j > 0 : j = i}. In particular, the generator reports the marginal cost with the parameter , the aggregator (or market operator) determines the price as where µ * is the price when all participating power generators report the true values of b j 's. The power generation profile corresponding to the new market priceμ(x) is given bỹ for j = i. We define the payoff function for the generator i by We want to solve the optimization Lemma 1: The optimization problem (11) is strictly convex and has the unique optimum with p * i defined in (5). Proof: Using the chain-rule, the first-order derivative of and the second-order derivative is computed as where the positivity follows the inequality This positivity of the second-order derivative implies that J i is strictly convex. Since J i is strictly convex and the constraint in (11) is compact and convex, a unique minimum exists and is given by The one-generator problem is not a game but a one-player decision-making process. For a game formulation, we consider the case in which all the generators except for two generators indexed as 1 and 2 report their true values of {b j > 0 : j = 1, 2}. Generators 1 and 2 report the parameter values b 1 + x 1 and b 2 + x 2 , respectively. For given (reported) the aggregator (or market operator) determines the price as where µ * is the price when all participating power generators report the true values of b j 's. The power generation profile corresponding to the new market priceμ(x 1 , x 2 ) is given bỹ andp We define the payoff functions for generators 1 and 2 as We want to solve the following two constrained optimization problemsx VOLUME 8, 2020 According to the definition of Nash equilibrium, ifx 1 (x 2 ) = x 1 andx 2 (x 1 ) = x 2 then the profile (x 1 , x 2 ) is a pure-strategy Nash equilibrium (PNE) of the associated bidding game. For mathematical convenience, we assume x min i = 0 in the remainder of this paper and it does not make any difference in our results.
and J 1 (x 1 ,x 2 (x max 1 )) is strictly convex in x 1 so that x max 1 is the projection ofx 1 x 2 (x max 1 ) onto X 1 and J 1 (x max 1 ,x 2 (x max 1 )) < J 1 (x 1 ,x 2 (x max 1 )) for all x 1 ∈ X 1 \{x max 1 }. The PNE of the other two cases can be obtained in similar ways. Figure 1 shows the best response correspondencesx 1 (x 2 ) (red line) andx 2 (x 2 ) (blue line), given in (18) and (19) for the four cases of PNE of the two-generator problem.

IV. BID-BASED ELECTRICITY MARKET EQUILIBRIUM
For given x −i ∈ X −i , each generator i ∈ N needs to solve the constrained optimization where the cost function J i : R N → R is defined as (6). Let x := (x i , x −i ) ∈ R N be the concatenation of bidding variables. The associated functions µ : R N → R and p i : R N → R are given as x j a j .
Proof: The first-and second-order partial derivatives of for all x ∈ X i and i ∈ N . In addition, the box-constraint in (22) is convex and compact for all i ∈ N . Therefore, this N -player game is a strictly convex game. Corollary 1: The game = (N , {X i : i ∈ N }, {J i : i ∈ N }) has a unique PNE.
Proof: In Theorem 2, it is shown that the game is strictly convex. It directly follows from the results in [18] that a strictly convex game has a unique PNE.

V. VARIATIONAL INEQUALITIES FOR EQUILIBRIUM
Consider the N -player game = (N , {X i : i ∈ N }, {J i : i ∈ N }). Define the best-response mechanism as follows: for each i ∈ N . Then by definition of Nash equilibrium, a strategy profile The set of solutions for VI (K , F) is denoted by SOL(K , F). With the basis of the VI problem, there is a useful equivalence between the supply-bidding game and a properly defined VI .
Proposition 1: A PNE of is a solution of VI (X , F ) where X := X 1 × · · · × X N and . . .
Proof: 1 Due to its convexity of the constrained optimization (22), the optimality conditions are of the KKT (inequality) systems By definition, these variational inequalities are necessary and sufficient conditions for a PNE x o . In addition, Definition 1 tells us that x o ∈ SOL(X , F ). Theorem 3: A PNE of the game is a fixed-point of the mapping x → X (x − τ F (x)) for any τ > 0 where X refers to a Euclidean projection onto a compact convex set X .
Proof: Let x = X (x) with an arbitrary vectorx ∈ R N . By definition of a Euclidean projection and its convexity, x ∈ X satisfies the condition 1 A similar proof is provided in [18] and we recall it for completeness.
Recalling from Definition 1, x ∈ SOL(X , F ) if and only if ξ − x, F (x) ≥ 0 for all ξ ∈ X . Proposition 1 tells us that a fixed-point x = X (x − τ F (x)) is a PNE of the game .

VI. DISTRIBUTED LEARNING FOR COMPUTING A PNE
In this section, we develop an approach for computing a PNE for the supply-bidding game defined in Section II-B. Our approach is to design a recursive formula x (k+1) = g(x (k) ) and its fixed-point x = g(x) would be a PNE. If we can ensure that g : X → X is a contraction, then the iteration converges to the fixed point of g and hence, the unique PNE. This is due to the well-known Banach fixed-point Theorem [19], also known as the contraction mapping theorem.

A. RECURSIVE FORMULAE
We propose two distributed iteration algorithms to compute a PNE for the supply-bidding game .

1) DISTRIBUTED PROJECTED GRADIENT DESCENT ALGORITHM
To find a PNE of the game defined in Section II-B, we propose a method of distributed learning that has the following form of a recursive formula: where τ (k) i > 0 refers to a step-size of the ith agent at iteration step k and the partial derivatives are explicitly computed as for i ∈ N and k = 0, 1, · · · .

2) DISTRIBUTED PROJECTED BEST-RESPONSE ALGORITHM
Another method of distributed learning for the game is to use the best-response mechanism of the following form: where the associated optimization problem is strictly convex and a unique solution exists for all i ∈ N (see proof of Theorem 2). The constrained optimization (24) has the following closed-form solution for each i ∈ N : that directly follows from the first-order optimality condition

VII. DISCUSSION WITH A CASE STUDY
For illustrating and demonstrating our results presented in Sections IV, V, and VI, we consider a test case of six generators. The generation cost function parameters and generation capacity limits are presented in Table 1. This system contains six generation units, 26 buses, and 46 transmission lines (see [20] for further details of this power network). Fig. 2 shows an iteration procedure of computing a PNE based on the distributed learning algorithm presented in (25). For a better convergence rate and stability, we refine the algorithm as the following: For i = 1, · · · , N and k = 0, 1, · · · ,

A. COMPUTATION OF A PNE
where α i ∈ (0, 1] is a weighting factor and One can see that setting α i = 1 for i = 1, · · · , N results in the same recursive formulae given in (26). Since the constraints X i , i = 1, · · · , N , are assumed to be compact and convex, the recursive feasibility is guaranteed. That is, x ∈ X i and the convex combination nature of the recursive formulae (27).

B. ANALYSIS OF INEFFICIENCY 1) INEFFICIENCY IN ECONOMIC DISPATCH
For quantification of inefficiency that is results from strategic bidding of market participating generators, we define a measure of inefficiency in economic dispatch as follows: where p * = (p * 1 , · · · , p * N ) andp * = (p * 1 , · · · ,p * N ) refer to the assigned power generation profiles with the true marginal cost function parameters b = (b 1 , · · · , b N ) and the equilibrium strategy profileb = (b 1 , · · · ,b N ) corresponding to a PNE x * = (x * 1 , · · · , x * N ), respectively. The equilibrium strategy profile is defined asb i = b i + x * i for i = 1, · · · , N . The PNE x * is dependent on the power demand P d > 0 because the pricing function µ takes a given power demand into account for computing its output price value. Fig. 3 shows how the inefficiency ed varies with P d ∈ [200, 420] MW.

2) INEFFICIENCY IN UTILITY
For quantification of inefficiency from the viewpoint of the utility company, we define a measure of inefficiency in a buyer's perspective as follows: that is a relative difference between the true (shadow) price µ * and the falsified priceμ * that is induced by a PNE x * ∈ X and the associated marginal cost function parameter vectorb = (b 1 , · · · ,b N ) withb i = b i +x * i for i = 1, · · · , N . Fig. 4 shows how the inefficiency utility varies with P d ∈ [200, 420] MW.

3) PRICE OF ANARCHY
In economics and game theory, the Price of Anarchy (PoA) is used to measure how the efficiency of a system degrades due to selfish behavior of strategic players. The PoA is defined as the ratio of the optimal centralized solution and the worst equilibrium: where the numerator and denominator are both negative values so that PoA is always greater than or equal to one. The centralized solution x opt is a global optimum for minimizing the total cost of generators. That is, and x * is a PNE that can be numerically computed by the distributed iterative method given in (27). In this definition of x opt and PoA, a coalition of strategic generators is assumed to be formed as if there is a single owner who can control all generators in the electricity supply market for bidding. Fig. 5 shows how the PoA defined in (30) changes with varying P d ∈ [200, 420] MW.
C. OPTIMAL ED vs. ANARCHIC ED Fig. 6 shows comparisons of allocated power generation profiles that are computed by solving the economic dispatch problem defined in (1). The blue dotted line denotes the optimal economic power generation profile with the true VOLUME 8, 2020 marginal cost function parameter vector b = (b 1 , · · · , b N ) for which another parameter vector a = (a 1 , · · · , a N ) is assumed to be common knowledge. The orange line denotes the optimal economic power generation profile with the falsified marginal cost function parameter vectorb = (b 1 , · · · ,b N ) is the associated PNE. Interestingly, the two generators indexed by 1 and 4 that have relatively smaller values of a 1 and a 4 would be asked to produce less or equal amounts of electric power from the system operator, whereas the other four generators would be asked to produce more or equal amounts of electric power if the generators play a PNE by reportingb i 's instead of reporting the true b i 's. The numerical simulations suggest that there is a threshold θ such that a i /b i ≤ θ impliesp * i ≤ p * i and a i /b i ≥ θ implies p * i ≥ p * i . However, this is just intuition based on observations of numerical examples and has never been theoretically proven. Fig. 7 shows comparisons of individual payoffs of generations. The blue dotted line denotes the net profit each generator can expect to make by generating and selling the allocated power when all generators bid the true values of b i 's. The orange line denotes the net profit each generator can expect to make by generating and selling the allocated power when all generators bid the parameter values ofb i 's. The simulation reveals that playing a PNE benefits all generators in the sense that they can expect to obtain more net profits compared to reporting the true marginal cost function parameters.

VIII. CONCLUSION
This research paper considers a bidding game for an electricity supply market. A rigorous mathematical study of one-and two-player games provide clear and comprehensive explanation of the existence and uniqueness of an optimal solution for the one-player case and a pure-strategy Nash equilibrium for the two-player game. The results are extended for the general N -player supply-bidding game to demonstrate the existence and uniqueness of PNE. We also present a variational inequality formulation of PNE and propose a distributed learning algorithm to compute a PNE. We introduce three different measures of inefficiency induced by falsified marginal cost function parameters that are strategically reported by the selfish generators. Numerical simulation results are provided with discussions of inefficiency analysis and characteristics of anarchic economic dispatch. The future work would be to consider economic dispatch with power transmission losses and constraints, in which the price and optimal dispatch profile should be numerically computed because there is no closed-form solution available. Different forms of the supply function parameterization also needs to be investigated, in which the existence of a PNE is not guaranteed in general and the best or better response dynamics might not converge to a NE, if it exists.