A Decentralized Game Theoretic Approach for Virtual Storage System Aggregation in a Residential Community

The current electricity market is better described as an oligopoly than a market of perfect competition from which, in fact, it may be rather far. The increasing penetration of residential distributed energy resources has led to a significant number of prosumers in the electricity market. Microgrid community, a group of single controllable entity prosumers, is a promising component of the smart grid which will potentially yield a free electricity market. In this paper, we present a novel formation for a residential community microgrid that includes a coalition of prosumer households with solar photovoltaic systems. These households are connected through a virtual power bank that consists of households’ storage batteries and that mediates the communications between the households and the main grid. Using an application of mean field game theory, we find Nash Equilibrium strategies under which such sharing could minimize a linear combination of the households’ energy generation cost, energy consumption cost and revenue of sold energy. The resulting approach is tested on a case study of a constructed community micro-grid in Montreal, Quebec, Canada. The proposed mean field game approach can help decrease the aggregated cost and the individual energy cost. A comparative analytical study on the benefit of sharing was also performed, demonstrating that each prosumer is expected to have at least a 40 percent reduction on their individual cost if belonging to a microgrid community of 100 prosumers located in Montreal city.


I. INTRODUCTION
Electricity, an essential commodity, is unlike other commodity markets in that it is traded through networks with sets of strict constraints both physical and operational. When modeling the competitive electricity market, one must consider the following: the electricity quantities, the market clearing price, and the strategic interactions among the participants which determine the market efficiency. By construction, the electricity networks provide opportunities and constraints on the market participants to adopt strategic behaviors. Thus, game theory enables a potential framework and analytical tool to model the electricity network problems [1]. Furthermore, in the smart grid paradigm, several decision makers The associate editor coordinating the review of this manuscript and approving it for publication was Arash Asrari . will interact to implement a policy where each has their own objective function and system dynamics. The coupling occurs in the cost functions as the decision of an agent affects the payoff of the other agents.
The concept of residential microgrid (MG) has emerged as a promising platform that coordinates and integrates a potentially large number of distributed generations, energy storage systems and local loads. MG consists of a network of distributed energy resources connected locally within a specified geographical area. Residential microgrid can operate not only in a grid-connected mode, but also in stand-alone mode. The objective of MG is to attain a steady and effective energy supply via a simplified implementation of smart grid functionality [2]. MG provides a collaborative environment wherein various participants cooperate to achieve energy savings, peak load shifting and increased social welfare by deploying flexibility assets such as energy storage systems. Coordinating these energy storage systems enables efficient use of renewable energy sources inside the residential microgrid, allowing end-users to cooperate and share the stored energy and achieve higher economic benefits. To this end, storage system aggregator can play an active role in coordinating users' charge and discharge decisions by setting the price of energy sharing; thus, maximising the social welfare. Formulating the MG network problem as a collaborative economic operation, however, is a major challenge. In fact, the problem of operational planning of residential community microgrid for end-users in an energy sharing environment is an active field of research.

A. RELEVANT WORK
The existing research on residential microgrid mainly focuses on: (i) demand response management using various optimization techniques, (ii) interactions with the main grid through the aggregator, and (iii) the coordination of local flexibilities such as photovoltaic systems, electric vehicles and storage systems. For instance, Celik et al. in [3] proposed a centralized energy management system to investigate the impact of distributed residential PV and aggregator energy storage assets on the economical performance of the aggregated network. However, in [3], the energy storage system belongs to the aggregator which is optimizing the network in a centralized fashion. In [4], a novel privacy-preserving distributed parallel optimization framework that allows the participation of a large-scale aggregation of prosumers with residential PV-battery systems in the market for ancillary services is proposed. Correa-Florez et al. in [5] presented an optimization model for aggregated home energy management systems, where the aggregator manages a set of electric water heaters. Although robust optimization is used to include uncertainties, the management of resources for day-ahead energy and local markets takes place in a centralized fashion. In [6], Paudyal et al. proposed a new integrated hierarchical control framework for residential energy optimization that coordinates and controls large electric appliances taking into account residents' comfort and suppliers' rewards. However, in [6], active distributed energy resources within the residential community was not considered.
Most recent research presents models based on offline schemes where the objective is to predict the day-ahead actions. Chen et al. in [7] modeled the Monte Carlo price-based demand response management for residential appliances via real-time stochastic and robust optimization. Huang et al. in [8] adopted a Lyapunov optimization technique that minimizes the MG operation cost. Their approach uses an adaptive electricity scheduling algorithm taking into account the quality-of-service in electricity virtual queue and energy storage virtual queue. In addition, Narayanaswamy et al. in [9] minimized the production cost of a MG by proposing an online convex optimization program for the MG with a single turbine-boiler generator. In [10], authors proposed a hierarchical smart grid interactive architecture to optimize the quality of service and the stability of the MG. Chen et al. in [11] proposed a smart energy management system to optimize the MG operation; the system consisted of an energy storage system and a power forecasting module. In [12], [13], Zheng et al. solved a fully distributed reactive power optimization problem for active distribution network. In essence, the approach there is to offer the potential of finding the globally optimal solution inspired by augmented Lagrangian method and the semidefinite programming convex optimization technique presented in [14].

B. MEAN FIELD GAME THEORY
Game theory enables an analytical framework consisting of mathematical tools to formulate the coupled and complex interactions between the strategic rational players. Recently, game theory and its applications have been used significantly to analyze and model power networks and communication networks problems. In particular, the contributions behind these novel models and algorithms are capturing and analyzing the following characteristics of the emerging smart grid: (i) the heterogeneous aspect of the smart grid, (ii) the necessity for a low-complexity distributed algorithms that model the collaborative or competitive scenarios between the different participants of the smart grid [15], and (iii) the various participants interacting in the smart grid. Chen and Cheng [16] proposed a hierarchical structure of users providing operating reserves to the power system through load aggregators. The bidding problem of load aggregators is formulated as Nash game, where both the existence and uniqueness of the market equilibrium is demonstrated. In [37], a monopolistic gamebased approach for the management of energy flexibility through end-users, aggregators, and the distribution system operator is proposed. Mayorga [38] proposed a data-based Stackelberg market strategy for a distribution market operator for the coordination of power dispatch among various virtual power plants.
Mean Field Game (MFG) theory is an emerging platform and a methodology that enables the analysis of a multiagent coordination problems in which the dynamic of each individual agent is impacted by the statistical behaviour of the population, and in which the individual agent effect on population distribution diminishes as the number of agents increases [19]. MFG theory studies the existence and generation of Nash equilibrium strategies for sophistical dynamical games with large population of agents [20]- [22]. Basically, the theory finds control solutions with negligible error to the finite population problem compared to the infinite limit population problems. The core of the MFG approach is embedded in the solutions to the the Fokker-Planck-Kolmogorov (FPK) and Hamilton-Jacobi-Bellman (HJB) partial differential equations of the generic agent, which are coupled by the state distribution of the generic agent. Independent of that sequence of papers, MFG was developed in [23]. Numerically applying the MFG approach is only concerned with estimating the aggregated effect (i.e. mass), thus, the numerical complexity of the MFG approach is indifferent to the size of the population.
In a smart grid paradigm, MFG has been applied to many types of populations such as thermostatically controlled loads in [24], [25] and electric vehicles in [26]. However, a concrete construction of mean-field games in a community microgrid that addresses all factors of energy exchanged has not yet been designed. In particular, the design of a decentralized autonomous MG where households, locally connected to the same MG, are independently able to find their best responses has not been fully addressed in the current framework of smart grids. In [27], Mojica-Nava et al. addressed the dynamic population games and a droop control via hierarchical microgrid management system. The notion of community energy storage has been addressed in litereture before where the game formed was non-cooperative [28] and where equilibrium prices for MGs and the main grid are only set by a centralized unit. Contrary to [28], this paper models the MGO as a cooperative game where households' aim to find their best response that minimizes their cost function and in turn minimizes the aggregate cost of the MG.

C. CONTRIBUTIONS
The objective of this paper is to bridge the gap between the aforementioned research directions on residential microgrid and the application of MFG theory. We intend to optimize the energy exchange within a residential microgrid community operating in grid-connected mode focusing on the formulation and the optimization of the microgrid problem. In this respect, we present a novel formation for a residential community microgrid that includes a coalition of prosumer households connected through a virtual power bank which mediates the communications within the MG and with the main grid.
To the best of the authors' knowledge, no prior studies have modeled the community microgrid optimization (MGO) problem as a dynamical game without a centralized unit. The proposed framework for microgrid management can be seamlessly integrated within recent grid technologies that eliminate the need for a central operator. Specifically, blockchain can be integrated with the presented mean field game theoretic optimization whereby energy crowd-sourcing can be performed in the operation time-scale; see the recent studies [29], [30] and the references therein for how blockchain can be utilized in grid management. The main contributions of this paper are as follows: • We present the concept of virtual power bank (VPB) which depicts a virtual network of the households' storage systems. In addition, the VPB is connected to the main grid and it plays the role of local electricity market, enabling sharing among a large number of prosumers in the microgrid.
• We develop the first application of non-linear decentralized mean field game theory to the residential community microgrid along with the associated numerical algorithms that solve the constructed decentralized nonlinear MGO problem. We demonstrate that the finite problem converges to the infinite limit population problem with negligible error.
• We formulate the MGO problem for the constructed MG via means of dynamical game theory, where the connected households cooperate to minimize the total energy cost of the formulated microgrid while simultaneously maximizing their net profit individually via means of decentralized mean field game theory.
• We prove the concept of the proposed approach through a test case simulations conducted on a constructed community microgrid in Montreal, Canada. We find that every prosumer can achieve a minimum of 40 percent reduction of their individual cost by joining a microgrid community composed of at least 100 prosumers.

D. PAPER STRUCTURE
The rest of the paper is structured as follows: Section II presents the microgrid architecture, its relevant elements, and the assumptions of the proposed approach. Section III discusses the formulation of the system dynamics including the models of solar PV, prosumers' total operational cost, and the dynamics of the virtual power bank. Section IV presents the game formulation problem via means of dynamical MFG where we specify the mean field Hamilton-Jacobi-Bellman equation (MF-HJB) and the mean field Fokker-Planck-Kolmogorov equation (MF-FPK) for each household. Subsequently, Section V describes the algorithmic implementation of the decentralized community MGO problem including the numerical results for a community MG. Finally, in section VII we conclude the paper. Fraction of Energy Met from the maingrid.

5) FUNCTIONS
µ y Probability Density Function for y. µ θ Probability Density Function for θ . C u MG Pricing Mechanism Function.

II. DESCRIPTION OF THE GAME
A. SYSTEM STRUCTURE Figure 1 illustrates the structure of the proposed residential community MG, which is operating in grid-connected mode. Each household has solar panels, representing their energy generation source, and has energy storage systems (i.e. batteries). Households' batteries are connected to each other through the virtual power bank. In the case of energy shortage in a household, the household will buy the required energy deficit from the MG through the virtual bank. If there is a shortage in the MG power bank, households buy their required energy from the main grid through the power bank. The proposed architecture presented in Figure 1 consists of a large number of prosumer households connected locally to each other through the virtual power bank, which connects the MG to the main grid. In this framework, the households' represented in Figure 1 are are uniform houses, in the sense that all households are identical in terms of loads (i.e. appliances installed), generation potential (i.e. solar panel technology) and the internal structures of these houses. This network of uniform entities can be seen in any new construction of chalet complexes or mass built townhouses; thus the microgrid constructed in this paper can be used to model these projects and the presented algorithm and framework can be generalised in future research to include non-uniform houses. For the scope of this paper uniform houses microgrid is considered.

B. VIRTUAL POWER BANK
In this framework, the network of households' batteries and households' energy profiles constructs the virtual power bank (VPB). Hence, the VPB can be defined as a virtual coalition of households' batteries (i.e. a virtual aggregator) available on the microgrid scale (community level), and connected to the main grid and to all prosumers within the microgrid. By construction, the aggregated capacity of the power bank will become high enough to trade energy with the main grid and to provide energy sharing among a large number of prosumers. In this context, MG is composed of a sufficiently large number of households and a virtual power bank wherein prosumers have the possibilities to exchange energy with each other and the main grid. Similar concepts related to virtualizing storage system have been discussed in [31] and recently in [32], however, both papers assume a centralised energy storage system. Recall Figure 1, each prosumer in the MG has an account in the virtual power bank and their energy balance represents the net amount of energy available in their related batteries. Households can store into their batteries, withdraw from their batteries, or buy from the power bank. Households with an energy surplus can make profit by selling their excess energy to other households through the virtual power bank.

III. MGO: SYSTEM DYNAMICS
Each household has a set of time dependent variables: load and generation, which depend on their location, the deployed technology, and net storage or in other word the balance in the virtual power bank. Their decisions are influenced by the equilibrium market price for electricity in the MG and main grid. Each household coordinates their energy consumption in order to minimize their individual costs so they collectively minimize the aggregated MG operation cost.

A. HOUSEHOLD GENERATION DYNAMICS
In this paper, we assume the MG is a two dimensional area. In addition, the community MG is composed of N households, where N is a sufficiently large number for the MFG theory to be applicable. Aziz et al. in [33], [34] have shown that the infinite control problem can be approximated by a decentralized MFG finite number problem for N ≥ 200. Later in subsection V.D we will show that a MG of 100 prosumers is sufficiently large enough for the MFG approach to be implemented. All the variables are time dependent but the time subscript is removed to simplify the notations.
For simplicity, we consider the scenario where households are homogeneous i.e. they deploy the same technology for generation and storage. Using [35], [36] the power generated by each solar photovoltaic pannels (PV) at household i is given by: Correspondingly, the generated energy θ i by household i at time t is governed by the following stochastic differential equation: where W i θ , 1 ≤ i ≤ N are N independent Wiener process (i.e. Brownian Motion) and θ is a positive constant.
Assuming that households are homogeneous, the stochasticity in PV power generation will be the result of (G i ING , T i c , T i r ) i.e. the effects of the technology deployed and the sun position along with air temperature. Denote by h i the set of generation variables i.e. h i : Also, we assume that each household has adequate technology to determine T c and T r .

B. HOUSEHOLD OPERATION COST MODEL
Dealing with PV, solar radiation incurs zero fuel cost. Hence, the cost is mainly due to the operation and maintenance costs (O&M) of the PVs and to the cost of storage. In this paper, we assume the cost of operation and maintenance O&M is K O&M = 0.1095($/kWh) for generation and C B,M = 0.001($/kWh) for storage. Thus, the cost of generation for household i is given to be: where C max is assumed uniform for all households.

C. ENERGY DEMAND MANAGEMENT
The energy demand is met by withdrawing from the household's storage, from the MG through the power bank (i.e. from other households) or from the main grid through the power bank. For simplicity, we assume that household i, 1 ≤ i ≤ N has a time varying energy demand profile, y i (t), which has to be met at every time t, 0 ≤ t ≤ T . Also, for the scope of this paper we assumed y i (t) a sufficiently smooth function [38]), and the probability density function for the MG community household energy demand, µ y (t, y), is known by each household. Hence, household i at time t observes their profile y i (t) and by using µ y (t, y), household i can find the expected aggregated demand in the MG.

D. HOUSEHOLD POWER BANK ACCOUNT
Each household has an account at the power bank. By analogy to financial banks, each household has a chequing account, an investment account, and a credit account, denoted by b i (t), r i (t), and γ i (t) respectively. Contrary to classical financial banks, the MG virtual power banks aims to maximize social welfare instead of maximizing its own profit. The VPB plays the role of a local electricity market, in which prosumers of the microgrid community decide on the resource type, resource amount and price of electricity. The MG community is composed of two layers: (1) virtual layer represented by the VPB and (2) physical energy layer represented by distribution grid or the physical microgrid distribution grid. In the virtual layer, the estimated generation, consumption and energy price are calculated, while in the physical layer, represented by the distribution grid or microgrid control, the physical distribution of electricity is accomplished. In the proposed framework, we assume that grid constraints are not stringent; the focus of this paper is the proof of concept that MFG framework presents a potential approach for the MGO problem.
Recall θ i (t) and y i (t) are the amount of energy generated by household i and the load of household i at time t respectively. The set of decision variables, u i r (t) and u i γ (t), represent the controls for the amount of energy sold through or withdrawn from the MG power bank, respectively. Define as the net energy after meeting current demand. The dynamics of b i (t), r i (t) and γ i (t) are as follows: where 0 < u i r (t) ≤ 1 when δ i (t) > 0 and u i r (t) = 0 when δ i (t) ≤ 0 and, where 0 ≤ b i (t) ≤ C max for all t and where C max is the storage battery capacity for household i and 0 ≤ u i γ (t) ≤ 1 for all t. The dynamics are formulated such that household i will not contribute to the MG power bank at time t when household i has a shortage (i.e. δ i (t) ≤ 0) and that y i (t) is always met for all time t and for all i, 1 ≤ i ≤ N . Using (5), (6) and (7) the dynamics of b(t) can be derived to the following set of differential equations. For simplicity we will drop the household superscript i. The observations can be categorized in two scenarios; (i) scenario one (SC1) where δ(t) ≤ 0 and (ii) scenario two (SC2) where δ(t) ≥ 0. In SC1, we have δ(t) ≤ 0 then u * r = 0, 0 ≤ u * γ and: where δ(t) > 0 will be divided into two cases: • Case 1: It is optimal to sell, then 0 < u * r , u * γ = 0 and • Case 2: It is optimal to recharge, then u * r = 0, u * γ > 0 and (10) Assuming P PV and y are piece-wise continuous and differentiable and using (8), (9) and (10) the readers can prove that db is continuous and differentiable and that u γ and u r are continuous.

IV. MGO: DYNAMICAL GAME FORMULATION
The objective of each household is to minimize their cost function over the time period 0 ≤ t ≤ T f . Treating households as prosumers, household i at time t can either withdraw from or sell through the power bank. Thus, in the case of energy surplus at time t (i.e. δ i (t) > 0), household i can either sell their surplus to the MG through the power bank (i.e. u i r (t) > 0) or buy from the power bank to fill its battery (i.e. u i γ (t) > 0). On the other hand, in the case of energy shortage at time t (i.e. δ i (t) ≤ 0), household i will withdraw energy from the power bank to meet its current load and optimally fill its battery; thus, u i r (t) = 0, u i γ (t) ≥ 0 and γ i (t) ≥ |δ i (t)|. The state variables of each household at time t are: energy generated θ(t), energy balance in the power bank b(t), demand y(t), γ (t) and r(t). Denote by µ y (y, t) and µ θ (θ, t) the probability density function for y and θ at time t, respectively. In this paper, µ y (y, t) is expected to follow a normal distribution while µ θ (θ, t) is expected to follow a beta distribution function [39].

A. MG EQUILIBRIUM PRICE: P MG
The overall objective of the MGO is to minimize the aggregated cost. Thus, regarding the energy equilibrium price in the MG, we will adopt a pricing mechanisim that maximizes the social welfare (i.e. households will minimize their individual costs which in turn minimizes the aggregated cost and maximizes the social welfare in the MG).
Walrasian Equilibrium is an economic concept in decentralized economic systems that aims to ensure the equilibrium between the demand and the supply. The primary goal is to find the optimal pricing mechanism based on the demand and the supply in order to maximize the social welfare in the community microgrid. The Walrasian Equilibrium can be defined based on three main components: (i) maximizing the service of the prosumers, (ii) market clearing, and (iii) maximizing the social welfare [40]. In essence the equilibirium price is found such that: (i) every household maximize his/her utility given prices, and (ii) markets clear (i.e. the total demand for each commodity just equals the aggregate endowment at every time t). For those reasons we will use Walrasian equilibrium (also known as the competitive equilibrium) to find the MG equilibrium price [41], [42]. In short, the key result of the Walrasian Equilibrium theory is the fundamental first welfare theorem.
Lemma 1: First Welfare Theorem [42]. If the tuple (price, {load} N 1 ) where N is the total number of households in the MG forms a Walrasian Equilibrium then the load matrices {load} N 1 are pareto-optimal. In other word if the tuple y and P MG are pareto-optimal (i.e. households minimize their objective function and the VB has a clear market) then the energy exchange economy through the VB follow a Walrasian equilibrium.
The aggregated demand in the MG at time t is the sum of the households' demands at time t i.e. i=N i=1 y i (t). Assuming that the retail price of energy is proportional to the first order derivative of the time-dependent generation cost and using the fundamental first welfare theorem, the pareto-optimal pricing mechanism denoted by P * MG in the MG is proportional to the aggregate cost of meeting the total load in the MG. (11) where y is the range of y(t) for all 0 ≤ t ≤ T .
The main grid energy pricing mechanisms considered here is mainly based on a flat rate (i.e. P MA is constant over time). Households in the MG know which mechanism is applied and have full access to P MA at any t, 0 ≤ t ≤ T .

B. DECENTRALIZED MEAN FIELD OPTIMAL CONTROL
The MFG approach is a decentralized control theory where each agent finds the expected value of the mean population and accordingly finds its best response. Compared to finite games, in mean-field games, the players (households) do not react to actions from individual players but rather to the aggregate behavior of all players. The result of such approach is proven to be the Nash Equilibrium i.e. households have no incentive to deviate. Each household action has a negligible influence on the aggregated value of the energy produced or required in the microgrid. Hence, household individual action has negligible effect on P MG i.e.
The state variables are θ i (t), y i (t), γ i (t), r i (t), b i (t) . The decision variables are: u i r (t) and u i γ (t). Each agent knows the probability density functions µ y (t) and µ θ (t) for y(t) and θ (t) respectively for all t, 0 ≤ t ≤ T . VOLUME 10, 2022 The MFG framework permits an analysis in terms of large population decentralized control problems where each user employs its local state information and an estimation of the MG aggregated behavior. The central equilibrium theorem of MFG theory provides a scenario in which the impact of all other users on a given user constitutes a deterministic component in its evolution when all the other agents use the MFG Nash Equilibrium strategy. More specifically, subject to technical conditions given in [21], which include H1-H5 below: (H1) The control value space U is a compact subset of R m and the final time T satisfies 0 < T < ∞.  (17) and (19) yields a unique solution continuous and uniformly Lipschitz in (θ, y, b). The main infinite population Mean Field Game Nash Equilibrium Theorem (see [20], [21]) states that for an infinite family of individual agent nonlinear stochastic dynamical systems, here of the form (2), and individual expected cost to go performance functions, (15), where here L is of the form (26), there exist best response feedback control laws as in (18) and in Table 1, where any agent's control depends upon the information set which consists solely of the probability density function of the generated energy and load (i.e. µ θ µ y and the individual agent's state. Furthermore, the best response control laws, the resulting generic agent's state distribution and its optimal cost to go (i.e its value function) are given by the infinite population MFG equations MF-HJB, given respectively in (17) and (19) below, and which together give the Nash equilibrium for the infinite population system for both scenarios.
Theorem 1: M FG -Nash Theorem. Assume that the conditions of the MFG Nash Equilibrium Theorem hold and hence that the best response control laws U ∞ = {u i 0 = u i 0 (t, x i |µ t ), 1 ≤ i < ∞} generating a Nash equilibrium for an infinite agent population system and associated performance functions exist.
where u i ∈ U, the set of all past dependent controls of the form u i = u i (t, x i |µ t ).

C. SYSTEM PERFORMANCE AND COST FUNCTIONS
The cost function L(b, y, θ, γ , r, t) is given by (12) where w 1 and w 2 denote the fraction of demand consumed from the MG and the main grid respectively. Recall that energy exchange happens through the VPB. household i meets his/her load from both MG surplus or maingrid. In our framework there is no centralized unit, the VPB is the medium where energy is exchange. The aggregate energy sold by households and aggregated load is calculated. Recall the assumption that household i buys from himself/herself when withdrawing from his/her battery. Fairness in the community MGO problem is interpreted as follows: all households in the MG concur the same share from both energy sold in the MG and from energy withdrawn from the maingrid. Thus fairness is guaranteed when households incur the same share from both the MG and the maingrid (i.e. w 1 and w 2 are the same for all households).
Adopting the fairness assumption, w 1 and w 2 are calculated as follows: where θ is the range of θ (t) for all t, 0 ≤ t ≤ T . Recall the P MG in (11), the expected cost function can be written as: Hence the cost function in (26) becomes: +γ (t)w 2 P MA )dt (14) In (14), coupling occurs in the cost function where the aggregated demand and the aggregated generated energy affect the decision of the household, particularly in P MG . Following the cost function L(•) in (14) the cost to go J (b, y, θ, γ , r, s) is given by: and accordingly the value function v(·) is given by: Assuming that all functions are sufficiently smooth then the mean field Hamilton-Jacobi-Bellman equations (MF-HJBs) for each scenario are: • SC1: δ(t) ≤ 0, u * r = 0 and the MF-HJB-SC1 is: and thus: • SC2: δ > 0 i.e. household can either sell the surplus, fill the battery or do nothing. The MF-HJB-SC2 is: thus: Using (20) and taking into account that (i) P PV represents the rate of change in energy production θ (i.e. supply), (ii) dy represents rate of change in load y (i.e. consumption) and (iii) the term ∂v ∂b represents the rate of change in cost to go function v with respect to household balance b i.e.
The optimal solutions for the MF-HJB-SC2 equation in (19) are derived. The optimal solutions are presented in Table 1.

D. EXISTENCE AND UNIQUENESS OF THE HJB EQUATION [34]
Consider the following Hamilton Jacobi Bellman equation scheme: where G is the generalized Hamiltonian defined by: x, u). (22) for all (t, x, u, p, P) ∈ [0, T ]×R n ×U ×R n ×S n and consider the following assumptions: (T1) The control value space U is a compact subset of R m and the final time Theorem 2: ( [44], Chapter 4,Theorem 5.2,p. 190,Theorem 6.1,p.198) Let T1 and T2 hold. Then the value function V (x, t) of the corresponding stochastic optimal control problem on D × [0, T ] is linearly bounded and uniformly Lipschitz in x ∈ D, and V (·, ·) = v(·, ·), where v(·, ·) is the unique viscosity solution of the HJB (21). Furthermore, v ∈ C 1,2 [0, T ] if and only if it is a classical solution to (21).

V. ALGORITHMIC IMPLEMENTATION OF DECENTRALIZED COMMUNITY MGO PROBLEM
Here, we will provide the algorithms that solve the MF-HJB in both cases: MF-HJB-SC1 and MF-HJB-SC2.

A. BOUNDARY CONDITION AND APPROXIMATION METHODS
In order to numerically solve the MFG equations (HJB and FPK), we need to specify the following: 1) boundary conditions and 2) the approximation methods used for discretizing the partial differential equations.

1) BOUNDARY CONDITIONS
Below are the boundary conditions for the MGO problem: where T is to be specified. • 0 ≤ u r , u γ ≤ 1 and the admissible set of control is U admissible := [0, 1].
• Power is assumed to be positive and bounded i.e. 0 ≤ P PV ≤ P max PV , where P max PV is to be specified. VOLUME 10, 2022 • µ θ (θ, t) and µ y (y, t) are specified and known to the households.
• C max is uniform for all household and is to be specified.

2) APPROXIMATION METHOD
To solve the partial equations numerically, these conditions must be satisfied and we have used these approximations methods. Time, energy, household balance, and demand step sizes are denoted by t, θ, b and y respectively. For the convergence of the numerical algorithms, the step sizes should satisfy the necessary conditions in [43] regarding the stability of the Fixed Point Argument Method. In particular, in this case the conditions are as follows: The techniques used for discretizing the 1st and 2nd order derivative for any function g over variable ρ are as follows: • 1st order derivative: • 2nd order derivative:

B. NUMERICAL ALGORITHM FOR UNIFORM AGENTS
The MGO presented in this paper focuses on a community MG in which the statistical information is found based on prior historical data. In addition, the statistical information, µ θ (t, θ) and µ y (t, y), are known to the households. The algorithmic implementation of the MGO presents the numerical solution for the MGO's MF-HJB equations. Algorithm 1 generates a best response strategy for a generic household in the community MG. Note that household i, 1 ≤ i ≤ N , observes his state variables and has complete information of his system dynamics. This is revealed in the input section of Algorithm 1. In addition, recall that household i, 1 ≤ i ≤ N , observes γ i (t) and r i (t), and controls u i γ (t) and u i r (t). The preliminary steps for Algorithm 1 are: (i) Discretize the MF-HJB equations in (17), (19), and (20) using (24) and (25), (ii) Discretize the state differential equations in (2), (8), (9) and (10) using (24) and (25), (iii) Find t, θ , y and b satisfying (23), (iv) Using households previous historical data find µ θ (t, θ) and µ y (t, y) for all 0 ≤ t ≤ T .

C. CASE STUDY: MONTREAL, QUEBEC
In this paper we consider a community microgrid in Montreal, Quebec as our case study. The residential pricing in Montreal's main grid is flat rate, thus regardless of the time of use or current load and aggregate demand the energy price is flat and known to households and P MA = 7.13 cent\kWh. The case study is done over the scope of a summer day where the starting time t = 0 is 12:00 a.m. and T is T =: 11:59 p.m. and households' PV area is assumed uniform and equals about 6m 2 . Previous data has been collected and studied for both households' loads and PV generation. In this framework, it is assumed that the initial statistical information of the load and energy are known to all households and Figure 2 and Figure 3 depict the load and PV generation statistical information respectively in constructed microgrid. The case study presented here can be implemented in any season and weather conditions and main grid dynamical pricing scheme has been introduced in future publication. The numerical results show a generic household's i profile where their state variables y i (t) (i.e. load) and θ i (t) (i.e. generated energy) are shown in Figure 4.
In addition, Figure 4 depicts the expected cost to go along with the microgrid Walrasian equilibrium price. Figure 5 presents the energy profile of household i in the virtual power bank i.e b i (t), δ i (t), γ i (t) and r i (t). Figure 6 depicts household i decision variables i.e. u i r (t) and u i γ (t) along with w 1 (t) and   w 2 representing the portion of energy consumed from the MG and the main grid, respectively. From the numerical results, the reader can infer the following: (i) household i sells their stored energy when their expected short term demand in the near future is met by their expected generated energy and (ii) household i charges their battery from the main grid when their expected shortterm demand exceeds their expected short-term generation.   In addition, we studied the scenario where MG does not exist and households only interact with the main grid. In this scenario the generic household presented in Figure 4 will incur 55.65$ where as the same generic agent incurred only 30.71$ when joining the MG. Running the comparison of the stand-alone cost incurred by a generic household i and the cost incurred by joining the community MG for all i, 1 ≤ i ≤ N , household i cost was reduced on average by 23.78$ by joining the MG which accounts for 42% of their total cost. In addition to the significant 40% cost reduction, the proposed approach will reduce the peak energy in the main grid. Table 2 presents the algorithm's computational performance for the Montreal, Quebec case. The specifications of the computer on which the simulations were generated are as follows: 1) processor: Intel(R) Core(TM) i7-4600U CPU @ 2.10GHZ, 2) RAM: 8GB and 3) system type: 64-operating system, x64-based processor.

E. DECENTRALIZED APPROACH VERSUS CENTRALIZED APPROACH
The loss function for each household in the infinite population decentralized case is given by: L(·) := E T f 0 (γ (t) (w 1 (t)P MG (t) + w 2 (t)P MA (t)) −r(t)P MG (t) + C O&M (t))dt (26) where P MG is dependent on the estimated aggregated generated power and load. We recall that this loss function is utilized to calculate the MFG-equilibrium control laws that are employed for a finite population according to the MGF methodology.
On the other hand, in the centralized scenario all the information is known to the centralized unit; thus, the aggregated demand and P MG the mass effect is calculated using the collected real data. Figure 7 presents the numerical comparison between the centralized and decentralized control performances in the 50 and 100 MG prosumers' cases. Analyzing the the data in Table 3 and the two results in Figure 7, one can infer that the MFG methodology equilibrium tends to approximate the performance of the centralized control as the number of agents is sufficiently large enough, in this case (N ≥ 100).

VI. CONCLUDING REMARKS
The work in this paper presents a game theoretic analysis for a community microgrid. The novel architecture of such MG is constructed by the deployment of a virtual power bank which (i) contains the households' batteries and (ii) mediates the communication between the MG households and the communication between the MG and the main grid. This paper presents a formulation for the decentralized MGO problem for the constructed community MG via mean field game theory. We propose a framework that solves both the MF-HJB-SC1 and the MF-HJB-SC2 equations in (17) and (19), respectively. The solutions for the MGO are proven to be Nash Equilibrium strategies that minimize household individual cost functions and in turn the aggregate cost in the MG by at least 40%.
Extending the framework presented in this paper to solve the general MGO problem where the main grid electricity market is free (i.e. P MA is based on the demand and supply solely) and households possess controllable loads will be further investigated in future work.