Consumer Flexibility Aggregation Using Partition Function Games With Non-Transferable Utility

This paper explores the aggregation of electricity consumers flexibility. A novel coalitional game theory model for partition function games with non-transferable utility is proposed. This model is used to formalize a game in which electricity consumers find coalitions among themselves in order to trade their consumption flexibility in the electricity market. Utility functions are defined to enable measuring the players preferences. Two case studies are presented, including a simple illustrative case, which assesses and explains the model in detail; and a large-scale scenario based on real data, comprising more than 20,000 consumers. Results show that the proposed model is able to reach solutions that are more suitable for the consumers when compared to the solutions achieved by traditional aggregation techniques in power and energy systems, such as clustering-based methodologies. The solutions found by the proposed model consider the perspectives from all players involved in the game and thus are able to reflect the rational behaviour of the involved players, rather than imposing an aggregation solution that is only beneficial from the perspective of the aggregator.


I. INTRODUCTION
The increasing penetration of renewable energy sources is leading to major changes in power and energy systems all around the world [1]. The importance of consumers is increasing in this context, as they provide the potential to balance the variation of renewable-based generation through consumption, or demand-side flexibility, e.g. the proposal for a regulation on EU internal electricity market places consumers in a central role in future energy systems, encouraging and enabling them to take part in the energy transition and electricity market transactions [2]. Demand-side flexibility can be defined as follows, according to the European Commission's European Smart Grids Task Force (ESGTF) [3].
Definition 1 Demand-side flexibility is the flexibility at the customer side, this includes both flexible load, generation and storage. Demand-side flexibility is ''behind the meter'' or The associate editor coordinating the review of this manuscript and approving it for publication was Sudhakar Babu Thanikanti .
''behind-the connection'', meaning that the measurements on connection level typically also include other (flexible or non-flexible) load or generation.
Definition 2 Flexibility is the ability to purposely deviate from a planned / normal generation or consumption pattern.
Demand-side flexibility can be implicit or explicit; these terms are defined as follows, according to the European Commission [3].
Definition 3 Explicit demand-side flexibility is the committed, dispatchable flexibility that can be traded (similar to generation flexibility) on the different energy markets (wholesale, balancing, system support and reserves markets). This is usually facilitated and managed by an aggregator that can be an independent service provider or a supplier. This form of demand-side flexibility is often referred to as ''incentive driven'' demand-side flexibility.
Definition 4 Implicit demand-side flexibility is the consumer's reaction to price signals. Where consumers have the possibility to choose hourly or shorter-term market pricing, reflecting variability on the market and the network, they can adapt their behaviour (through automation or personal choices) to save on energy expenses. This type of demand-side flexibility is often referred to as ''price-based'' demand-side flexibility.
The change of consumption patterns may occur in a single time-scale, e.g. by reducing or increasing the consumption at a certain time; or it may be achieved by shifting the consumption to other periods, e.g. using a washing machine some hours later or earlier than initially planned [4].
The consumption flexibility may be activated by different means, usually associated to demand response programs or events [5]. The most common way is through real time pricing, which means that different electricity prices are set for different time periods, thus incentivizing the consumer to adapt its consumption accordingly, i.e. reducing the consumption when prices are high and increasing it when prices are lower. However, this flexibility activation measure is associated to a high uncertainty, as it depends directly on consumers' willingness to change their consumption patterns, as well as on their awareness on the electricity prices variation. Another, more effective, demand response program is the direct load control, in which consumers are paid to allow an aggregator to control directly some of the devices (usually water heaters or other devices with low impact on consumers' comfort, and usually only in situations of high need from the system) [6]. Consumers' flexibility can also be activated in advance, by providing incentives to consumers (in the form of payment or others) to change their consumption pattern at a certain time in the future. This is usually accomplished by aggregators, which sell consumers' flexibility in the electricity market. Although some changes in current electricity market models are already taking place, market models are still not able to accommodate small-sized energy resources, and therefore, small consumers such as houses and buildings can only participate in flexibility transactions through aggregators, which guarantee the minimum volume to enable the market participation.
Consumer and demand response aggregation models in power and energy systems are typically based on clustering approaches. E.g. the K-means clustering algorithm is used in [7] to aggregate consumers reduction capability. In [8] consumers are aggregated using an optimization-based clustering approach, as facilitator to their participation in electricity markets. This optimization-based clustering approach is done in a centralized way by the aggregator. A similar approach is proposed by [9], which presents a distribution system clustering optimisation problem from the perspective of the system. Another optimization-based approach is proposed in [10] to minimize the total operating cost of the whole system (i.e. aggregated households). An energy management model for clusters of commercial buildings is proposed in [11], however, the model considers model pre-defined clusters of buildings. The model proposed in [12] aggregates consumers demand response participation using hierarchical clustering and fuzzy C-means. In fact, a comparison of clustering methods for demand response aggregation is carried out in [13] and the study concludes that hierarchical clustering is the most appropriate approach, as it aggregated the resources with a minimum cost for the aggregator. Hierarchical approaches are, indeed, promising solutions in this field, although, despite often introducing some degree of distributed computation, they are usually developed from the perspective of a central entity (e.g. system or aggregator). Some examples are [14], which proposed an hierarchical control approach to aggregate energy storage systems into groups that can be scalably controlled. The proposed control strategy is formulated based on an ideal power transfer model for the microgrid, and there is no consideration of energy storage units with different characteristics (nor even location within the microgrid) nor goals; hence only the system perspective is considered. An hierarchical control scheme is also proposed in [15] to coordinate large-scale residential demand response through the coordination of house energy management systems, using distributed optimization. Another hierarchical energy management strategy is proposed in [16], considering a multistep hierarchical optimization algorithm based on a multiagent system. The community energy management system reaches decisions in a centralized way. The work presented in [17] includes a multi-agent coalition formation process to support players' electricity transactions. This coalition formation process considers the localization of each player and the available power in the area, The proposed coalition model performs a simple allocation of the player to nearest aggregation, if there is enough power available to cover its needs. This model does not, however, consider the competition and cooperation between agents within the same aggregation, nor the competition between different coalitions. Another coalition formation model in multi-agent systems is proposed in [18] using ant colony optimization. This model considers cooperative agents, which put the entire system goal before their own goal. Computing large scale cooperation problems in a centralized often leads to a high computational burden; hence distributed approaches are emerging as promising solutions to distribute the computational effort among the involved players [19]. [20] presents an overview of distributed control strategies for microgrids. Some relevant examples deal with the energy management of multi-energy systems, such as the event-triggered coordination approach presented in [21] and the distributed double-Newton descent algorithm proposed in [22]. However, distributed optimization approaches are also used for solving economic dispatch problems [23] and to determine optimal power flows [24]. Distributed methods are promising, as they can provide more flexibility for consumers, safeguarding their individual information, while distributing the computation tasks to individual consumers. However, these models still present some shortcomings, such as the need for relying on computational tasks on the consumerside, which may be subject to failures (e.g. of equipment or individual energy management system), and is largely dependent on the widescale adoption of automated systems that enable consumers (often unexperimented) to abstract from the management and automated decision process. Moreover, distributed approaches often deal with partial data processing, refraining from an overall knowledge of the system as a whole. The widespread of such approaches comes to mitigate the significant problem of solving large scale problems in an acceptable execution time, but the compromise, especially in dealing with partial data, while necessary in critical operational problems, is not always needed when dealing with strategic planning problems, in which the time available for reaching decisions is not as strict.
These works present, undoubtedly, very relevant contributions in the topics they address, and in coalition formation in particular. However, the coalition formation process is often either overlooked or very simplified, almost always done from the perspective of the system and assuming individuals have nothing (or little) to say about the coalition process. Individual players have their own characteristics and objectives (e.g. gaining and much profit or as little cost as possible), which are often conflicting with the perspective of the system. It is certain that a number of different approaches, including clustering-based approaches, can model some of these characteristics and objectives, but they are very limited in terms of defining and balancing the different perspectives. The model proposed in this work contributes to overcoming this problem by bringing together the different goals and perspectives of the individual players, of each distinct coalition, and of the system as a whole, in a way that all the preferences and objectives are taken into account and that a global solution that balances the outcomes from the different perspectives is achieved.
Hence, new models are required, considering the individual perspective and benefit of each player independently, in order to achieve more solutions that assume players that have their own rational behaviour, and their own objectives and goals.
Such solutions should provide more robust approaches for aggregators/operators to foresee the expected outcomes from the coalition formation process; while at the same time representing solutions that benefit both the system and the individual players.
The rational behaviour of individual players has been extensively explored by game theory studies, including several applications for problems in power and energy systems, see e.g. [25] for an overview on game theoretic methods in this domain. Worth highlighting is the work presented in [26], in which a game-theoretical model for energy scheduling of demand side resources is proposed. [27] presents an energy management model based on game theoretic assumptions, in [28] the optimization of the distribution system planning is performed using game theory, and in [29], a game theory-based strategy for electricity market participation is proposed.
Several studies on the formation of coalitions between agents have also been developed in the coalitional game theory field. A relevant review in coalition structure formation is provided in [30], while some relevant approaches are emerging considering aspects such as players' hesitation and exposure to risk [31]. However, many problems in power and energy systems introduce many complex issues that cannot be dealt with by traditional general models. One of these issues is uncertainty. [32] addresses the problem of forming a coalition of agents in a similar way to a problem of data clustering, grouping agents with similar characteristics; a stochastic game is proposed for coalition formation considering uncertainty. Another relevant, and unexplored problem refers to the constant need of considering externalities in the coalition formation process. In the subject addressed in this paper, when reaching decisions on how to aggregate, players need to consider the competition in the market (total number of coalitions), and the internal competition within each coalition (if they are more likely to be called to provide flexibility when such is needed -the diversity of players within the coalition and volume of flexibility of each player relative to the total volume of all players in the same coalition). This type of problem means that the value or utility of each agent cannot be assessed by simply evaluating the coalition the agent is part of, but also requires evaluating other coalitions and even other external factors. A coalition formation game that considers externalities is called a partition function game [33]. Partition function games, introduced by Thrall and Lucas [34], generalize coalitional games by allowing for externalities across coalitions. They arise naturally in environments where players can form binding agreements, and cooperate inside coalitions but compete across coalitions [35]. As an example, in [36], an extensive game is defined based on a partition function game. This is a propose-respond sequential bargaining game where the rejecter of a proposal exits from the game with some positive probability.
As an aggravator to the complexity of coalitional gametheoretical models application to this problem, the agents that act in this environment usually have very distinct characteristics, goals and objectives (e.g. generators may be based on renewable energy sources or not, consumers may be able to provide flexibility or not, consumers may be individual or retailers-which are themselves an aggregation of small-sized consumers). Moreover, the gain or preferences of each agent also depends directly on the individual profit they will get from the transaction of their own flexibility. This means that the utility of players in such problems cannot be assumed as equal among all involved agents. The assessment of players' utility is dependent on their own particular characteristics, goals and objectives, which leads to the need of modelling problems as Non-Transferable Utility (NTU) models [37]. Although partition function games have been widely studied, the modelling of these problems with NTU has not been explored in depth.
In summary, current coalition formation models for flexibility aggregation in power and energy systems are limited, being typically developed by means of aggregation methods [12]. These models are almost exclusively developed through clustering methods, which are usually conceived VOLUME 9, 2021 from the perspective of the aggregator, which is conflicting to the goals, objectives and preferences of the players. Hence, models that enable considering and balancing the different perspectives are required. Moreover, coalitional game theory models that may provide suitable solutions for this problem, namely based on partition function games with NTU have not been explored. Finally, most game theory models are usually not applicable in real large-scale problems, having a reputation of being just ''toy problems''.
This paper overcomes these gaps by proposing a new model for consumers' flexibility aggregation based on coalitional game theory. The original contributions of this work are as follows: • Design and formulation of the proposed model as a partition function game with non-transferable utility, thus enabling modelling players' independent characteristics and preferences, while considering external factors that are not intrinsic to the players themselves -unexplored issue in coalitional game theory literature.
• Formalization of the proposed model considering the perspectives from all players involved in the coalitional game, thus reflecting the rational behaviour and preferences of the involved players while respecting the needs from the system, thus resulting in solutions that are beneficial for both the players and system, rather than imposing an aggregation solution that is only defined from the perspective of the aggregator, as is commonly done in current literature.
• Experimentation and validation of the proposed model under a realistic large-scale scenario comprising 20 310 consumers, demonstrating the scalability of the proposed model and applicability to real-world problems, which is usually disregarded in game theoretical works. The proposed model is formalized as a partition function game with non-transferable utility. The consumers' flexibility aggregation model is formalized, and utility functions are defined to measure players' preferences, considering the economic gain -income from selling their flexibility volume, and the expectation component that comprises the competition in the market, the internal competition within the player's coalition, and the diversity of players within the coalition. The proposed model is evaluated under two case studies. A smallsized illustrative example is provided as means to explain in the detail how the proposed model works; and a largesize case study based on real data, comprising 20 310 consumers is presented in order to access the applicability of the proposed model under realistic application conditions. The performance of the proposed model is compared to the results from current state-of-the-art models for consumers flexibility aggregation, namely several clustering approaches used in several related works. Results show that, although the proposed model adds increased complexity when computationally solving the problem; the achieved results demonstrate a higher social welfare, thus providing results that are more appropriate and fair to the whole community of players, while respecting the needs from the system perspective. Thus, the proposed model enables considering and balancing the perspectives from players and from the system.
After this introductory section, section II formalizes the partition function game with NTU, and presents the proposed consumers' flexibility aggregation model, including the utility functions used to assess the players' preferences. Section III presents the experimental findings, including a complexity analysis of the problem, the illustrative case, and the real case scenario. Section IV provides the conclusions of this work.

II. PARTITION FUNCTION GAMES FOR FLEXIBILITY AGGREGATION A. PARTITION FUNCTION GAMES WITH NTU
Let N be a finite, non-empty set of players {1, . . . , n}. Any subset C of N is called a coalition. The grand coalition is the set N of all players. A coalition structure over N is a collection of non-empty subsets CS = {C 1 , . . . , C k } such that When the payoff that C can earn is dependent on the coalition structure formed by the players in N \C, the problem is defined as a partition function game [38]. In order to introduce this concept, we start by introducing the notion of an embedded coalition. Definition 5 An embedded coalition over N is a pair of the form (C, CS), where CS ∈ CS N is a coalition structure over N , and C ∈ CS. We denote by E N the set of all embedded coalitions over N .
A partition function game can accordingly be defined as a game that assigns a value to each embedded coalition [39].
Definition 6 A partition function game G is given by a pair (N, u), where N = {1, . . . , n} is a finite non-empty set of players and u: E N → R is a mapping that assigns a real number u(C, CS) to each embedded coalition (C, CS).
When the value attributed to each coalition (i.e. the coalition payoff) can be distributed arbitrarily among the players, we are in the presence of a transferable utility game. However, in most practical application scenarios, such is not possible, and the scenario must be modelled as a non-transferable utility (NTU) game. In NTU games, each coalition has available to it a set of choices, from some overall set of choices = {λ, λ 1 , . . .}; The value of the choices available to a coalition C is captured by a characteristic function, such that v(C) denotes the set of choices available to C. In this way, λ ∈ v(C) means that λ is one of the choices available to C. Players preferences over their choices are captured through preference relations.
λ λ means that the choice λ is preferred at least as much as choice λ'. Let λ λ represent that λ is strictly preferred over λ'. If the player is indifferent between λ and λ then λ ∼ λ'.
If each player i ∈ N is associated with a preference relation i we have a formal description of a NTU game [39].
. . , n} is a non-empty set of players, = {λ, λ 1 , . . .} is a non-empty set of choices, v : 2 N → 2 is the characteristic function of game G, which for every coalition C defines the choices v(C) available to C, and, for each player i ∈ N , i ⊆ × is a preference relation on . It is usually assumed that v(∅) = ∅.
A partition function game with NTU can, therefore, be defined by assigning preference relations to the players' choices that lead to each possible coalition structure.
Definition 9 A partition function game with NTU is given by a structure G = (N , , v, 1 , . . . , n ), where N = {1, . . . , n} is a non-empty set of players, = {λ, λ 1 , . . .} is a non-empty set of choices, v : E N → 2 is the characteristic function of game G, which for every coalition C defines the choices v(C) available to C, and, for each player i ∈ N , i ⊆ × is a preference relation on , which represents the preference of the player on each coalition structure (C, CS).
Modelling a problem as a partition function game with NTU enables defining coalitions' payoff considering externalities, i.e. factors that are external to the actions or choices of the players allocated to the coalition; while also allowing each individual player to consider its own individual assessment of its personal choices. In this way, it is possible to model the consumers' flexibility aggregation problem, as introduced in the introductory section and as detailed in section B.

B. FLEXIBILITY AGGREGATION MODEL
The flexibility aggregation model proposed in this work aims at capturing the essence of a game in which players, representing energy consumers, are aggregated into several coalitions. Each coalition is managed by a flexibility aggregator, which will negotiate the consumption flexibility of the members of its coalition in the energy market. Players (consumers) will get paid for their volume of flexibility at the average price among all players of the coalition they belong to (different price calculation alternatives could be considered, e.g. the maximum price among all players of the coalition, the minimum price, or a weighted average considering the players flexibility volume; however, in this model the average price is considered, as it enables reaching a compromise value among the coalition members). Moreover, players are interested in: (1) being part of coalitions in which their relative volume of flexibility in the coalition is high (so that when the associated flexibility aggregator needs to use some flexibility among its members, it has a higher chance of being chosen); (2) reaching an outcome in which the number of different coalitions is as small as possible (in order to reduce the competition in the market, and thus avoid the consequent potential decrease of the return prices); (3) being part of a coalition with as much diversity among the players as possible, in order to reduce internal competition. Diversity refers to: the volume of flexibility, the type of consumer (residential, industry, commerce, etc.), the price, previous participation in flexibility provision, flexibility profile (e.g. times at which the flexibility volume is available). Fig. 1 illustrates the general process of the proposed model, as explained as detailed in this section. The players' interests and objectives are modelled through the definition of the preference relations on the different choices available to them. These preferences can be quantified, by attaching values to the preferences on the possible choices G of the game G. These values are called utility values or utilities. A utility function u: λ → R is said to represent a preference relation that a player has on an choice over other iff The utility function proposed to model the value of the preference on each choice u n (λ), is formalized as in (1), (2) and (3). u n (λ) = I n Q n , n ∈ N (1) I n represents the income that player n gets by selling its flexibility when being part of coalition C as result from its choice λ. I n is calculated, as in (2), as the expected value of selling player's n flexibility volume V n at the average monetary price M of all players present in the same coalition C as player n. T C stands for the number of players in C. Q n stands for the expectation component of player n, and it incorporates, as shown in (3), the indirect payoff perspectives of the player. x 1 , x 2 and x 3 are scaling weights that enable defining the importance of each of the three elements of the Q n component: The first element calculates the relative flexibility volume of player n according to the players present in the same coalition. The volume of flexibility V n of player n is divided by the sum of all the flexibility volumes V t of all players t that are part of the same coalition C as player n. this means that the smaller the total volume of flexibility in the coalition, the better the assessment of this element will be.
The second element represents the assessment of the total number of coalitions NC, in which the best possible value is achieved when there is only one coalition, and the value decreases as the number of coalitions increases.
Finally, the third element calculates the diversity D C among the players included in C, considering their intrinsic characteristics. D C is calculated as in (4).
where H is the number of components that define the diversity of players in each coalition (e.g. volume of flexibility, type of consumer, price). Hence, D C calculates the average scaled standard deviation sσ C g in coalition C for all characteristics g in a way that sσ C g ∈ [0, 1]. For each characteristic g, σ C g considers the values associated to each member t of the coalition C. Firstly, the mean value µ C g for each characteristic g in the coalition C is calculated as in (7); then we reach the mean deviation σ C g of each value of characteristic g (from each player in coalition C) to the mean, as in (6). The standard deviation σ C g is scaled according to the maximum and minimum values of g, in order to get sσ C g , as showed in (5). Finally, the average between the sσ C g of all characteristics is calculated, giving us the value of D C , as in (4).
The evaluation of the game (coalition formation) results, is usually performed by means of a social welfare assessment [39]. Several types of social welfare calculation methods can be found in the literature; one of the most common, and least controversial is via Pareto optimality, which evaluates if a solution is Pareto efficient by comparing the combination of utilities among the different players. This approach becomes, however, hard to apply and assess when the number of players is high. Another approach is the Utilitarian Social Welfare (USW), which represents the sum of utilities each player gets from λ. A λ * that maximizes USW thus satisfies the following condition.
A solution that maximizes the USW guarantees that the highest global utility is allocated. However, this assessment does not look at the distribution of utility among the players. This is especially relevant when the utility functions of the different players are defined in very different scales, which may result in e.g. only one or a small number of players contributing to the USW maximization by having big utility function values, while all the others are negligible due to the small scale of their utilities. The Egalitarian Social Welfare (ESW) attempts to provide a sense of fairness to the solutions evaluation. ESW says that the worst off member of the society should be as well of as possible. A λ * that maximizes ESW satisfies the following.
By maximizing the worst player's utility, the ESW provides a more fair assessment of the players' choices. However, once again, this does not guarantee the quality of the solution when the utility functions of the players are in very different scales.
Maximizing the utility of a player that is always going to get a small utility value due to the scale of its utility function may mean a large decrease in the utility values of all the other players, or at least of some players that have large utility function values, which will result in a large waste of global utility.
Guaranteeing that the utility functions of all players are in the same scale is, in most cases, not possible, as they represent each player's individual perspective on value of the game preferences. However, in the addressed problem, the scaling of the utility functions ranges can be accomplished as follows. The Q n component is already defined in a range [0, 1] regardless of the player; The I n component is dependent on the average price of all players of the coalition, which, by default are in the same range of values; and the player's available volume of flexibility. This volume is, thereby, the only piece that makes the scale of the different players utility functions vary. The scaled utility function u n (λ) is hence defined as in (8).
Additionally, besides assessing the global social welfare, considering all players N , we are also interested in evaluating the specific social welfare of each coalition. Consequently, USW C , ESW C , USW C and ESW C are defined as in (11), (12), (13) and (14).
USW C and ESW C enable calculating the utilitarian social welfare and egalitarian social welfare of each coalition C individually. USW C and ESW C represent the scaled calculation of USW C and ESW C . Fig. 2 presents a flowchart that describes the implementation steps of the proposed methodology according to the presented model formalization and description.

III. EXPERIMENTAL FINDINGS A. COMPLEXITY ANALYSIS
According to the proposed model, for N players, there is a total of 2 N −1 possible coalitions. Another problem is that the number of coalition structures grows rapidly (considerably faster than the number of coalitions grows) as the number of players increases. The exact number of coalition structures NCS is as in (15).
where Z (N, i) is the number of coalition structures with i coalitions. The quantity Z (N, i) is also known as the Stirling number of the second kind [40], and it is captured by the following recurrence (16): where Z(N, N ) = Z (N ,1) = 1. This recurrence can be understood by considering the addition of a new player to a Notice that finding the optimal coalition structure is NP-complete [40], [41]. The search space is very large since the number of possible coalition structures grows exponentially with the number of players. The number of coalition structures NCS is so large that not all coalition structures can VOLUME 9, 2021 be enumerated-unless the number of players is extremely small. Therefore, exhaustive enumeration is not a viable method for searching for the optimal coalition structure. Optimal algorithms based on dynamic programming have been proposed (e.g. DP [42], IDP [43]), which offer guaranteed run-times over arbitrary coalition value distributions; their complexity is (3 N ). This is, however, still too heavy for cases with a large number of players. Another relevant solution in this domain is the deterministic coalition search algorithm SPLIT [40], which searches breadth first from the bottom of the lattice. This algorithm does well if the grand coalition (a coalition structure with a single coalition of all players) is the optimal structure. The performance of this algorithm, as measured by the number of evaluations required before finding the optimal coalition structure, degrades as the level of the optimal coalition structure increases. Alternative solutions to find the optimal coalition structure in more acceptable complexity are based on meta-heuristic approaches, e.g. Genetic Algorithms (GA), as in [44]; however, due to their stochastic component, these approaches do not guarantee finding the optimal solution, rather the best approximate solution within the desired time frame. (ii) the player type, in which 1 is an industrial consumer, 2 is a commercial consumer and 3 is a residential consumer; and (iii) the price for the sale of flexibility, in monetary units (m.u.). Table 1 shows the characteristics of the considered players. As Table 1 shows, in this game, player a is the one with the larger amount of available flexibility and the player with the lowest price. On the other hand, player c is the one with the highest price, but also the player with the lower amount of flexibility available for trading. Fig. 3 summarizes the players' preferences regarding the different possible choices, assessed according to u. In this figure, the blue bars quantify the preference of player a on each possible choice, the orange bar represents player b, Although the players' individual preferences become clear with this assessment, the very different volumes of flexibility available by the players result in very distinct scales of utility when using u (e.g. u c is always smaller than u a and u b regardless of the choice). This is a problem when assessing the global utility, as some players will have more influence than others. u' has been introduced to overcome this difficulty and provide the means to balance the scale of the utility assessment by each player, by removing the influence of the flexibility volume on the calculation. Table 2, Table 3 and Table 4 present the u n (λ) for each of the possible choices, from the perspective of player a, b and c, respectively. Additionally, the values of the two main components of u n (λ) calculation, namely I n and Q n are also shown. Once again, x 1 = x 2 = x 3 = 0.33.
From Table 2 it is visible that the best choice for player a is {a, b, c}, as this choice maximizes both I a and Q a . On one hand, a benefits from the presence of the other players to increase the sale price of the coalition, as its individual price is the lowest. On the other hand, {a, b, c} enables a to minimize the total number of coalitions and to maximize the diversity of the players within the coalition. The remaining element of Q : relative flexibility volume of the player according to all the players present in the same coalition, is the smallest from all choices, but it is still not enough to have a significant impact on the decrease of Q a . because of a's large volume of flexibility when compared to the other players, which means that its relative flexibility volume is always high, regardless of the choice. Table 3 shows that the best choice for player b is {{a}, {b, c}}. b prefers to be in the same coalition as c in order to increase the coalition price (as c is the player with the highest price), while avoiding player a, as this player contributes to  lowering the price, and also reduces the relative flexibility volume of b within the coalition, due to a's large volume.
The best choice for player c is to stay alone, while the other two players join together in a single coalition, as shown by Table 4. By being alone, c maximizes the coalition price and the relative flexibility volume (being in a coalition with any other player has a strong impact on this component due to c's small flexibility volume when compared to the others). c prefers a and b to be together rather than apart, as in this way, the total number of coalitions is decreased.  Table 5 shows a comparison between u n (λ) of all players as well as the USW (λ) and ESW (λ) for all choices, in order to facilitate the assessment of global utility. {a, b, c} maximizes USW' as it is the best choice for player a, the second best for player b, and this compensates for the worse utility for player c, which is the best from all the worst utilities of each choice for all players. The utility of player c in choice {a, b, c} represents, consequently, the maximum value for ESW'. Table 6 presents the u n (λ) of all players for all choices when considering a larger influence of the relative flexibility volume of the player according to all the players present in the same coalition, when compared to the other components of Q n , namely: x 1 = 0.7; x 2 = x 3 = 0.15.  Table 6 shows that when the relative volume of each player in the coalition is the most influential factor in the calculation of Q a . players a and b prefer to be in the same coalition as c, as it maximizes the expected price, and small volume of c is not enough to significantly decrease the relative volume of the players in the coalition. On the other hand, c prefers to be alone, as joining any other player decreases the expected price and causes a large impact in the relative volume of this player in the coalition. In this case, the best choice from the overall social welfare perspective is {{a}, {b}, {c}} as this choice maximizes both USW' and ESW': The best overall solution in this case refers, therefore, to the choice in which each player maximizes its own relative volume of flexibility in the coalition. Table 7 presents the u n (λ) of all players for all choices when considering a larger influence of the total number of coalitions, when compared to the other components of Q n , namely:  Table 7 one can see that when the most relevant factor for the calculation of Q n is the total number of coalitions, VOLUME 9, 2021 the best choice for all players is {a, b, c}, as the grand coalition minimizes the total number of coalitions. In this case, {a, b, c} is the best outcome from the social welfare perspective.
Finally, Table 8 presents u n (λ) when the larger weight in Q n is attributed to the diversity of the players within the coalition, namely: Table 8 shows that when the diversity of the players within the coalition is the most relevant factor for the calculation of Q n , the best choice for all players is once again the grand coalition. Due to the miscellany of the considered players' characteristics, {a, b, c} is the outcome that maximizes the diversity. In this case, {a, b, c} is again the best outcome from the social welfare perspective.
In summary, the results presented in this section show that, depending on the objectives and perspective of each player, the proposed model is able to represent a suitable measurement of their utility. This, in turn, enables the assessment of the best expected choices for each player and from an overall social welfare perspective. In this way it is possible to anticipate the expected rational behaviour of the players and also to identify solutions that are fair to the global players' population.

C. REAL CASE SCENARIO
This section presents the results from the application of the proposed model to a real case scenario. Besides demonstrating the applicability of the proposed model in realistic cases, this section also has the objective of comparing the results of the proposed model with those achieved by several benchmark algorithms typically used to address the problem of consumers aggregation in power and energy systems. The algorithms used in this domain are usually based on clustering approaches [45], with the goal of aggregating players into several coalitions, according to players' characteristics. Clustering algorithms enable creating a set of groups of players (coalitions), while maximizing the similarity of players within the same cluster and also the difference between the several groups. Thereby, the coalitions resulting from the aggregation process include players that are as similar as possible to each other and as different as possible from the players in other coalitions. This approach has demonstrated to be a good solution for cases in which there is a central entity managing these players and dictating the formation of the coalitions [12]. The centralized management is facilitated when all players of the same group are similar. However, this kind of approach completely disregards the perspective, goals and benefit of the individual players, rather only considering the viewpoint of the central management entity. This is a major gap, since the rational behaviour of the players is neglected, which may have a huge impact on the coalition formation, as players will look to maximize their own gain (and not that of the system), if they are allowed to choose how to undertake their coalition process. Moreover, the sense of fairness is also neglected, as these models only care about the gain of the system and do not look for a balance between the benefit for the system and for the involved players as well. The comparison of results between the proposed model and several clustering approaches enables concluding on the advantages and potential of each approach.

1) SPECIFICATIONS
The power network used in this case study is a real 30 kV distribution network, located in Portugal, supplied by one high voltage substation (60/30 kV) with 90 MVA of maximum power capacity distributed by 6 feeders, with a total number of 937 buses and 464 MV/LV transformers [46]. Fig. 4 shows the summarized scheme of the distribution network. The 20 310 consumers connected to this network are classified into five consumer types: 1) domestic (DM); 2) small commerce (SC); 3) medium commerce (MC); 4) large commerce (LC); and 5) industrial (ID). These consumers are distributed throughout the 937 buses of the network, and divided into 6 geographical areas, each corresponding to one feeder, managed by a distinct Virtual Power Player (VPP) [47]. More details on the network and consumers can be found in [48]. Table 9 shows a summary of the most relevant data referring to each consumer type, namely the number of players of each type, the total volume of flexibility available by each player type, and also the average price for each player type. This data is based on the real data used in previous studies [12]. From Table 9 it is visible that the smaller consumers, namely DM and SC, are present in a much larger number than the big ones. Additionally, DM are the players with the highest price for flexibility trading. Besides the price, volume and player type; the location of the player, the previous participation in consumption flexibility / demand response programs, and the demand response types in which the player is currently participating, are also considered as characteristics of the players for this study.
Since it is impossible to discuss each player's individual benefit from the coalition process, due to the large number of considered players, the assessment is performed globally using the USW' and ESW' of the solutions. The clustering algorithms considered in this study are: • Hierarchical clustering [49]. This algorithm builds a hierarchy of clusters using is a top-down approach. All observations start in one cluster, and splits are performed recursively as one moves down the hierarchy. The splits are determined in a greedy manner.
• Fuzzy C-means [50]. This algorithm assigns data points to clusters such that items in the same cluster are as similar as possible, while items belonging to different clusters are as dissimilar as possible. With Fuzzy C-means, each data point can belong to more than one cluster.
• K-Means [12]. This algorithm assigns n observations into K clusters in which each observation belongs to the cluster with the nearest mean. The assignment of observations to the corresponding cluster is performed iteratively.
• Automatic Differential Evolution Load Pattern Clustering (ADE-LPC) [51]. This algorithm solves the clustering problem using a multi-objective version of the differential evolution algorithm using a Pareto tournament selection.
• Single-linkage [52]. This is one of several methods of hierarchical clustering. It is based on grouping clusters in bottom-up fashion (agglomerative clustering), at each step combining two clusters that contain the closest pair of elements not yet belonging to the same cluster as each other.
• Electrical Pattern Ant Colony Clustering (EPACC) [53]. This, meta-heuristic based algorithm uses the ant colony process to guide the clustering process, using the clusters' centroids as basis for the clustering process. Notice that all these algorithms, except from EPACC, require the number of clusters K as input. For this reason, the Calinski Harabasz criteria (CH) is used to automatically identify the best K for these algorithms. CH is also known as variance ratio criterion and evaluates the quality of the grouping based on the mean variance between cluster sum of squares and within cluster sum of squares [54].
The search process to find the best choices in terms of USW' and ESW' is performed using a GA approach based on [44]. This method does not guarantee finding the optimal solution, but it enables reaching near-optimal solutions in a reasonable execution time and computational resources.
GA is a highly parallel search and optimization technique within computational intelligence. It is inspired in the principles of Darwinian where the natural selection and genetic reproduction are natural processes [55]. GA replicates these principles by considering a population of initial solutions, which are used to start the search for new and better solutions. Evolution usually starts from a randomly created set of solutions and is run across multiple generations. In each generation the population is evaluated, determining the most fit and unfit individuals. Some individuals are selected for the next generation and then are combined with individuals previously selected or mutated to form a new population. This new population is the input to the following iteration of the algorithm.
After performing the parameter tuning, the GA parameters used in this study include a population size of 50 individuals, a crossover rate of 0.8 and a mutation rate of 0.1. The number of generations is defined as dynamic depending on the execution time of the algorithm. Hence, a maximum execution time of 2 hours is set as stopping criterion for the search process, regardless of the number of generations.

2) RESULTS
All the simulations were performed and implemented in MATLAB 2014a 64-bit using a computer with 1 processor Intel Xeon E5-1650 3.20 GHz with 6 cores, 10 GB of random access memory, and Windows 10 Professional 64-bit operating system. Table 10 presents the results achieved by the different methods, in a total of 1000 executions, considering x 1 = x 2 = x 3 = 0.33 in the calculation of Q n . For each method, the average number of coalitions, and the mean, maximum and Standard Deviation (SD) for USW' and ESW' are presented.
From Table 10 one can see that the best solution for both USW' and ESW' is found by the proposed method. In fact, the mean and maximum values of USW' and ESW' are more than twice the values found by any of the reference clustering algorithms. This can be easily explained by the different nature of the methods. While the proposed model aims to find the choices that maximize the social welfare of the players; the clustering approaches are focused on finding coalitions between players that are the most similar as possible to each other. This causes the selected outcomes to be composed of between 3 and 9 coalitions, which refer to the separation of players by type (total of 5, as shown in Table 9); by zone (total of 6, as shown by Fig. 2), or other combinations between the players characteristics. This type of separation does not contribute to maximizing any of the components of u . On the other hand, the outcome that maximizes USW' and ESW' is the grand coalition, as shown by the results of the proposed method, in which the average number of coalitions found in the best solution is near 1. This result averages the price between all players (I n component). As for Q n the grand coalition minimizes the total number of coalitions and reaches a fair result in terms of players' diversity. The relative volume of flexibility of each player is minimized, but unless the number of coalition comes close to the number of players, this value will always be very low and thus has low impact on differentiating the quality of the solutions.
The clustering methods that reach the best results are those based on meta-heuristic methods, including a significant random component, namely ADE-LPC and EPACC. Since the random component has a higher influence than that of the other clustering methods, these methods are able to, occasionally, find better solutions in terms of social welfare. This also causes the SD of the identified solutions to be slightly larger than in the other clustering methods. The larger SD is, however, that of the proposed model, since the objective is to evaluate the search space of possible choices using GA. Due to the large search space, the process has significant stochasticity, and it is only natural that the SD is higher than that of the other methods, whose objective is to minimize the diversity between clusters: a much lighter process.
Similarly to the case presented in section B, we will now analyse the results for different combinations of the parameters that are used to calculate Q n . Table 11 presents the best result achieved by each method, in a total of 1000 executions, considering a larger influence of the relative flexibility volume of the player according to all the players present in the same coalition, when compared to the other components of Q n , namely: x 1 = 0.7; x 2 = x 3 = 0.15.  Table 11 shows that the best choice, from the social welfare perspective, for x 1 = 0.7; x 2 = x 3 = 0.15 is found when each player is in its own coalition, thus maximizing the relative flexibility volume of the player according to all the players present in the same coalition. It is noticeable from Table 11 that the best found solutions among all clustering methods are very similar to each other. Since the evaluation of the solutions in the clustering process does not consider the Q n parameters, the solutions do not differ from those found for x 1 = x 2 = x 3 = 0.33, in Table 10. Moreover, it is interesting to notice that, although the best choice is found when the total number of coalitions is maximized, the solutions found by the clustering methods that include a larger number of clusters, e.g. 9 fond by K-Means, do not represent an increase in the value of social welfare when compared to solutions with a smaller number of coalitions, namely 5 as identified by the Hierarchical clustering, Fuzzy C-means, Single-linkage and EPACC. This occurs because, although there is a small increase in the relative volume of flexibility of each player, the difference is just too small to cause a significant impact and compensate for the decrease in the diversity and total number of coalitions components, which are much more affected by this increase in the number of coalitions. Table 12 presents the best result achieved by each method, in a total of 1000 executions, when considering a larger influence of the total number of coalitions, when compared to the other components of Q n , namely: When the parameter with the larger influence on the calculation of Q n is the total number of coalitions, the best identified choice is the grand coalition, as shown by Table 12. In this case, the best solutions found by the clustering algorithms are the solutions that result in the smallest number of clusters. In particular, ADE-LPC finds a solution with 3 coalitions, which represents the best solution found by all clustering methods. Even so, this result is still too far from the best outcome identified by the proposed model. Table 13 presents the best result achieved by each method, in a total of 1000 executions, when the larger weight in Q n is attributed to the diversity of the players within the coalition, namely: x 1 = x 2 = 0.15; x 3 = 0.7. Table 13 shows that when the diversity of the players within the coalition is the most relevant factor for the calculation of Q n , the best choice for all players is a solution with 82 coalitions. In this case, the 82 coalitions refer to the maximum separation that may exist between the players, with 1 MC in each group, 1 LC in almost each group, etc. The grand coalition, with all players in the same cluster, decreases dispersion because there are many similar players. In the case of the clustering methods, as the grouping is done according to the similarity, the higher the number of coalitions, the smaller the dispersion within each cluster, hence the best result is obtained when the number of clusters is smaller.
ADE-LPC ends up getting better results in almost all cases, among the clustering methods, because it is the one with the highest SD, so sometimes it finds some solutions that are farthest apart.
In order to compare the methods' results under the most similar conditions as possible, the following results consider all methods' performance with the number of coalitions fixed as 3. This is the only of the Q n components that is possible to replicate with the clustering algorithms, since the clustering algorithms own nature is contrary to maximizing the players' diversity in each coalition; and the relative volume of flexibility is something specific of each player, which is a nonexisting concept in clustering, which evaluates only the global solution. Table 14 presents the best result achieved by each methods, in a total of 1000 executions, considering x 1 = x 2 = x 3 = 0.33 in the calculation of Q n, for a fixed number of 3 coalitions (i.e. K = 3 in clustering algorithms).  Table 14 shows that, for a pre-specified number of 3 coalitions, the results among all methods are more balanced than in previous cases. In this case, the value for the total number of coalitions component of Q n is the same among all methods. However, the players' coalition structure identified by the proposed model find solutions that increase the diversity between players in the same coalition, while the solutions found by the clustering methods do exactly the opposite. This is the main factor that causes the difference between the quality of solutions (social welfare) of the clustering methods and the proposed model. Additionally, the outcomes identified by the proposed method also take into account the maximization of the average price and the relative volume of flexibility. Even if slightly, the difference in the values of all of these components together cause the proposed model to find solutions that are significantly better in social welfare than those achieved by the clustering methods. It is noteworthy that the Hierarchical clustering, Single-linkage and EPACC find exactly the same solution when the total number of clusters is set as 3, which is due to similar objectives of the different clustering algorithms. Table 15 presents a summary of the execution times the different algorithms take to reach solutions in this study. From Table 15 it is visible that the faster algorithms to reach solutions are the meta-heuristic based clustering algorithms, namely EPACC and ADE-LPC. The stochastic nature of these methods enable a faster search process, although at the cost of a higher SD, both regarding execution times and the achieved results, as shown in Table 10. The simple Hierarchical clustering and Single-linkage methods solve the clustering problem with a time complexity of O(n 3 ), where n is the number of entities to be clustered, which makes them too heavy for large data-sets. Hence, the solutions found are not guaranteed to be the optimal solutions, as the clustering process stops before reaching the optimal solution. This is evidenced by the results shown in the previous tables. However, these are still faster to execute than the K-Means and Fuzzy C-means algorithms. The clustering problem with K-Means is NP-hard for a general number of clusters K . If K and d (the dimension) are fixed, the problem can be solved in time O(n dk+1 ). The execution time is hence higher than that required by the Hierarchical clustering approaches. The fuzzy dimension added by the Fuzzy C-means further aggravates the execution time and the associated SD. Due to the higher problem complexity dealt with by the proposed model, as discussed in section A, the proposed model needs additional time to reach suitable solutions. The stopping criterion of 2 hours that is imposed results in a fixed execution time for this algorithm. As shown by the previous results, this time limit is enough to guarantee an acceptable quality of results for this problem. Nevertheless, this time limit may be increased (or reduced) in order to meet the requirements of new problems with different characteristics.
In summary, clustering is a good approach from the aggregator/operator perspective, since the groups/coalitions are formed according to the similarity of the players' characteristics, it divides players according to their types, prices, localization, etc. Therefore, it is easier for the aggregator to manage these groups separately, and apply specific measures to each. However, it completely neglects the players' individual perspective, and ignores the individual expected benefit; hence it becomes unrealistic if players are able to choose and change their coalitions (it is only realistic if these groups are imposed by the central management entity). On the other hand, the proposed method considers the individual perspective and benefit (utility) of each player independently, therefore resulting in a more realistic outcome, assuming that players have their own rational behaviour, and their own objectives and goals. It is, therefore a more robust approach for aggregators/operators to foresee the expected outcomes from the coalition formation process; while at the same time representing solutions that benefit both the system and the individual players.

IV. CONCLUSION
Exploring the potential of energy consumption flexibility is of upmost importance to enable the energy sector dealing with the variability from renewable-based generation. The transaction of consumption flexibility is facilitated by aggregators, which combine the flexibility from multiple consumers and trade it in the energy market. Current models for aggregation and coalition formation in the power systems domain are, however, not adequate to enhance the potential from the consumers' side, as they neglect the perspective from the consumer, and focus exclusively on the needs and objectives from the system and from the aggregator.
This paper proposes a model for consumers' flexibility aggregation, based on coalitional game theory. The proposed model is formalized as a partition function game with non-transferable utility. Utility functions are defined to enable measuring the preferences of the players and facilitate the assessment of the results. Two case studies are presented, including a simple illustrative case, in which the model is assessed and explained in detail. The subsequent case study considers a large-scale scenario based on real data, comprising more than 20,000 consumers. Results show that the proposed model is able to reach solutions that are more suitable for the consumers when compared to the solutions achieved by traditional aggregation techniques in power and energy systems, such as clustering-based methodologies. The solutions found by the proposed model consider the perspectives from all players involved in the game and thus are able to reflect the rational behaviour of the involved players and reach balanced solutions that meet the goals and preferences of all involved players and from the system itself, rather than imposing an aggregation solution that is only beneficial from the aggregator perspective. In this way, the achieved solutions are beneficial for individuals when compared to system-centric aggregation models, while also being a relevant model form the aggregator perspective, in order to identify coalitions that are expected to be formed when consumers are able to decide which players to aggregate with. Hence, by disengaging from the traditional system-centric decision models, adopting the proposed model as coalition formation support leads to an improvement of individual players' results at the cost of a possible decrease of the aggregator profits. The balance is, nevertheless, guaranteed by considering the maximization of the overall social welfare, from the perspective of the individual player, the coalition, and the entire system. Limitations of this model include the reduced number of players' characteristics considered in this study, rather it has been opted to address the scalability of the model regarding the increase of the number of players. The dynamics of the market itself and their impact in the achieved prices and actual traded flexibility volume are not considered, since the proposed model focuses on the coalition process itself, leaving the market participation responsibility for the aggregator.
As future work, this model will be extended in order to include different perspectives on the objectives and preferences of the involved players. The notion of fairness will also be explored, namely by assessing solution concepts including the Shapley value and Banzhaf index. The stability and equilibrium of both the global and coalitional perspectives will also be explored, including the individual stability, contractual individual stability and core stability. Moreover, machine learning models will be considered to analyse previous behaviour of players in order to reach expected actions that may be incorporated in the coalition formation process. Finally, distributed approaches for solving the problem will be explored, as means to mitigate the problem related to high computational burden, especially when addressing large scale scenarios. NOMENCLATURE sσ C g Average scaled standard deviation in coalition C for all characteristics g u n (λ) Utility function u n (λ) Scaled utility function µ C g Mean value for each characteristic g in coalition C σ C g Mean deviation of each value of characteristic g from each player in coalition C v Characteristic function Set of choices V n Flexibility volume of player n x 1.. 3 Scaling His research interests include multi-agent systems, and in particular, in the computational theory aspects of rational action in systems composed of multiple self-interested agents. His work is characterised by the use of techniques from computational logic, game theory, and social choice theory. He is also a Fellow of the Association for the Advancement of Artificial Intelligence (AAAI), the European Coordinating Committee for Artificial Intelligence (ECCAI), the Society for the Study of Artificial Intelligence and the Simulation of Behaviour (AISB), and the British Computer Society (BCS). In 2015, he was made the Association for Computing Machinery (ACM) Fellow for his contributions to multi-agent systems and the formalisation of rational action in multi-agent environments.
ZITA VALE (Senior Member, IEEE) received the Ph.D. degree in electrical and computer engineering from the University of Porto, Porto, Portugal, in 1993. She is currently a Professor with the Polytechnic Institute of Porto, Porto. Her research interests include artificial intelligence applications, smart grids, electricity markets, demand response, electric vehicles, and renewable energy sources. VOLUME 9, 2021