Capacity Aggregation and Online Control of Clustered Energy Storage Units

With the growing penetration of renewable energy and gradual retirement of thermal generators, energy storage is expected to provide flexibility and regulation services in future power systems. Battery is a major form of energy storage at the demand side. To better exploit the flexibility potential of massive distributed battery energy storage units, they can be aggregated and thus get enough capacity to participate in auxiliary service markets or receive direct orders from the power system operator. This paper proposes an analytical method to determine the aggregate MW-MWh capacity of clustered energy storage units controlled by an aggregator. Upon receiving the gross dispatch order, a capacity-aware water-filling policy is developed to allocate the dispatched power among individual energy storage units, which is called disaggregation. The policy endeavors to track the dispatch order while reducing the difference among the storage level of each battery. It provides effective real-time power control strategies for a particular class of energy management problem without referring to the prediction of dispatch order, although storage operation must respect inter-temporal constraints. Case studies validate the proposed method through applications in wind power ramping alleviation and frequency regulation.


Ω ag
Feasible set of actions for the aggregator.

I. INTRODUCTION
T HE power grid is undergoing the transition to a renewable dominated system, with the proliferation of wind and solar plants.The volatility of renewable energy requires more flexible resources to maintain power balance.However, controllable thermal generators are gradually replaced by renewable plants to achieve carbon neutrality, leading to a lack of adequate flexibility resources.Deploying energy storage unit is an effective way to support renewable energy integration.Existing works focus on the coordinated operation of large centralized energy storage units (ESU) with grid-side facilities, such as in unit commitment [1], [2], [3] and economic dispatch [4], [5] problems.
Considering the relatively short lifetime of battery, deployment of large centralized ESU is an expensive option.At the demand side, various consumers own small ESUs, but it is difficult for the power grid operator to dispatch numerous distributed ESUs directly.Hence, aggregation of distributed ESUs offers a hierarchical framework that allows distributed storage resources to act together as a large centralized storage unit [6], [7].Under some rental agreements, storage owners would like to share some capacity of their ESU, and an aggregator who manages the storage cluster can obtain adequately large capacity to enter auxiliary service markets or receive direct orders from the grid operator [8], [9], [10].In such a hierarchy architecture, the aggregator submits the aggregate power and energy capacities as well as the storage level to the system operator, which is termed aggregation; then from the power grid side, the aggregator is a large ESU.When the aggregator receives a charging or discharging order, it must allocate the dispatched power among small ESUs, which is called disaggregation.Existing works focus on either of the two aspects.The literatures are summarized in Tables I and II.
1) The Aggregation Problem: Many works have investigated the flexibility aggregation problem [11], [12], [13], and existing approaches could be roughly categorized into two classes.
The first one aims to find an envelope of the aggregate power profile, such that for any trajectory in this envelope, there exists a disaggregation control strategy that allocates the aggregate power among each device while retaining their operation feasibility.In [14], robust model predictive control (MPC) is adopted to declare the aggregate power reserve flexibility of flexible load in a rolling horizon manner.In [15] and [16], the lower and upper bounds of the aggregate power over multiple periods are calculated from a linear or quadratic program; it is proven that any power trajectory residing in the envelope is implementable via a non-anticipative quantile policy.To reduce the conservativeness of hypercube approximation, an ellipsoidal approximation method is proposed in [17].In [18], distributed energy resources (DERs) are classified as generator-like and battery-like devices; then a polytope-based bound shrinking approximation method is proposed to further improve accuracy.The outcome is a feasible interval of total power in each period.Such aggregation method is suitable for power aggregation of integrated flexible load, renewable and storage-like device.For the aggregation involving only energy storage devices, the aggregator can be characterized by two parameters (power capacity and energy capacity) and a real-time state (storage level or state-of-charge, SoC for short), which is called the Energy-Power-SoC model.In this work, we want to obtain the Energy-Power-SoC triple.However, the envelope-based methods do not offer the desired information.
The second one is based on polyhedral set operation, mainly Minkowski sum.From a geometric perspective, the feasible operating set of each device can be described via a polyhedral set.The flexibility aggregation can be viewed as calculating the Minkowski sum of all these polyhedral sets.However, this set lacks intuitive interpretation.Given the difficulty of computing the exact Minkowski sum under facet representation [19], an alternative way is to build inner/outer approximations for aggregated power feasible region.For the inner approximation, the key is to find a simple but feature-capturing prototype set to conservatively approximate the exact Minkowski sum polytope.
In [20], [21], [22], virtual battery model is first adopted to describe the aggregate flexibility of thermal controllable loads or electric vehicles; then polyhedral projection is employed to project the augmented polytope to a low-dimensional space and obtain the aggregate power feasible range.In [23], optimal zonotope is calculated to inner-approximate individual polyhedral feasible set; since zonotope is closed under Minkowski sum, the aggregate flexibility can be calculated efficiently by direct summation.The selection of prototype set is vital because an improper prototype will lead to conservative results and loss of flexibility.In [24], an outer approximation of Minkowski sum of polytopes is developed to represent the aggregate capabilities of heterogeneous flexible loads via homothetic polyhedra; one step further, the outer approximation is extended from [24] to convex conic sets with second-order cone and semidefinite constraints in [25].But the outer approximation may appear to be optimistic and cause infeasibility in disaggregation stage.The above aggregation methods mainly focus on determining an aggregate power feasible interval/polytope for flexible resources, but the aggregation technique for ESUs remains an open problem.The power flexibility of individual ESU is a polytope; thus we can use Minkowski sum method to model the aggregate flexibility of the ESU cluster.Considering operation security, inner approximation is suitable in representing aggregate flexibility.The Energy-Power-SoC model is a natural choice to describe the aggregated flexibility of clustered ESUs.The problem is how to determine the aggregated capacity parameters.
For ESUs aggregation, the aggregated power and energy capacities are acquired by directly summing all individual capacities in [26], but these individual ESUs have the same rated charging time; otherwise, direct summation is not applicable.In addition, the aggregated power and energy capacities should be acquired while the operating feasibility of individual ESUs must be guaranteed, which is affected by how individual ESUs are controlled (i.e., the disaggregation policy).
2) The Disaggregation Problem: When the aggregator receives a charging/discharging order, it needs to decide the real-time power control strategy of each ESU, which affects the flexibility in future periods due to the inter-temporal storage level dynamics.However, the future dispatch signal is not known in advance.If the signal is not properly disaggregated, some ESUs may be fully charged or depleted earlier than others, thus reducing the aggregate flexibility in the next period.The most fundamental requirement of disaggregation policy is non-anticipativity, which means no information on future dispatch signal is needed, so the causality of control actions is respected.Existing works on power disaggregation can be classified into two categories based on their topics: Disaggregation feasibility and ESUs management.The first category focuses on the existence of a disaggregation strategy.In [16], disaggregation is implemented by a non-anticipative quantile policy.Soft robust optimization is used to ensure the existence of disaggregate action even in the worst-case scenario.Adaptive robust optimization is adopted in [17], where an affine policy is proposed to ensure the existence of feasible disaggregation strategy.A power disaggregation policy can be obtained by solving a cost minimization problem [23], tracking contracted power with rolling horizon stochastic optimization [27], or following a pre-determined open-loop control policy [28].But the above literatures only focus on the internal uncertainty originated from its participants, while the external uncertainty from dispatch signal is neglected.The second category is for the management of multiple ESUs, which is closer to the power disaggregation problem, and SoC consistency is a widely adopted philosophy, such as cooperative hierarchical control [26], leader-follower consensus control [29], droop control [30] and consensus-based control [31].These methods focus on the control scheme design and dynamic response performance; but the uncertainty of control signal received by the aggregator is not considered.In practice, the dispatch signal is received period-by-period, and the aggregator needs to disaggregate the current dispatch signal and control individual ESUs without relying on predicting future dispatch signals.An improper power disaggregation strategy will reduce the flexibility of ESUs cluster.Model predictive control [32] and priority control [33] are also mature techniques, but the former relies on high-quality prediction and the latter leads to over-use of some ESUs.
This paper addresses capacity aggregation and real-time disaggregation control of clustered energy storage in an integrated framework.The contributions include: 1) An analytical solution of capacity aggregation.We adopt the Energy-Power-SoC model to describe the aggregated ESU.In the state-of-the-art geometric approach based on Minkowski sum, approximation is needed and the aggregation problem is finally cast as an optimization problem after complicated reformulation process.Inspired by the SoC consistency idea, we discover a closed-form solution of the MW-MWh capacity of the ESU cluster without solving an optimization problem or calculating the Minkowski sum.The aggregation result is shown to be equivalent to the Minkowski sum based approach but significantly simplified.The analytical expression of aggregated capacity enables various online evaluation, decision and control tasks.For example, when small ESUs are allowed to freely enter or exit the aggregator, the MW-MWh capacity of the ESU cluster should be updated in real time.The proposed method is very convenient to show the aggregated capacity online, while other approaches entail substantial computation effort.It also offers insights on how to preserve flexibility as much as possible through the idea of SoC consistency.2) A capacity-aware water-filling (CAWF) policy for power disaggregation control.Inspired by the SoC consistency idea, the disaggregation control problem is cast as a convex optimization problem with a resource constraint and two-side bound constraints on charging/discharging power of each ESU.Compared to the standard water filling algorithm [34], where action is not upper bounded, the maximum charging-discharging power is imposed, which captures the practical requirements of storage operation.Compared with SoC consistent control methods, proposed method aims to track uncertain dispatch signal and preserve flexibility for future use.Compared to the water filling methods for distribution system Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.operation which renders a temporal scale energy allocation for load-flattening [35], [36], [37], the proposed method gives spatial-scale coordination for allocating dispatched power among distributed ESUs.Although ESU operation involves inter-temporal constraints on storage dynamics, the CAWF policy is prediction-free and can be applied in an online fashion.We further develop a closed-form of the CAWF policy, which can be implemented with simple hardware without an advanced computational platform.The rest of this paper is organized as follows: The aggregation and disaggregation problems are formally stated in Section II.The former one is addressed in Section III, in which the aggregated capacity of the ESU cluster is analytically characterized; Section IV resolves the disaggregation problem and develops the CAWF.The holistic framework is presented in Section V. Case study is reported in Section VI; the proposed method is validated through applications in wind power ramping alleviation and frequency regulation.Finally, Section VII concludes the paper.

A. System Configuration
We consider an aggregator that operates a cluster of ESUs indexed by i ∈ N = {1, 2, . .., N}.Each ESU submits its power capacity P i , energy capacity E i and SoC s i = e i /E i to the aggregator.The power grid dispatches the aggregator as a large ESU whose power and energy capacities are P ag and E ag , respectively.After receiving the dispatch signal, the aggregator needs to disaggregate the power signal among small ESUs.The hierarchical structure is shown in Fig. 1.
In this business model, we assume the aggregator has designed an effective mechanism to incentivize the ESU owners to share their surplus/unused capacity with the aggregator.A simple mechanism is that the aggregator pays rental fee to the ESUs owners.The rental fee may include the opportunity cost and degradation cost.The mechanism design is related to storage sharing and is not the main scope of this work.With a fixed number of small ESUs and rental fees, the aggregator would focus on the interaction with the power system.It has to submit the aggregated MW-MWh capacity and control small ESUs to fulfill the dispatch order.The former is determined in the aggregation problem and the latter is addressed in the disaggregation problem.

B. Capacity Aggregation
Because it is difficult for the power grid operator to manage massive small ESUs individually, the aggregator reduces the complexity by acting as a centralized ESU and submitting aggregated power capacity P ag and energy capacity E ag to the grid operator.The basic requirement for aggregation is that whenever the aggregated ESU is not fully charged or empty, it is responsible to respond to a charge or discharge order no greater than P ag .
By disclosing the power capacity P i and energy capacity E i , the operation of i-th ESU is constrained by where η c i /η d i denotes the charging/discharging efficiency; Δ t is the duration of period t; e i,t represents the storage level; control action p i,t = {p c i,t , p d i,t } includes charging/discharging power p c i,t /p d i,t .Constraint (1a) restricts charging/discharging power within the power capacity.Inter-temporal constraint (1b) describes the dynamics of storage level which must reside in the interval (1c).Expressing e i,t through p i,t , constraints (1b) and (1c) reduce to The feasible set of control action is denoted as a polyhedron depending on the values of P i and E i .It characterizes the individual flexibility of each ESU.As a large ESU, the power grid only perceives the total power and energy of the aggregator, i.e.
where p ag,t = {p c ag,t , p d ag,t }.By the definition of Minkowski sum, the feasible set of aggregate power profile is where symbol ⊕ denotes the Minkowski sum.However, it is complicated to compute the hyperplane representation of Ω sum , and the number of constraints in Ω sum increases exponentially with a large number of ESUs and time periods.Hence, it is less attractive to compute the exact expression of set Ω sum in engineering practice, although it is interesting and important from a theoretical perspective.In this paper, the ESU cluster is treated as a large ESU with aggregated power capacity P ag and energy capacity E ag .Its operation is constrained by Similar to (2), the feasible set characterized by (P ag , E ag ) is Although polyhedra Ω sum and Ω ag (P ag , E ag ) share the same dimension, they can be different in shape.To ensure the existence of an operation strategy for individual ESUs satisfying constraint (1), the condition Ω ag (P ag , E ag ) ⊆ Ω sum (7) must hold.The capacity aggregation problem aims to find the optimal tuple (P ag , E ag ) such that Ω ag (P ag , E ag ) is the largest among all eligible candidates satisfying (7).Clearly, the optimal (P ag , E ag ) depends on (P As Ω ag is a subset of Ω sum , dispatching the aggregated ESU is less flexible than dispatching all small ESUs individually.We propose an index LoF to quantify the loss of flexibility during the capacity aggregation process: where vol(Ω) is the volume of a full dimensional polytope Ω; D is its dimension.The D (•) operator normalizes the volume ratio into dimension-1.
Although the aggregation process sacrifices some flexibility, it significantly simplifies the information exchange between the aggregator and grid operator.Since the aggregator submits aggregated information to the operator, from the power grid aspect, the ESU cluster becomes a single ESU, and hence existing results on dispatching centralized ESUs can be applied.
The loss of flexibility is a trade-off between conservativeness and security.On the one hand, the aggregator hopes P ag is as large as possible to provide more flexibility; on the other hand, the aggregation method does not provide an explicit rule to control individual storage unit under which the desired ability (to respond to a charge or discharge order no greater than P ag ) is warranted, which inspires the power disaggregation control problem.

C. Power Disaggregation Control
In each period t, the grid operator sends either a charging order D c t or a discharging order D d t to the aggregator, where The aggregator controls each ESU to track the dispatch order.Given the sequence {D c t , D d t } T t=1 of dispatch orders over a certain period, the disaggregation control problem is formulated as follows: where the objective function (9a) aims to minimize the total deviation from the dispatch order.Upon receiving the dispatch signal, the aggregator has to respond by controlling small ESUs.
Because the capacity is limited, the aggregator may not be able to completely fulfill the dispatch order, and also has no incentive for an overreaction, which incurs no benefit but consumes more flexibility potential.So we restrict the charging and discharging actions of aggregator within the intervals [0, D c t ] and [0, D d t ], as in (9c) Constraints (9d)-(9e) stipulate the status of each ESU according to the dispatch order, preventing simultaneous charging and discharging.The remaining constraints in (9b) respect the operational feasibility of individual ESUs.Problem ( 9) is able to model a variety of applications, such as wind power ramping alleviation and frequency regulation, which will be detailed in the case study.The power disaggregation problem is summarized below.

III. CAPACITY AGGREGATION
This section presents the analytical expression of aggregated capacity of the ESU cluster.First, the case with ideal ESUs is discussed, and then the result is extended to lossy ESUs.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

A. The Case With Ideal ESUs
We consider the case in which all ESUs are lossless, which means We also assume e ag,0 = 0, implying e i,0 = 0, ∀i.This is merely an assumption to derive the aggregated capacity that is independent of the initial SoC.The disaggregation policy allows different and non-zero initial SoCs, and the real value of e ag,0 can be easily obtained.
Clearly, the aggregated energy capacity should not exceed the total capacity of all ESUs in the cluster, that is Given the aggregated energy capacity E ag , the SoC of aggregate ESU is expressed as: where e i,t = s i,t E i .It can be seen that the SoC of aggregated ESU is the weighted average of individual SoCs.By condition (7) and the lossless assumption on ESUs, P ag solves the following optimization problem:  (7).
In the following, we will show how to acquire P ag analytically instead of solving problem (12).The insights lead to the development of disaggregated power control policy, in spite of the fact that solving problem (12) is not difficult.Let T i = E i /P i be the rated charging time of storage unit i, and Proposition 1: The aggregate power capacity can be explicitly expressed as (13) To see this, for any t = 1 : T , adding constraints (12c) together for i ∈ N and considering (12d) yields: Add constraint (12f) from t = 1 : Combining ( 14) and ( 15) we obtain The above inequality is tight, which means equality in ( 16) can be achieved.To see this, assume all ESUs are empty at the beginning.Because the aggregator wants to charge the aggregated ESU with a constant power that is as much as possible, it uses the SoC consistent policy: the ESU with largest rated charging time T i is charged at maximum power P i , and the charging power of ESU j, j = i is calculated by such that the SoC of all ESUs are kept at the same level.Under this policy, the maximum charging power is If a dispatch order exceeds P ag , the storage levels can no longer be maintained at the same level, because the i-th ESU already reaches the maximum charging power.If we allow SoC discrepancy, some ESUs will be fully charged earlier than the remaining ones.Once any ESU is fully charged, the aggregated charging power must be reduced.In this regard, the SoC of the aggregated ESU must follow the ESU with the largest E i /P i , which is equal to T ag .So P ag in (13) is optimal.

B. Extension to Non-Ideal Storage
Now consider non-ideal ESUs.When the aggregator receives a charging signal D c , to keep SoC levels of all ESUs at the same level, we have where represents the aggregated charging efficiency.A similar condition can be given for a discharging signal D d as follows: where represents the aggregated discharging efficiency.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

A. Intuitive Idea of Power Disaggregation
In the disaggregation control problem (9), the difficulty arises from the fact that in period t, the future dispatch order is unknown.An improper disaggregation strategy may either deplete or fully charge some ESUs earlier than others, thus reducing the response ability of aggregator in future periods.
An example is given to demonstrate the above phenomenon.Consider a problem with 2 periods and two ideal ESUs, whose parameters are listed as follows: In period t 1 , the aggregated charging power is 15 MW.We consider three disaggregation strategies: All three strategies are valid responses.At the beginning of period t 2 , the respective storage levels of the two ESUs are: Case 1 s 1,t 1 = 1.000, s 2,t 1 = 0.6250 Case 2 s 1,t 1 = 0.875, s 2,t 1 = 0.6875 Case 3 s 1,t 1 = 0.750, s 2,t 1 = 0.7500 Hence, the admissible charging power ranges under the three strategies are: In period t 2 , the power grid requests the aggregator continue to charge.The response ability of the aggregator in three cases is analyzed a) If the charging request is less than 10 MW, then the aggregator can respond to the request in all three cases; b) If the charging request is between 10 MW and 12.5 MW, then the aggregator is unable to completely follow the request in Case 1 and can still fulfill the charging request in Case 2 and Case 3; c) If the charging request is between 12.5 MW and 15 MW, then the aggregator is unable to completely follow the request in Case 1 and Case 2, while it can still fulfill the charging request in Case 3. In practice, when the aggregator makes decision in period t 1 , it has no information of the charging request in period t 2 .We can see that if a situation where strategy S1 is applied can be handled in period t 2 , then the situation where strategy (S2)/(S3) is applied can also be handled in period t 2 .So the aggregator is more flexible in Case 2 and Case 3 than it is in Case 1, and hence strategies S2 and S3 dominate strategy S1.For the same reason, strategy S3 dominates strategy S2.
The above observation inspires the SoC consistency principle.If we coordinate all ESUs and keep their storage levels as close as possible, and then the aggregate power flexibility will be reserved to a greater extent.Based on this observation, we develop a capacity-aware water-filling algorithm for disaggregated power control policy.

B. Capacity-Aware Water-Filling Algorithm
Water-filling algorithm is usually applied in communication system for allocating limited power over a set of communication channels, aimed at utilizing maximum channel capacity.In the disaggregated power control, considering the charging order, the capacity-aware water-filling problem is cast as: where constant α i corresponds to the storage level, and the dispatched power x i is normalized to be equal to 1, as in (19c); the capacities of small ESUs are taken into account in (19b).The detailed correspondence will be explained later.In the communication system, channel capacity is not a main concern, so x i is not upper bounded by xi ; see the example in page 245 [34].However, in energy storage control, charging power limit must be taken into account in (19b).In this regard, the proposed method is called capacity-aware water-filling algorithm.Although problem (19) remains convex, introducing upper bound constraints complicates the development of an explicit policy, compared to that in [34].
To solve problem (19) without calling a solver, its Karush-Kuhn-Tucker (KKT) optimality condition is analyzed.By introducing Lagrange multipliers λ l i and λ u i for lower and upper bound constraints in (19b) and a multiplier ν for equality (19c), we can obtain the following optimality condition By (20a), λ l i can be expressed using other variables and thus eliminated, then the optimality conditions give rise to Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
where 0 ≤ a ⊥ b ≥ 0 represents complementary slackness, requiring at least one of two non-negative quantities a and b to be zero.By condition (21), we have four cases: i is positive, it follows that x i = xi due to (21b).But λ u i + ν − (α i + xi ) −1 must be positive by assumption; consequently, complementary slackness constraint (21a) is violated.Thus λ u i = 0, and from (21a) we have ν = (α i + x i ) −1 , hence i must be positive.According to (21b), we must have x * i = xi .In summary, if we have the value of ν, then the optimal solution of problem ( 19) is For convenience, we write x * i as x * i (ν), which is a piecewiselinear increasing function in 1/ν.Substituting ( 22) into (21c), we have As each x * i (ν) is increasing in 1/ν, so is g(ν).Thus ( 23) has a unique root ν * , which can be found by Algorithm 1.
Similarly, considering the discharging order, a water-draining problem is established, which is where A is a unified upper boundary of capacity.The solution can be acquired in the same way as (19), which is The ν value is determined through (24c).The graphic interpretation is shown in Fig. 2.

C. Online Analytical Disaggregation Policy
Based on the closed-form solutions of problems ( 19) and ( 24), we develop a disaggregated policy for real-time power control, which can be implemented on microprocessor without an advanced computational platform.The relations between online power disaggregation problems ( 26)/( 27) and problems ( 19)/( 24) are summarized in Table III, where A = 1 is the unified upper bound.Based on the previous discussions, problems (26) and ( 27) can be solved by Algorithm 1.

V. HOLISTIC FRAMEWORK
The proposed method is schematically shown in Fig. 3.At the bottom layer, the individual ESU owners submit individual parameters to the aggregator; at the middle layer, the aggregator calculates parameters through (10), ( 11), ( 13), (17), and ( 18) and then submits them to the power system operator at the top layer.This constitutes the aggregation process.For the disaggregation control, the operator treats the aggregator as a large centralized ESU and sends real-time dispatch signal to the aggregator; then the aggregator allocates the dispatch command among small ESUs following the online disaggregation policy obtained from the solution of problems ( 26) and ( 27); finally, individual ESUs execute the control signal from the aggregator.

VI. CASE STUDY
Numerical simulation is conducted to verify the performance of the proposed method.The aggregation method is independent of the application scenario, while the disaggregation relies on the concrete implementation of problem (9), and is applied in wind power ramping alleviation and frequency regulation provision.Simulation is done on a laptop with Intel i5-1130G7 CPU and 16 GB RAM.The Minkowski sum is implemented by Multi-Parametric Toolbox [38].

A. Results of Capacity Aggregation
We test our analytical capacity aggregation method under different cases with various capacity parameter settings of massive distributed ESUs.Besides, the proposed analytical method is compared with Minkowski sum based approach [21] and direct summation method [26]   ESUs are fully charged or depleted, which violates the basic requirement for aggregation.The results of proposed method and Minkowski sum method are identical and smaller than P Σ ag , validating the analytical expression in (13).
It is also observed that aggregating all ESUs together leads to the largest LoF as high as 0.3591.In contrast, aggregating ESUs in each cluster can enjoy higher flexibility, reflected by smaller values of LoF ranging between 0.1 and 0.16.The combination of two clusters will get a medium level of flexibility.LoF largely depends on the difference among the ratios T i of individual ESUs.Therefore, the aggregator should cluster ESUs with similar T i values to avoid unnecessary loss of flexibility.
Define T min = min i {T i } and T max = max i {T i }.We choose 100 ESUs whose T i is a uniform distribution on the interval [T min , T max ].We increase T max /T min value from 1∼5 and record the corresponding LoF value.The simulation result is plotted in Fig. 4. It can be found that the LoF grows with the increase of T max /T min .To this end, a recommended principle is where γ is a threshold value.A higher value of γ will include a larger number of ESUs in a cluster and incur a higher LoF.In Fig. 4, when γ = 1.5, the LoF has an acceptable value of 0.1608.So we recommend γ = 1.5.Based on the above criterion, all ESUs can be categorized into several groups.The group with a smaller average value of T i can be used to provide frequency support; the group with a larger average value of T i can be used for energy arbitrage.

B. Application in Wind Power Ramping Alleviation 1) Wind Power Ramping Event:
Large fluctuations of wind energy output within a short period, known as wind power ramping event, would threaten the secure operation of power system due to the lack of ramping capacity.A wind ramping event happens when the change of wind output during a certain time period exceeds a threshold.According to the definition in [39], we define upward/downward ramping events as follows: where w t denotes the wind power output in period t; w up t /w dn t represents the upward/downward ramping value; W up /W dn represents the upward/downward ramping threshold.In this test, the duration of each time period is Δ t = 15 minutes.W up /W dn is set as 10% of the installed wind capacity.
Traditionally, the penetration of renewable energy in power system is moderate, and there are plenty of thermal generation units which are controllable.The volatility of renewable output is compensated by re-dispatching thermal generators.If the penetration of wind power reaches a very high level, thermal generation capacity is replaced by renewable plants and ramping capacity from thermal units becomes insufficient.In an upward ramping event, the excessive wind power may be curtailed; in a downward ramping event, some load may be shed.We assume the power system operator could dispatch an ESU aggregator to alleviate wind ramping event, and the dispatch order is: In the upward (downward) scenario, the operator sends charging (discharging) signal D c t (D d t ) to the aggregator.Once receiving D c t /D d t , the aggregator allocates the required power into each ESU.The goal of the aggregator is to track D c t /D d t as close as possible across all periods {1, . . ., T }.So the wind ramping event alleviation problem can be formulated as problem (9) in Section II-C.
In fact, the power grid does not have to completely absorb wind power variation, which means the dispatch signal D c t could have a smaller value than w up t .Equation ( 29) is just used for testing performance.The real challenge in practice is that in period t, the aggregator has no information on the future dispatch signals D c t+1 , D c t+2 , . ... The proposed CAWF policy is applied to this problem.
2) Simulation Results: We collect real data of a wind farm in Inner Mongolia, China, whose installed capacity is 100 MW.The time series of wind power output was recorded every 15 minutes.The time series over 3 days with 288 points is shown in Fig. 5. High magnitude upward / downward ramping events are observed.Based on the particular definition of wind power ramping event and the compensation rule in (29), the dispatch order sequence is also given in Fig. 5.
To alleviate the impact of wind power ramping, we assume the aggregator manages group A storage, which consists of type {1, 2, 3, 4} ESUs in  capacity parameters are P ag = 17 MW, E ag = 51 MWh, while P Σ ag = 19 MW.In this case, the dispatch order given by ( 29) may be greater than P ag , and should be tracked to handle wind ramping event.Particularly, we assume that all ESUs are half charged with an identical storage level of 0.5.For example, if the initial storage levels differ significantly, we can charge some batteries using remaining ones, which is technically viable.Nevertheless, this is not a practical limitation as the initial value is just a parameter in problem (9); any initial value can be used.In the next case, we will assume different initial storage levels.
We compare the performance of the following four policies in alleviating wind power ramping event: 1) Offline (hindsight) policy: Provided with the entire sequence of dispatch orders, problem (9) gives the optimal offline solution, which offers a hindsight policy.The hindsight optimum is an ideal result, which serves as a benchmark for comparison but cannot be implemented in practice.
3) MPC policy: We investigate the performance of MPC under different look-ahead time windows ranging from 4-16 time slots.4) Priority control policy: An intuitive online control policy for multiple ESUs is to charge / discharge them one by one.In the priority control policy, the ESUs are dispatched according to the following rule: The ESU with the highest efficiency will be charged / discharged with the highest priority unless its storage limit is reached.For ESUs with the same efficiency, those with a larger energy capacity will be dispatched first, until the dispatch order is satisfied.
A clear disadvantage of this policy is that the utilization rate of each ESU is different.The ESU with a large energy capacity and high efficiency will be dispatched much more frequently compared to the small and inefficient ones.From a long-term perspective, such a policy may harm the health of some high-performance ESUs.However, this long-term effect is not considered in this test.We refer an optimality gap is the difference in the optima between the hindsight policy and a particular policy.The optimality gap clearly shows how the results corresponding to a given policy deviate from the ideal optimum.Monte Carlo (MC) simulations are conducted under different control policies.We randomly choose 100 time series for the tests, each consisting of 288 points with Δ t = 15 minutes.The MC simulation results are summarized in Table VIII.
The hindsight optimum gives the benchmark of optimality.The average optimality gap of the proposed CAWF policy is only 0.2524% and performs much better than the priority control with 0.5358%.The MPC method is also investigated under different look-ahead time windows.Clearly, the optimality gap decreases when the length of prediction window grows.However, even with exact wind power forecasts in the next 3 hours (12 time periods), MPC is still worse than CAWF in terms of optimality gap.To get a better performance, MPC needs at least exact forecasts in the next 4 hours (16 periods).Such a requirement is still a challenging task even for ultra-short-term wind power forecast, because prediction error can be reduced only if special spatial and temporal correlation patterns in a large area are utilized in the forecast method.
The optimality gaps of the proposed CAWF policy in 100 simulation scenarios are shown in Fig. 6.The maximum optimality gap is only 0.5574%.From Fig. 6, the gap is less than 0.2% (0.4%) in approximately 71% (95%) of the simulated scenarios.Given the 0.2524% average value and distribution Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply. of the optimality gap, the CAWF policy is competitive and attractive in practical applications because it is prediction-free.
We randomly select a sequence in the MC simulation.In Fig. 7, the aggregated charging / discharging power trajectories (in one day) are plotted.As the ramping power may exceed P ag , the aggregator responds to the dispatch order with its maximum power, which is 19 MW, during periods 10 ∼ 18.The hindsight policy obtains the optimal power profiles for coping with this ramping event.The aggregate response trajectories under 4 control policies differ in some periods.Storage levels of each type of ESU under hindsight and CAWF policies are shown in Fig. 8. Compared with the hindsight result which can utilize full future information, CAWF policy prioritizes SoC consistency, but this does not mean that SoCs must be the same in all periods, especially when the dispatch order exceeds P ag and initial SoCs exhibit significant difference.For example, in period 10, the SoCs are different because D c 10 > P ag , and keeping SoCs identical will jeopardize the response capability.
We further increase the number and difference of ESUs.We choose 200 ESUs whose T i obey a uniform distribution in the interval [2,3]; the power capacities uniformly range from [0.05,0.5]MW; initial SoCs follow a uniform distribution in [0.1,0.9].The aggregated capacities are P ag = 48.36MW and E ag = 145.08MWh.We randomly choose 100 time series for the MC simulation, each consisting of 288 points with Δ t = 15 minutes.The optimality gaps of the proposed CAWF policy in 100 simulation scenarios are shown in Fig. 9.The average optimality gap is 0.2619%; the maximum optimality gap is 0.7293%; the gap is less than 0.3% (0.5%) in approximately 61% (93%) of the simulated scenarios.Given the 0.2619% average value and distribution of the optimality gap, the CAWF policy is still performing well in large-scale application.

C. Application in Frequency Regulation
Energy storage is expected to provide frequency regulation service because, in future power system, thermal generation capacity will be limited.We assume that the aggregator who  manages a cluster of small ESUs has adequate capacity to enter the frequency regulation market.
1) PJM Market for Frequency Regulation Service: In PJM performance-based regulation mechanism [40], the total revenue R reg of frequency regulation service consists of a capacity revenue R cap and a performance revenue R perf , which can be expressed as follows: where P r is the contracted regulation power capacity with the system operator; λ cap and λ perf are capacity clearing price and performance clearing price, respectively.Due to the relatively small scale, we assume that the aggregator is a price taker; M is mileage ratio between the mileage of the dynamic regulation signal (RegD) and the traditional regulation signal (RegA), whose value is approximately 3 and fixed for each hour [41]; K perf is the performance score during a certain period, which is calculated as the relative error between the assigned AGC signal P r r and actual response power p [42]: where the normalized AGC signal r = {r t ∈ [−1, 1]}, and P r r − p 1 represents the absolute error summation of the response.r is the average value of AGC signal based on the historical signal.Finally, the R reg can be expressed as We assume the aggregator submits the regulation capacity P r = P ag .Then, maximizing R reg comes down to maximizing the performance score K perf , in other words, minimizing the relative error P r r − p 1 .
In real-time control, the operator sends AGC signal P r r t every 2 seconds to the aggregator, which can be rewritten as the dispatch order D c t /D d t .
The aggregator responds to the dispatch order with p t = p ag,t accordingly.Thus the relative error minimization problem can be transformed into optimization problem (9) in Section II-C.
From the perspective of the aggregator, future AGC signals are uncertain and unpredictable.Once the aggregator receives a dispatch signal, it must determine the power control strategy of each small ESU without future information.Thus CAWF policy is applied to this problem.
2) Simulation Results: The AGC signals are collected from the PJM market with a time resolution of 2 seconds.The RegD signal requires net zero energy over a certain time horizon.In the following tests, the time span of numerical simulation is 2 hours.The aggregator manages type {5, 6, 7} ESUs in Table VI.As listed in Table VII, the aggregator submits P r = P ag = 10 MW to the market, and the total energy capacity is E ag = 7.5 MWh.According to the rule of frequency regulation, r t ∈ [−1, 1], so the dispatch order can never exceed P ag .
We compare the performance of following four policies: 1) Offline (hindsight) policy.2) The proposed CAWF policy.
3) 1-minute exact look-ahead MPC: assuming the aggregator can exactly predict the next 30 AGC commands.4) Priority control policy.We consider different initial storage levels in this test.We still assume the aggregated initial SoC is s ag,0 = 0.5.We use variance σ 2 to measure the degree of individual s i,0 deviating from s ag,0 .We compare four policies under different values of σ 2 .Results are listed in Table IX.
2) Scenario 1: Identical initial SoCs: In this case, all ESUs have the same initial SoCs, so σ 2 = 0, s i,0 = s ag,0 = 0.5, which is a perfect initial state.Results in Table IX show that all policies achieve the same and also the highest performance score, which is K perf = 1.The aggregated response power profiles and storage level dynamics are shown in Fig. 10.Since AGC signal has a zero mean and E ag /P ag = 45 minutes, the energy capacity is sufficient, and the aggregate SoC will never reach the lower or upper limits.Moreover, because the AGC signal cannot exceed the power capacity P ag as required and the initial storage levels are identical, the aggregator can always track the AGC signal accurately to obtain the highest performance score.Compared with the hindsight results in Fig. 10(b), the aggregator can keep individual storage levels much closer under the CAWF policy, as in Fig. 10(c).
2) Scenario 2: Small discrepancy in the initial storage levels: See Case 2-5 in Table IX.We gradually increase the discrepancy of initial storage levels of individual ESUs, causing σ 2 to change from 0.0561 to 0.1941.Under such settings, the performance score of hindsight policy is always equal to 1, which means that Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.if the entire sequence of AGC signal is known in advance, the aggregator is still able to faithfully respond to the dispatch order through a comprehensive control of individual ESUs.However, in practice, AGC signal is hardly predictable.The CAWF policy could achieve the same score as the hindsight policy without requiring future AGC signals.MPC fails to get a full performance score in Case 2, Case 4 and Case 5, and the priority control fails to get a full performance score in Case 4 and Case 5.But it is difficult to say which one of MPC and priority control is better.For example, in Case 4, Type 5 ESUs are nearly empty while Type 6 ESUs are almost fully charged; without proper coordination, the empty (full) ESUs can no longer be discharged (charged), thus the regulation capability decreases.Under priority control, Type 6 ESUs with the largest efficiency η is dispatched first; given the initial storage level, Type 6 ESUs quickly become fully charged and cannot store more energy, leading to a poor performance score.
2) Scenario 3: Large discrepancy in the initial storage levels: See Case 6 in Table IX.This is an extreme case, in which σ 2 = 0.2633.The aggregated response power profiles and storage level dynamics are shown in Fig. 11.At the beginning, some ESUs are empty while others are fully charged, inevitably restricting the capability of charging and discharging.Despite knowing the exact sequence of AGC signals, the hindsight policy cannot perfectly follow dispatch signals.The theoretically optimal bound of performance score is K perf = 0.9902.Even under such an extreme condition, CAWF policy could still receive the optimal performance score.In contrast, the optimality gaps of MPC and priority control are as large as 1.4197% and 3.0199%, respectively.As shown in Fig. 11(c), the aggregator aims to keep individual storage levels as close as possible under the CAWF policy.In this way, the charging/discharging flexibility of ESU cluster is preserved for future usage.
We further increase the number and difference of ESUs.We choose 100 ESUs whose T i obey a uniform distribution on the interval [0.5, 0.75]; the power capacities uniformly range from 0.05 MW to 0.5 MW; the initial SoC is randomly chosen from the interval [0.05, 0.95].Then, the aggregation results are P ag = 24.19MW and E ag = 18.14 MWh.The aggregator submits P r = P ag and receives frequency regulation signal within [−P ag , P ag ].We randomly select 100 AGC signal sequences with 2-hour time span.The optimality gaps of 100 scenarios are equal to zero.Additionally, we plot the SoC variance of 100 ESUs under different policies in Fig. 12.It can be observed that CAWF converges to zero variance, which means that all SoCs converge to a consistent value.In this way, the aggregated flexibility of ESU cluster is preserved.
In summary, the above case studies show that the proposed CAWF policy can achieve high performance scores in frequency regulation service provision.It outperforms MPC and priority control, exhibiting a promising potential in future frequency regulation markets.

VII. CONCLUSION
This paper studies the aggregated use of small energy storage units.An analytical expression is derived to determine a reasonable pair of power and energy capacities of the storage cluster, which is shown to be consistent compared to the Minkowski-sum approach.The CAWF policy is developed for real-time power disaggregation control of individual storage units without forecasts on dispatch signals.The key findings in the case studies include: 1) If the E/P values of managed ESUs differ significantly, the aggregator should cluster ESUs into several groups.Each group includes ESUs with close E/P values to avoid loss of flexibility.A recommended criterion for clustering is max i∈G {T i }/ min i {T i } ≤ 1.5, where G is the set of ESUs in the cluster.The group with a smaller average value of T i can be used to provide frequency support; the group with a larger average value of T i can be used for energy arbitrage.2) For the task of wind power ramping alleviation, the CAWF and MPC policies perform better than priority control, and CAWF outperforms 12-slot exact lookahead MPC.To get a better performance, MPC needs exact forecasts in the next 4 hours.3) For frequency regulation provision, if small ESUs have nearly consistent initial SoCs around 0.5, then CAWF, MPC and priority control yield the optimal performance.Otherwise, if the initial SoCs exhibit larger discrepancy, CAWF still achieves optimal performance, while MPC and priority control cannot.In view of the fact that predicting AGC signals is difficult and priority control uses some batteries more frequently, the proposed CAWF is attractive in practical applications.Another salient feature is that CAWF leads to the similar utilization rate of individual ESUs, while other policies do not have such a guarantee, especially priority control.In the future work, we will consider more flexible resources such as virtual power plants, deferrable loads, thermostatically controlled loads, and so on.We will develop more general online policies that can also utilize imperfect forecasts.

Fig. 7 .
Fig. 7. Simulation results of aggregate power profiles under different policies.
Calculate 1/(α i + xi ) and 1/α i for all i.Merge two points with the same value.Denote the total amount of break points as L. The sorted sequence is{ν 1 , ν 2 , . . ., ν L }. Divide ν ∈ R into L + 1 intervals.2: For Each interval [ν j , ν j+1 ] 1) Disaggregation of Charging Power: If the aggregator receives a charging order D c t , the control actions p c i,t of individual i , (E i − e i,t−1 )/(η c i Δ t )}.i∈N p d i,t ≤ D d t (27c) where P d i,t = min{P i , (e i,t−1 )η d i /Δ t }.The objective function and constraints have similar interpretations as those in (26).

TABLE III CORRESPONDENCE
IN THE CAWF PROBLEMS Fig. 3. Flowchart of proposed ESUs aggregation and disaggregation methods.
A simple case where 4 types and 150 distributed ESUs have same T i is simulated; parameters are given in Table IV.For brevity, we omit the efficiency term and focus on the capacity term.The aggregation results are summarized in Table V.Clearly, aggregate energy (power) capacity is equal to the sum of energy (power) capacities of individual ESUs, and the aggregation process has no flexibility loss, leading to LoF = 0. 2) Case 2: Clustering ESUs With Different Parameters: A case where 10 types and 100 distributed ESUs with parameters in Table VI is studied.We first categorize 10 types of ESUs into 3 clusters (A, B and C) based on their T i values.ESUs in each cluster share a close T i value.The aggregation results are summarized in Table VII.It is apparent that E ag = i E i .However, the aggregated power capacity P ag is different.The direct summation method gives the largest aggregate capacity value P Σ ag = i P i , but this capacity may degrade after some Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
Table VI, to compensate for rapid changes in wind power output.As in Table VII, the aggregate

TABLE VIII MC
SIMULATION RESULTS OF DIFFERENT CONTROL POLICIES

TABLE IX RESULTS
OF4 POLICIES UNDER DIFFERENT INITIAL SOC LEVELS