Grid Reserve and Flexibility Planning Tool (GRAF-Plan) for Assessing Resource Balancing Capability Under High Renewable Penetration

High penetration of intermittent generation increases uncertainty and variability in balancing reserve needs. New tools are needed to help the balancing authority system operator plan for intraday and intra-hour balance between generation and load. The Grid Reserve and Flexibility Planning tool (GRAF-Plan) helps plan for adequate balancing reserves for future years or seasons for expected wind and solar generation. It also assesses the flexibility of the scheduled generation fleet to meet such requirements. The estimations are based on utilities’ operational practices (e.g., forecasting and time frame of reserve deployment), and it incorporates detailed data from renewable generation and load. Application of the tool in estimating reserve requirements in Central America under different levels of renewable generation (high and low) and for the Western Electricity Coordinating Council (WECC) 2030 Anchor Data Set scenario is discussed.


I. INTRODUCTION
T HE inherent variability and uncertainty of inverter-based renewable power generation add difficulty to system operators' efforts to schedule adequate balancing reserves [1]- [4].Planners need high-fidelity intraday and intra-hour studies.To maintain a power balance, system dispatchers rely on a series of rapid (regulation [RL]) and slow (loadfollowing [LF]) adjustments of resources through automated and manual balancing operations [5].Balancing reserves considered in this work are the intraday balancing requirement, as depicted in Fig. 1.These are formally defined below.Contingency reserves are not considered here.
Regulation: Real-time (RT) adjustments for regulation service are done by generators under automatic generation control (AGC).The difference between actual net load (demand − solar generation − wind generation) and RT forecast (5-15 minutes ahead) is the balancing reserve for regulation.This is part of secondary frequency control.Load Following: Load-following (LF) adjustments are tertiary control, where spinning and fast-start non-spinning reserves are manually adjusted to follow the slow imbalance between generation, load, and scheduled interchanges with neighboring balancing areas.The reserve requirement for LF is the difference between the RT forecast and the hour-ahead (HA) forecast.Day Ahead Balancing: If the system has only day-ahead (DA) operation, the reserve requirement is often the difference between the actual net load and the DA forecast.From these definitions, it is clear that the balancing reserve requirement arises chiefly from forecast error and minute-tominute variability.Hour-to-hour, seasonal, and monthly variability and forecast error arising from different combinations of generation and load would dictate the hour-wise balancing needs.Therefore, (1) simplistic, deterministic approaches for securing reserve (e.g., certain percentages of net load) are no longer applicable because they might under-or overestimate the reserve requirement during different time slices.Underestimation leads to load shedding or system reliability issues whereas overestimation leads to over-allocation of resources, increasing cost; (2) the tolerance to imbalance dictated by the frequency control performance standards implemented by the system operator needs to be accounted for balancing reserve estimation; (3) committed generator's flexibility to track balancing signal must be assessed against hour-wise, expected distribution of balancing needs [1].

A. RELATED WORK
Over the past several decades, much work has been dedicated to estimating operating reserves under high penetration of renewable power.Prada [6] proposed allocating operating reserves through capacity markets using a stochastic demand model.In [7], a method was proposed to reduce the variability of the reserve requirement under high penetration of wind generation using balancing authority (BA) cooperation and sub-hourly dispatch.In [8], the authors empirically studied the operational impact of procuring additional reserves in a system, such as cost, reliability, and pricing.One of their major observations was that, although the additional reserve did not affect the reliability metrics, it provided additional ramps to meet system requirements and cause price volatility.
Authors of [9] modeled the wind forecast error for reserve estimation as a gamma distribution with a time-varying parameter based on lead time.They proposed reserve allocations for both event and nonevent scenarios by dynamically allocating reserves and keeping the ''risk'' level constant throughout the day.In [3], Ela et al. showed that the level of operating reserve needed for wind generation variability is not constant during all hours of the year; they proposed dynamic allocation of reserves to reduce the amount needed in the system for most hours by statistical analysis of wind generation.
Paterakis et al. [10] quantified a two-stage stochastic programming model to account for DA hourly market clearing and minute-resolution RT operations.This study allowed demand-side resources, such as load flexibility from dedicated commercial buildings, to provide LF and contingency reserves.Intra-hour variations from load and wind and transmission line contingencies were modeled.A subsequent study [11] extended this analysis to calculate LF reserves resulting from variability due to high wind penetration, with load variability held constant.In recent work, [12] Wang et al. introduced a multi-period stochastic optimization framework to economically dispatch different energy resources, including planned operating reserves.The method is adaptive with a short horizon and tailored for day-to-day operation.
All the methodologies discussed above are operation tools for accurately estimating reserves on a daily basis.This approach is not tuned for long-term planning studies.In operational tools, the effort lies in the recent data trend to improve the forecast in the forecast interval.Where as, for long-term planning studies, recent data trends cannot be leveraged as hour-to-hour forecasts into the future cannot be obtained with high confidence.Therefore, for the planning studies, the stress is given to generating forecast errors representative of typical forecast error and take a probabilistic approach on the reserve estimation based on projected systems conditions.
Estimation of reserve requirements under long planning horizons is understudied in literature.Aigner et al. [13] studied the effect of additional wind generation in reserve requirements for future years of planning in the Nordic system.In [14], the authors propose a dynamic investment model to assess the cost and availability of balancing reserves for a 100% renewable European grid of 2050.However, in both these cases, no generic methodology to estimate reserves from hour to hour is outlined that could be adopted for other BAs.Therefore, a holistic approach to estimate reserves under different long-term scenarios of adopting renewables with various compositions is needed.

B. CURRENT PRACTICES
The power systems market operators and ISOs around the world adopt various strategies to estimate and plan for reserves for future years.In a recent report [15], Australian Energy Market Operator (AEMO) has shown the importance of moving to a time of day profile for reserve estimation for the next year due to significant wind and rooftop solar integration [16].However, as of 2019, they have used static reserve procurement.Bonneville Power Administration (BPA) uses a methodology very similar to the methodology outlined in the paper.However, their error model is simplistic.To generate an intra-hour forecast, only tenminute average data sets are developed based on actual load.Such a simplistic error model might not mimic the actual error, which usually follows specific probability distribution.In the report [17], reserve procurement for 2030 is estimated for India based on national standards include 3% of peak demand and 5% of peak renewable energy production.Some of these strategies are simplistic, ad-hoc, or do not utilize time-of-the-day values.It is imperative how reliable, hour to hour reserve procurement could have significant cost savings, increase reliability and improve the utilization of resources.

C. BACKGROUND
Over the past decade, the team at Pacific Northwest National Laboratory (PNNL) has developed statistical methods for balancing reserve estimation for LF and regulation services under different renewable penetration scenarios.The methodology has been successfully implemented and adopted by California Independent System Operator (CAISO), Northwest Power Pool, Nevada Energy, Bonneville Power Administration, Western Electricity Coordinating Council (WECC), Duke Energy, six countries in Central America (CA), and in an initial feasibility study of small modular nuclear reactors in Puerto Rico [18].In [4], the methodology was implemented to account for wind integration in the CAISO system; the balancing signals were separated into LF and regulation, and a truncated normal distribution (TND) model for forecast error was introduced.In [19], the requirement of constraining the regulation to the BA area control error (ACE) limit was introduced.In [20], an early application of the balancing estimation method was used to show the advantage of consolidating BAs in WECC.A PNNL report [21] discussed how different balancing requirements could be integrated into various stages of production cost modeling.

D. CONTRIBUTION
The contributions of this work include the following: a) Planning tool for high wind and solar integration Recent application of the methodology and development of a software tool, Grid Reserve and Flexibility Planning (GRAF-Plan), is described.GRAF-Plan was developed for routine assessment of balancing reserve needs as part of the planning process.The application of GRAF-Plan is discussed for the WECC 2030 System Stability Planning Anchor Data Set (ADS) and for Central American countries.b) Generator fleet flexibility assessment The capability of the PNNL balancing reserve estimation tool is extended with a probabilistic assessment of a generator fleet's flexibility to meet the estimated reserve requirement.The method for identifying potential shortages in specific hours is discussed in Subsection III-B.The detailed algorithm of previously developed reserve estimation is also included in this paper to complete the description of GRAF-Plan.
The rest is organized as follows.Section II gives an overview of the method.The detailed methodology is described in Section III.Application of the tool for the WECC 2030 ADS case and in Central America is discussed in Section IV.Section V concludes the discussion.

II. PROCESS, INPUT, AND OUTPUT
This section outlines the major input, process, and output of the tool.

A. INPUT
GRAF-Plan requires accurate representation of the variability and uncertainty of load and renewable generation.A sample, representative, minute-wise profile of load, wind, and solar generation for a study window (month/season/year) is needed.Historical data for existing plants is used.For prospective plants, simulated production using weather models may be used.The BA's typical load projection for future years can serve as the load profile.These time series are binned into daily curves to represent typical days over the study period.

B. PROCESS
Based on the time-series input, Monte Carlo runs of forecast error signals are generated for load and resources and corresponding balancing requirement curves are generated.Variability from one source could offset variability from another source and thereby reduce reserve requirements.Next, for a given period and confidence interval, the maximum daily hour-wise reserve requirements are extracted from the analysis.For assessing a generation fleet's capability, the aggregate upward and downward flexibility of the generator fleet is calculated and compared with Monte Carlo runs of balancing requirement curves.This helps identify potential minute-wise shortages.GRAF-Plan could also assess the distribution of shortages for a given hour.

C. OUTPUT
The reserve requirements provided are characterized by upward and downward capacity (MW), ramp rate (MW/minute), and duration (minutes) of ramps.These parameters help compare requirements to the characteristics of the generation fleet that is available to provide required reserves.For capability assessment, a sample generator fleet's hour-wise probability distribution of hour-wise capacity and ramping exceedance is obtained.

III. METHODOLOGY
The GRAF-Plan tool has two functions: (a) estimating balancing reserve requirements and (b) assessing generator fleet capability.Fig. 2 shows the methodology.For both functions, Monte Carlo simulations of balancing curves over three look-ahead horizons-DA, LF, and RL-are generated using forecast error models.The capability assessment extends the reserve requirement using a sample generator fleet schedule.These two methods are described in detail in the following two subsections.

A. BALANCING RESERVE REQUIREMENT ESTIMATION
The balancing reserve requirement is calculated in four steps: model forecast error, generate a balancing curve, calculate the ramping requirement, and analyze the performance envelope.Fig. 3 outlines the steps along with data and parameter ingestion and Monte Carlo simulation.

1) FORECAST ERROR MODELING
Estimating reserves for future scenarios relies on forecast errors, which mimic the empirically derived statistical characteristics of historical forecast residuals.The forecast errors are added to simulated actual minute-wise time series of load, wind, and solar, which are typically obtained from the regional operators, and weather models for the planning year.
Reproducing the empirically derived statistical characteristics requires different techniques depending on the forecast regime (DA/HA, RT) and the resource type (load, wind, solar).Table 1 lists the forecast types for each regime and resource type and their error model.The techniques used for error generation are sampling from an autocorrelated truncated normal distribution (TND), using the TND with the clearness index (CI), persistence of errors (PST), and persistence with CI.

a: AUTOCORRELATED TND
The HA and DA load, wind, solar, and RT load forecast errors are modeled as autocorrelated samples from a TND.The mean of the forecast errors is assumed to be zero and the standard deviation is derived from either historical data or weather models.The assumption to use historical forecast error parameters is justified if the system operator do not foresee any significant change in forecast methodology within the planning horizon.The truncation of the distribution constrains the resulting forecasts to a realistic operating space.Because the empirically measured forecast residuals also show strong serial correlation, the samples from the TND are given autocorrelation with one lag.The algorithm for generating autocorrelated forecast errors using the TND for all three types of ancillary services is presented in Algorithm 1.Here, r 1 and r 2 are the minimum and maximum bounds of truncation and N is the length of the time series.The mean, standard deviation, and lag-1 autocorrelation values of the forecast error signal E are µ, σ , and a.In steps 2-5, The methodology implements a pseudo random generator to obtain a sequence of independent and identically distributed (iid) numbers drawn from a general TND.Then an auto-regressive model of first order (AR(1)) is used to obtain forecast error signal E (step-7) [22].In steps 2-3, the truncation points (r 1 , r 2 ) from the general TND are converted for a standard TND by shifting the truncation points by mean µ and scaling down by standard deviation σ .The new truncation points are r 1 −µ σ , r 2 −µ σ .To implement the pseudo TND generator [23], two MATLAB functions; uniform distribution generator rand(•) and inverse error functions erfinv(•) are utilized [24] in step-4.The exact steps of the implementation of general TND generator using standard uniform distribution generator and inverse cumulative distribution function could be obtained from Section 2.5 of [25].
Fig. 4 is an example series generated using Algorithm 1.

b: CLEARNESS INDEX AND TND
Generation of HA solar forecast errors requires binning the solar production time series by CI, which is a ratio of the actual to ideal extraterrestrial solar production.The variance of the errors is lowest when the CI is close to either 0 or 1; prediction of solar production becomes more precise when it is either entirely sunny or entirely cloudy.This differs from wind and load prediction, where the standard deviation is assumed fixed for the entire period of the study.
Algorithm 1 Forecast Generation Using TND ▷ Draw from TND  As per practice from organizations such as CAISO, the following CI bins are assumed.
Each CI bin will have a different standard deviation; periods with CI close to 1 or 0 will have much lower variability attributable to fully sunny or fully cloudy days, respectively [26].The standard deviation sent to this algorithm, σ , is unique to the CI bin.The errors generated for each CI bin retain the ordinality of the original time series.

c: PERSISTENCE
Real-time wind production is assumed to persist from one period to the next: the actual generation value from the previous interval is used as the forecast for the current interval.An example generated RT wind forecast signal is compared to actual values in Fig. 5.

d: CLEARNESS INDEX AND PERSISTENCE
The persistent value of the CI is used to calculate the RT solar forecast.The CI from the previous interval is multiplied by the ideal generation for the forecast interval to obtain the forecast value for the current interval.See Algorithm 2 for an example, where P A is the actual solar generation, P I is the ideal solar generation, and t i is the reference interval for calculating the CI.The persistent forecast, which is scaled by the CI for the previous period, is an average of the ideal generation over the previous five minutes.
Algorithm 2 Forecast Generation Using CI and Persistence ▷ Mean of ideal in current interval 4: return E ▷ The forecast error signal 6: end procedure

2) GENERATION OF BALANCING CURVES
A simulation is performed to generate the appropriate forecasts based on the user-provided error statistics.To do this, the Monte Carlo method is applied to generate a userspecified number of forecasts.The difference between the generated forecasts and the actual operating curves represents the balancing capacity requirement at each time step of the study period.Taken in combination, errors in load forecasting, wind forecasting, and solar forecasting will interact.For example, an overestimation of solar production could partially or completely offset underestimation of wind production or overestimation of load during the same period.
Several forecast error series RT L , HA L , HA W , DA L , and DA W are generated using Algorithm 1, HA S and DA S with methodology in subsection III-A.1.b,RT W with methodology in subsection III-A.1.c,and RT S with Algorithm 2. Here, AC, RT , HA, and DA denote the minute-wise actual, RT forecast, HA forecast, and DA forecast generation.Subscripts L, W , and S denote the series associated with the load, wind production, and solar production.For each Monte Carlo run, a single balancing reserve curve is calculated, accounting for all available load and production data, and the following RL and LF DA minute-wise balancing curves are generated: The indices for time instants and the Monte Carlo runs ∀i ∈ N are dropped from all the above expressions for simplicity.

VOLUME 10, 2023
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

3) DETERMINING RAMPING REQUIREMENTS
Once the balancing curves are generated, the ramp rate and duration requirement information is extracted.In reality, the ramp rate values change every minute, or corresponding to the temporal resolution of the series.Therefore, the ramp duration requirement corresponding to a fixed ramp rate value would vary every time instant without conveying any useful information.Therefore, the consecutive slow changing ramps are clubbed under one value of ramp rate requirement.If a 'drastic' change in ramping is observed, the ramp rate requirement is updated and corresponding duration value is determined.The swinging door (or swinging window) algorithm is used [4] for this approximation.Algorithm 3 describes how the ramp rate and duration information can be determined from any reserve curve.Here the ACE signal can be any of the balancing balancing reserve curves, G LF , G R , or G DA .Fig. 6 illustrates how this algorithm works.Initial ramp rates ρ 1−2 , ρ 2−3 . . .are determined by calculating the 1-minute jumps in ACE series (step-7).Next, beginning with the first point, the maximum number of subsequent ramping requirement values are identified whose value vary within tolerance dev (steps: 8-14).These consecutive points form a band.In Fig. 6, the first band is represented by the region that includes π 1 to π 4 .In this example, the initial ramp rate is ρ 1−4 and corresponding required ramp duration is δ 1−4 (step-17).Beyond π 4 , the next balancing requirement point, π 5 , increases sharply and therefore falls outside the tolerance dev, and establishes a new band.The tolerance value dev is usually a conservative value taken as a fraction of the L − 10 parameter of the system [27].This ensures the worst case frequency deviation due to the approximation is within control performance standards.For a balancing area not in North America, a reasonable tolerance value is determined from its market's control performance standards.

4) PERFORMANCE ENVELOPE ANALYSIS
In most balancing areas, there is an allowance for the balancing signal or the ACE to deviate from zero, within some limits.This limit is specified in the BA's performance standards to allow the system to tolerate some instants of balance violation [28].
After the hour-wise balancing capacity and ramping requirements have been determined, a performance envelope for k = 1 : j − 1 do 17: return ρ, δ ▷ Returns the ramp and delta series 27: end procedure analysis is done to discard extreme values.The ramp rate, ramp duration, and RL capacity are plotted for every time step within the hour for all Monte Carlo simulation results, and a 3D envelope is applied that encompasses some percentage of these results.Please refer to [4] for more explanation.

B. GENERATOR FLEET CAPABILITY ASSESSMENT
Flexibility is the ability of a system to respond to changes in demand and generation [29].
After the balancing reserve requirements have been estimated, the system planner can check whether the generation fleet designated to provide RL and LF service is flexible enough to do so.This method assesses the hour-to-hour probability distribution of a generator fleet's inability to match the balancing requirement with the given commitment.This method was developed for Central America's DA market, so the capability is assessed for the DA combined (RL + LF) reserve requirement.
An important aspect of considering balancing reserve requirement is to identify resources that can provide reserves, e.g., fast-start units like hydropower (impoundment type or  pumped storage), diesel generators, combined cycle gas turbine power plants, and steam turbine power plants that run on natural gas.The ability of the generator fleet to provide reserves depends on the following: whether each generator is committed, their operating set points (G * i ), and generator characteristics, such as maximum (G i,max ) and minimum G i,min power output, maximum ramp-up and -down rates [r i,up , r i,down ]), etc. Fig. 7 shows the flexibility of an online generator i.The set point of a generator is at G * I .When online, the generator can go up to G i,max to provide upward reserves.To provide downward reserves, the generator can go down to G i,min .Often, G i,min is not 0. Fig. 8 illustrates the process of identifying reserve requirement exceedance, the difference between requirement and capacity.First, three curves are plotted: requirement curves, capability-up curves, and capability-down curves.

a: REQUIREMENT CURVES
The balancing requirement curves are the Monte Carlo runs used to estimate the requirement from (1).In Fig. 8, these curves are denoted by faint dotted lines.

b: CAPABILITY-UP CURVE
The blue solid and dotted lines of the capability-up curve show the aggregate capability of the online generators to provide additional capacity during different minutes.Individual generators' upward capabilities, defined as the difference between maximum capacity and scheduled value, are determined.For each generator G i , the capability-up value G up for a particular hour is the difference between maximum capacity G max and its scheduled value G * for a particular minute m, G i,up (m) = G i,max − G * i (h), where the hour corresponding to minute m is h = floor( m 60 ).To account for the inter-hour ramping adjustment, the capacity curves from individual generators are ramped up and down at a steady rate to reach the next hour's set point, constrained by their ramp rates.After individual capability curves are obtained, they are aggregated to generate the blue solid and dotted line in Fig. 8. Since the generation fleet's schedule is a viable solution from the unit commitment problem, the steady ramp connecting the setpoints of two consecutive hours during the inter hour adjustment period should obey ramping constraints.

c: CAPABILITY-DOWN CURVE
Similar to the capability-up curve, the line of red dots in Fig. 8 shows the aggregate downward reserve capability.For Central America, the generators can reach a minimum value of 0 MW.Just like the calculation of the ''capability up,'' each generator for a particular hour is adjusted for inter-hour ramping and constrained by the ramp-down rate provided for it.For each generator at minute m, the capacity to provide downward reserve is given by G i,dw (m) = G i,min − G * i (m).After the capability and requirement curves are plotted, four types of deficiencies are studied.To define these deficiencies, two regions in the capability curves are identified.The ramping period between intervals, t r , is when the AGC steadily ramps to reach the next interval's target output and the effective operating period t c is when the AGC is scheduled to provide steady output.
The outputs from the capability assessments is a hour-wise summary of box-plot of the upward exceedance instances.Only the positive capacity exceedances indicate a shortage of balancing reserve.Similarly, for the downward exceedance, only the negative values indicate failure to provide required downward reserve.We denote these cases shortage scenarios, given by cup The distributions of these hour-wise shortages are summarized in the form of box plots (e.g., as depicted in Fig. 15).Such box plots show the mean, standard deviation, median, minimum, and maximum values of shortage points (if any) for a particular hour of a month.

A. BALANCING AUTHORITIES IN THE WECC REGION
Results in this section reflect data from CAISO.The total installed generation capacity in the 2030 case for CAISO is 310,882 MW, of which 85,690 MW (27.5%) is solar and 50,474 MW (16.2%) is wind.Significant behind-the-meter (BTM) and utility solar are considered.
From available BTM and utility solar data in hourly resolution, one-minute data is generated by emulating the historical minute to minute variability.Typical forecast error statistics for CAISO are obtained from the CAISO Open Access Same-time Information System (OASIS) [30].GRAF-Plan estimated CAISO's reserve requirement for the WECC 2030 scenario, assuming the forecast errors and market practices remain the same.Fig. 9 shows the hour-wise maximum and minimum capacity requirement for RL for January and July.Those for LF for the same months are shown in Fig. 10.The total capacity has been decomposed into solar (yellow), wind (green), native load minus BTM solar (red), and the total requirement (blue).Note that this decomposition is not additive, because it was obtained by running GRAF-Plan with and without each resource.Shifts arising from Daylight Savings Time are not considered in the results for January.Several observations for capacity requirements may be made from this decomposition: 1) Solar affects RL more than it affects LF.
2) The reserve requirement attributable to native load less BTM solar is a function of rooftop solar, thereby causing increased variability during the day.Cdw ← prctile( C dw , p 2 ) 21: Rup ← prctile( R up , p 2 ) 22: Rdw ← prctile( R dw , p 2 ) ▷ Returns the array for the percentile above p 2 ; i.e., removes extreme values.

23:
end for 24: end procedure 3) Longer summer days affect RL more severely than LF, with no significant hour-to-hour difference apparent from wind.

B. CENTRAL AMERICA
In Central America, wind and solar generation are increasing rapidly; they can potentially replace diesel generation, which is widely used but expensive.Intraday and intra-hour balance between generation levels must be maintained in each of the member countries of Mercado Eléctrico Regional (MER, the regional electricity market) is necessary to manage high penetration of solar and wind generation.The countries under study were Costa Rica, Nicaragua, Honduras, El Salvador, Guatemala, and Panama.We applied GRAF-Plan in the MER countries by generating scenarios, obtaining data, and applying the method to analyze balancing reserve requirements and generation capability in a selected member country.The steps to process the information from the region to use in GRAF-Plan is shown in Fig. 11.
All MER member countries provided scenarios for renewable generation for baseline, low, medium, and high penetrations of wind and solar generation.For future scenarios, generation and load data were obtained from the site-specific weather model provided by our weather partners.

1) RESULTS FOR BALANCING RESERVE REQUIREMENT
Here, limited, normalized results are discussed; actual results cannot be published.Two scenarios for a Central American country are discussed.
• S0: S0 is the baseline case, with low penetration of renewables, for the near future.S0 for the selected country has 1 MW of solar generation and about 300 MW wind generation by the end of the study year (which was taken as baseline scenario, E0).
• S1: S1 is a scenario with high penetration of renewables a few years in the future.S1 for the study country will have about 100 MW of solar generation and about 500 MW of wind generation.These graphs yielded several observations: • Moving from DA operation to intraday operation would significantly reduce the reserve procurement.• Higher penetration of renewables significantly affects reserve capacity and ramp rate requirements for all three services.
• Reserve requirements vary significantly from hour to hour.A fixed amount of reserve procurement would over-or underestimate the requirements at different hours.

2) RESULTS FOR GENERATOR FLEET CAPABILITY ASSESSMENT
After the balancing reserve requirements were calculated, we used typical generator scheduling as per the actual projected net load for the S0 case.Four types of shortages were analyzed for different times of day.The example country has many hydro units that can provide the steep ramping required for ancillary services; no significant flexibility shortfall was observed for any hours.For illustration purposes, it is assumed that significantly less flexible generation capacity was available in this country by scaling down by an arbitrary factor.The shortages resulting from reduced flexibility within confidence ranges are summarized in box plots for different hours of the day in Fig. 15.Not all hours have shortages.
Another important aspect of the generator fleet's capability assessment is that shortages do not always indicate resource inadequacy, but rather indicate places power system operators and planners should monitor.Shortages could mean that operators need to take action, such as bringing additional flexible generation online or modifying the set points of online generation.
To assess the distribution of the hour-wise exceeding are estimated using Algorithm 4. Inputs to the algorithm are the minute-wise upward and downward capability curves defined in Section III-B and the cutoff percentile for extreme values of p to remove values in percentiles below p 2 and above 100 − p 2 .

V. CONCLUSION
This paper presents the inner workings of PNNL's GRAF-Plan tool for estimating balancing reserves and assessing committed generator fleet capability.The tool's adaptation for the WECC 2030 ADS case and for a Central American country are discussed.Reserves for future years with high renewable penetration could be estimated for a BA with the method outlined in the paper.The inputs are minute-wise load projection for a given future year, typical production from existing and future sites of renewable generation, and forecast error statistics.Typical production values for a month or season and Monte Carlo runs would create autocorrelated forecast error curves.Balancing needs are estimated and binned into daily hour-wise values.For a generator fleet capability assessment, the output of unit commitment is used to assess whether the committed generators with the chosen set points would be able to provide the required reserve in either direction.This tool would be ideal for a system planner to study the effects of planned penetration of renewable integration, assess the feasibility of adopting a new technology, e.g., marine kinetic energy, and assess the generator fleet's flexibility to match added reserve requirements.

FIGURE 2 .
FIGURE 2. Major functions of the tool.

Algorithm 3 1 ▷
Swinging Door Algorithm for Ramp Calculation 1: procedure SwgrDr(ACE, dev, dt) ▷ Calculates ramp requirement using swinging door algorithm 2: i ← Index for each element of ACE data 3: while i < N do▷ Traverse all elements of the ACE data 4: instant's ramp rate and duration 18: Break out of the loop ▷ Close the door 19:

FIGURE 7 .
FIGURE 7. Generator's flexibility to provide reserve in upward and downward directions and its ramp rates.

FIGURE 9 .
FIGURE 9. CAISO 2030: Estimation of RL capacity requirement for a winter (January) and a summer (July) month.Contribution of reserve requirements from wind, solar, and native load are decomposed.Total LF capacity requirement is shown in blue.

FIGURE 10 .FIGURE 11 .
FIGURE 10.CAISO 2030: Estimated LF capacity requirement for a winter (January) and a summer (July) month.Contribution of reserve requirements from wind, solar, and native load are Total LF capacity requirement is shown in blue.

FIGURE 12 .
FIGURE 12. Load Following requirement for the example country for January of the study year with confidence interval 95%.Scenarios: baseline (S0) and high renewable (S1).(a) Max/min capacity requirement (MW).(b) Ramp rate (MW/min).(c) Ramp rate (MW/min) vs. ramp duration (min) requirement.Values in vertical axis are normalized by the peak native load of the system.

Fig. 12 compares
Fig. 12 compares January LF balancing requirements in S0 and S1 with confidence interval 95%.Figs.12a, 12b, and 12c show maximum and minimum capacity, ramp rate per minute, and ramp rate with respect to duration.Figs. 13 and 14 show the same for RL and DA reserves.These graphs yielded several observations:

FIGURE 13 .
FIGURE 13.Regulation requirement for the example country for January of the study year with confidence interval 95%.Scenarios: baseline (S0) and high renewable (S1).(a) Max/min capacity requirement (MW).(b) Ramp rate (MW/min).(c) Ramp rate (MW/min) vs. ramp duration (min) requirement.Values in vertical axis are normalized by the peak native load of the system.

FIGURE 14 .
FIGURE 14. Day-ahead (RL + LF) requirement for the example country for January of the study year with confidence interval 95%.Scenarios: baseline (S0) and high renewable (S1).Higher renewable penetration results in higher requirement.(a) Max/min capacity requirement (MW).(b) Ramp rate (MW/min).(c) Ramp rate (MW/min) vs. ramp duration (min) requirement.Values in vertical axis are normalized by the peak native load of the system.

FIGURE 15 .
FIGURE 15.Illustrative distribution of hour-wise shortages, assuming that the Central American country had significantly less flexible generation available than in reality, during capacity adjustment period (t c ) [above] and ramping period (t r ) [below] with confidence interval of 95% for the baseline (S0) case.Upward shortages are shown on the left side and downward shortages (not observed) are on the right.

1 :
procedure CI+PERSISTENCE(P A , P I , t i ) DA , C up , C dw , R up , R dw ▷ Returns the ramp and delta series