A Control-based Method to Meet TSO and DSO Ancillary Services Needs by Flexible End-Users

,


1
A Control-based Method to Meet TSO and DSO Ancillary Services Needs by Flexible End-Users Giulia De Zotti, S. Ali Pourmousavi, Senior Member, IEEE, Juan M. Morales, Senior Member, IEEE, Henrik Madsen, Senior Member, IEEE, and Niels K. Poulsen, Senior Member, IEEE.
Abstract-This paper presents a new methodology to exploit consumers' flexibility for the provision of ancillary services (AS) in the smart grid era.The proposed framework offers a control-based approach that adopts price signals as the economic driver to modulate consumers' response.In this framework, various system operators broadcast price signals independently to fulfil their AS requirements.Appropriate flexibility estimators are developed from the transmission system operator (TSO) and distribution system operator (DSO) perspectives for price generation.An artificial neural network (ANN) controller is used for the TSO to infer the price-consumption reaction from pools of consumers in its territory.A proportional-integral (PI) controller is preferred to represent the consumers' price-response and generate time-varying electricity prices at the DSO level for voltage management.A multi-timescale simulation model is built in MATLAB to assess the proposed methodology in different operational conditions.Numerical analyses show the applicability of the proposed method for the provision of AS from consumers at different levels of the grid and the interaction between TSO and DSOs through the proposed framework.
Index Terms-Ancillary services, TSO-DSO interaction, flexibility resources, demand response, artificial neural network.Base-line consumption at the TSO level for end-users' category j at time h [ Starting and stopping binary variables for flexibility provision at level T at time h for end-users' category j and regulation type α. w h,j , x h,j , v h,j Binary variables for modelling the rebound effect at time h of end-users' category j.

I. INTRODUCTION
Ancillary services (AS) are key elements to guarantee the stability and continuity of the electricity supply.They consist of up-and down-regulation services, among other services, in different timescales to assist in grid frequency and voltage regulation, and congestion management.Traditionally, AS were provided by conventional generation units (CGUs) with fast ramp-up and down capabilities.However, in a power system with large penetration of renewable energy sources (RES), where most of the RES are not able to provide balancing services effectively [1], AS provision cannot solely rely on the CGUs [2].This issue is intensified by many CGUs retiring from the generation fleet due to low energy prices.In addition, higher penetration of RES leads to a higher demand of AS [3], which must be properly planned to avoid extreme AS pricing events.This is happening already in California Independent System Operator (CAISO), where the total AS market value raised from US$20M in 2015 to US$172M in 2017 [4].Therefore, finding cheap flexibility resources (such as load demand flexibility) is necessary to provide short-and mediumterm AS [5] that can cope with the sources of uncertainty involved in the future power system operation [6].Although the potential of demand flexibility for AS has been proven in many research studies, only a marginal contribution from load flexibility has been realised for AS provision in practice.One reason is that involving millions of consumers in the AS provision requires tremendous computational power and increases the complexity of the existing AS markets due to non-linearity, stochasticity and dynamic characteristics of the demand.Therefore, the true potential of the demand flexibility has yet to be realised in power systems.
In the last decade or so, the potential of different types of flexible consumers has been investigated for AS provision at different levels of the grid.However, most of the existing studies only focused on individual demand flexibility modelling that is computationally expensive and is not scalable for AS provision application.Moreover, specific category of loads (e.g., residential consumers) are considered in the existing literature, overlooking the tremendous potential of other electricity sectors, e.g., industrial loads.In [7], a methodology is proposed to facilitate demand response (DR) participation by industrial loads in the day-ahead energy market through bilateral contracts, which is completely different from this paper.
A flexibility platform (called Flex operator) was proposed in the SmartNet project [8] to aggregate demand flexibility and offer AS to the system operators (SOs) in real time.However, as discussed in [9], such a framework might lead to operational conflicts (i.e., prioritisation of operators) and remuneration issues (i.e., double remuneration when an asset can satisfy the needs of both transmission system operator (TSO) and distribution system operator (DSO)).It also increases the complexity of the AS market by dealing with numerous aggregators with specific capabilities and drawbacks.In [10], the transactive energy (TE) approach was proposed as a marketbased solution to unlock flexibility from the end-users through adoption of a two-way communication scheme.However, requiring feedback from the end-users complicates the grid infrastructure, compromises scalability of the solution, and raises concerns regarding cyber-security and required computational efforts.The pros and cons of the TE framework are outlined by the authors in [11].In [12], a mathematical approach is proposed to aggregate flexibility of thermostatically controlled loads to provide regulation services to the TSO only.In [13], DER flexibility was used in the planning studies to meet reserve requirements.The FlexPower project [14] proposed a real-time market for balancing power considering participation of aggregated small-scale DER.In these papers, a holistic approach that can facilitate demand flexibility procurement from different sectors with various capabilities in a simple and practical structure has not been offered and the impact of the proposed method has not been investigated on the performance of frequency and voltage regulation at the TSO and DSO levels, respectively.
Quantifying demand flexibility is key to AS planning of future power systems.To this end, [15] characterises energy flexibility by a dynamic function.Such a tool enables the SO to determine which grid problem could be managed by the consumers' flexibility after the submission of a certain signal.In [16], aggregate flexibility of residential loads is estimated based on consumption availability, typical usage patterns, and technical constraints.However, such an approach is based on individual appliance's model and not the operation data, which makes the estimation less practical.[17] dealt with the possibility of estimating aggregate consumers' flexibility, although the proposed framework was limited to the distribution level.Regarding the interaction of TSO and DSO, [18] proposed models to manage the reciprocal impacts of the SOs through the activation of flexibility from consumers.However, the proposed interaction model only focused on addressing the technical issues at distribution level, neglecting the impact of consumers' flexibility activation on the transmission system.
In this paper, we implement a new AS mechanism (called ancillary services 4.0, AS4.0 1 ) based on delta price signals that facilitates application of demand flexibility for AS provision.In the proposed method, each SO is allowed to optimally fulfil its requirements by quantifying available demand flexibility in its area.Each SO generates a real-time price that is submitted to a pool of price-responsive consumers.Such prices are created by the demand flexibility estimator that each SO formulates based on its requirements and the pool of consumers.When consumers receive the time-varying prices, they alter their consumption to minimise their operation cost using local controllers, i.e., energy management systems (EMSs).In order to examine the performance of the proposed AS method, a multi-timescale simulation model is developed in this study including TSO and DSO operation.The load-frequency control (LFC) model is implemented at the TSO level for frequency regulation.At the DSO level, voltage is monitored at steady state by solving a power flow problem.The goal is to allow TSO and DSO to regulate frequency and voltage, respectively, by submitting a single delta price to their respective pool of consumers.The time-varying prices are generated at the TSO and DSO levels independently through an artificial neural network (ANN) and a PI controller, respectively.At the TSO level, aggregate price-response of the consumers is modelled through a mixed-integer linear program (MILP) that minimises the operational cost of the end-users [22].Multiple simulation studies are carried out to reveal the performance of AS4.0 for frequency and voltage regulation.The proposed approach can be thought of as a supporting tool for AS provision, similar to the Flexible Ramping Product (FRP) in CAISO [23] and the Ramp Capability (RC) in Midcontinent ISO (MISO) [24].The main contributions of the paper can be summarised as follows: • Implementing the AS4.0 mechanism in a simulation framework by proposing a full-fledged multi-timescale model for the provision of AS by flexible consumers at the transmission and distribution levels.Specifically, we model, simulate and numerically assess the potential of the AS4.0 setup which was hypothesised in [11], by addressing operational issues at different voltage levels of the grid by exploiting consumers' flexibility.
• Modelling consumers' flexibility response to time-varying electricity prices in a realistic manner, i.e., considering different types of loads and proposing a general and dynamic formulation that can represent different operational conditions of loads.Furthermore, we improve the formulation of the rebound effect (RE) that was initially introduced in our previous paper [22].Our new formulation of the rebound effect only forces the changes in load (linked to the activation of flexibility) to cancel out within a time window starting from the moment the provision of flexibility was activated.
• Proposing an ANN-based price generator for the TSO operation based on the required AS.In fact, the central idea of the AS4.0 framework is the time-varying delta prices that shall be generated to describe the true condition of the grid and consequently obtain a certain reaction from flexible consumers.It requires a model that can describe the price-responsiveness of aggregate consumers' behaviour.Although previous studies have used ANN for general price forecasting, no study considered the adoption of ANN for modelling consumers' reaction to time-varying delta prices similar to a controller.
• Developing a realistic model that can be used for assessing TSO-DSO interactions.This model is dynamic at the transmission level, since the TSO is mostly concerned with frequency regulation, which is a dynamic process.
Although previous studies estimated the flexibility potential that can be achieved by the TSO at the distribution level, they only focused on its impact on the distribution grid.In other words, they omitted the modelling of the transmission system to quantify the need and assess the ultimate performance of the proposed solution on frequency regulation.The rest of the paper is organised as follows.In Section II, AS4.0 setup is briefly explained, while Section III provides mathematical models for implementation.In Section IV, simulation results are discussed in detail.Finally, Section V concludes the paper.II.A BRIEF DESCRIPTION OF AS4.0 MECHANISM AS4.0 mechanism uses control techniques to provide AS at different spatio-temporal scales of the grid using a delta price signal.Through the generation and submission of timevarying prices that depend on the actual conditions of the grid, each SO is able to exploit the flexibility of consumers that are located in its territory.Upon receiving time-varying prices by the EMSs [25], consumers react to minimise their electricity cost.A high-level discussion of AS4.0 setup is provided in [11] and [22].Structurally, the grid can be divided into three spatial levels, i.e., ξ ∈ Ξ = {T 1 , ⋯, T M , D 1 , ⋯, D N , F}, for the operation of the AS4.0 in an interconnected power system with multiple control areas, as shown in Fig. 1.These levels consist of G control areas, i.e., ξ T = {T 1 , ⋯, T M }, N distribution systems, i.e., ξ D = {D 1 , ⋯, D N }, and demand flexibility resources, F. Based on the required AS, each spatial level can further be divided in different time scales.AS is required when a disturbance occurs in the power system (e.g., unexpected outages, renewable generation variations, load changes, etc.).Regardless of the source of the disturbance, the TSO operation will observe a frequency deviation.Let the total power disturbance (which is the one that is seen by the TSO) be denoted by Once the power disturbance hits, the TSO solves a control problem, denoted by M T in Fig. 1, to quantify the required AS based on the frequency deviation and formulates the price signal, denoted by ∆λ α ξ T = {∆λ α t,ξ T ∈ R + ∶ t ∈ τ }.Superscript α specifies the type of regulation (i.e., α = u for up-regulation, and α = d for down-regulation).The price signal is submitted to the EMS of all flexible consumers located within the TSO's territory [26].If the delta price is appropriate, collective consumers' reaction will result in the desired change in consumption to compensate for the original disturbance and, therefore, stabilises system's frequency.
At the distribution level, a similar idea can be adopted for voltage regulation, congestion management, or reducing reverse power flow.In this case, only the flexible consumers connected to the distribution system in the DSO's territory will receive a time-varying price, denoted by In principle, it is possible for the two SOs to broadcast delta prices asynchronously to their respective territories according to their requirements at different timescales.
The issues related to the DSO (e.g., voltage violation) are local and the DSO requires flexibility from a limited number of consumers, as opposed to frequency issues, which are systemwide.Therefore, it is unlikely for the TSO and DSOs to compete for flexibility procurement.However, with the lack of coordination between different SOs, contradicting delta prices could be submitted to the same group of consumers with the aim of unlocking flexibility in opposite directions [9], leading to system instability.Therefore, a coordination scheme between different SOs is imperative to avoid such conditions.Since a TSO-territory involves a larger pool of consumers compared to that of a DSO, it is reasonable to assume that the TSO has a higher chance to gain a certain aggregate response.Hence, the priority is given to the DSO in times of conflict in this study.This way, consumers in the conflicting zones will only receive the time-varying prices submitted by the DSO.The remaining pool of consumers will receive the prices generated by the TSO.
Notwithstanding the above, at the occurrence of rare events, service priority might be reversed.For instance, an N-2 contingency event may threaten the integrity of power system operation as a whole, where all flexibility resources should be called to restore system frequency instead of providing other services to DSOs.To hedge against those rare events, the authors hypothesised a regulated entity (called ancillary services operator (ASO)) in [11] to dynamically determine priority of services.Specifically, by observing the condition of the system in real time and considering reliability/stability standards set by the regulators, the ASO should be able to identify the most immediate threats to system integrity and relay the delta prices accordingly.For more details on the ASO operation and characteristics, please refer to [11].Nevertheless, the introduction of this new entity (i.e., ASO) does not affect the analysis of the proposed AS4.0 setup conducted in this paper.

III. AS4.0 MODELLING
In order to assess the performance of AS4.0 framework, appropriate models of TSO and DSO are needed.In this study, and without loss of generality, the network issues at the TSO and DSO levels are limited to frequency and voltage regulation, respectively.
Frequency regulation is performed continuously in a power system, while voltage regulation is typically required in larger time intervals.As a result, the problem should be solved in different timescale while necessary interactions and power flow between TSO and DSO are maintained.As shown in Fig. 3, the behaviour of the system frequency is described by an LFC model at the transmission level, ξ T , accounting for load changes at the lower level of the grid.Layer D models the aggregate effect on the low-and medium-voltage distribution networks, in which nodal voltages are computed by solving power flow (PF) equations.
The mutual impact of simultaneous TSO and DSO flexibility procurement on the frequency and nodal voltages is captured using the power exchange links between the two levels, as shown in Fig. 2. In this case, every time a change in consumption/generation occurs at one level of the grid, LFC model and PF problem are solved to determine the new condition of the grid, i.e., frequency and nodal voltages.It is worth mentioning that the nodal voltages at the TSO level has not been considered in the model since frequency regulation is the most challenging issue in the future smart grid.The different parts of the simulation model in Fig. 3

A. Grid Models
In this sub-section, appropriate transmission and distribution network models are developed to investigate the behaviour of the power system under AS4.0 mechanism.
1) Transmission System: To model the real-time frequency regulation at the transmission system level, we use the wellknown LFC model, which consists of an equivalent smallsignal model of the grid with synchronous generators based on the swing equation [27].It studies the real-time system's frequency response to disturbances, capturing system's frequency behaviour through a continuous set of equations [28].More  specifically, the LFC model includes primary and secondary frequency control loops.Primary frequency regulation is implemented by the speed governor that measures the frequency locally (for the sake of robustness) and adjusts the steam valve position to confine frequency excursion.In the LFC model, it acts through the proportional controller with 1 R gain, where R = (f0−f ) f and f 0 is the reference frequency [28].The secondary frequency control loop is added to the LFC model as a central automatic control carried out by the TSO to correct steady-state frequency error within a couple of minutes by ramping up or down eligible generators [28].The response of the equivalent generating unit depends on the time constants of the generator and turbine, i.e., T g and T t in the LFC model [28].The feedback controller in the LFC model is responsible for frequency error correction, where its output is the amount of power, i.e., ∆P α,C ξ T = {∆P α,C t,ξ T ∈ R + ; t ∈ τ }, that should be changed to stabilise frequency.Various control methodologies, such as a linear-quadratic regulator (LQR) in [29] and a model-predictive control in [30], have been developed for this purpose.In addition to primary and secondary loops, there is a tertiary frequency control loop that is eliminated from the LFC model as it is comparatively slow.In modern power system operation, tertiary frequency regulation is achieved through high-resolution market operation.
In Fig. 4, a two-area LFC model (resembling the Danish transmission network with DK1 and DK2 areas [31]) is shown with an inter-tie connection, and primary and secondary control loops, in black and red blocks.In the figure, the interconnection between the two areas introduces three main elements in the LFC model: β 1 and β 2 representing the response coefficients of the two areas; T 1,2 describing the time constant of the tie-line flow; and α 1,2 , ensuring correct power flow direction at the interconnection (i.e., areas 1 and 2 see opposite sign of the power flow direction), where α 1,2 =-1 [28].For simplicity, the overall effect of the CGUs is modelled by a single non-reheat steam turbine unit [28].The power disturbance and the resulting frequency deviation are denoted by ω t,ξ T 1 and ∆f t,ξ T 1 , respectively, for area T 1 , and ω t,ξ T 2 and ∆f t,ξ T 2 , respectively, for area T 2 at time t ∈ τ .An LQR is used in this study as the LFC controller, whose design is discussed in [29].
The LFC model is modified in this study to represent AS4.0 framework, as shown in black and blue blocks in Fig. 4 for , and the realised flexibility affecting the balance between generation and demand is denoted by ∆P α,F t,ξ T 2 .
2) Distribution System: As it was explained before, voltage issue at steady-state is perhaps the biggest challenge for the DSO in the smart grid era, which is tried to be solved in the proposed AS4.0 framework.To do so, we solve PF problem to investigate the nodal voltages within the DSO territory.The solution to the PF equations provides node voltages and branch power flows in an interconnected system and is based on Ohm's laws and the definition of apparent power.In this study, a modified IEEE 33-bus standard radial distribution system is implemented as the distribution system network, where original loads are modified to avoid voltage violations at the beginning of the simulations.In the absence of larger distribution network model that could represent the amount of loads within T 2 area, we repeated the modified IEEE 33-bus system for 158 times to scale it up to the required level in such a way that the peak load in T 2 area is similar to the peak load in 158 distribution networks.For AS4.0 simulation studies, however, we assumed that only 10% of the DSOs encounter voltage issues for the sake of practicality.
Since the voltage at a node depends on the load and generation at that node as well as the neighbouring nodes, the nodes of the distribution system are grouped into different clusters based on their physical proximity.It will also improve voltage regulation by increasing the chances of getting enough load demand alteration according to the system operation requirements.In this arrangement, each cluster of nodes will receive a unique delta price signal that reflects the condition of that portion of the distribution network.Without loss of generality, only two clusters are considered in this study due to the size of the distribution system at hand.As explained in Section III, PF problem is solved in ∆t intervals while the LFC model is performed continuously.

B.Time-varying Delta Prices
This sub-section discusses the delta price generation mechanisms at the transmission and distribution levels.
1) Delta price formulation at the TSO level: A functional relationship between the amount of flexibility required by the TSO, i.e., ∆P α,C t,ξ T 2 , and the price signal, i.e., ∆λ α t,ξ T 2 , is needed for the TSO operation in AS4.0 framework.ANN is found to be a suitable tool as it can map complex and nonlinear inter-dependencies between electricity price, historical consumption and other factors (e.g., temperature and day of the week) [32].In this study, we assumed that consumers react to price signals by shifting their loads throughout the day.As a result, the input/output parameters to/from the ANN model should be daily profiles.Required data for the ANN training are generated by simulation studies, where thousands of daily price profiles, ∆Λ α ξ T = {∆Λ α h,ξ T ∈ R + ∶ h ∈ N 24 }, are generated using a normal distribution, as proposed in [22].Then, the reaction of the consumers to the price signals is modelled through an MILP problem.This problem and the aforementioned ANN model are described in Subsections III-C and IV-A1, respectively.
Due to the heterogeneous condition of the transmission and distribution system infrastructures in different areas, consumers in under-developed areas will potentially face higher prices compared to others.In an attempt to avoid price discrimination, the sum of all delta prices is enforced to be as close as possible to zero over a day.To achieve that, the TSO solves a linear program (LP) that tries to marginally change a given delta price profile so that ∑ H h=1 ∆Λ α h,ξ T → 0, ∀α ∈ {u, d}.The new price signal is denoted by ∆ Λα ξ T and the LP is formulated as: ) where Eq. (1b) defines the overall deviation from neutrality, denoted by L over a day; and ψ is the maximum allowed difference between the new and old price, which is enforced by Eq. (1c) and (1d).
2) Delta price formulation at the DSO level: When voltage violation occurs due to disturbance ω t,ξ D , the issue may be resolved by load demand flexibility in that area.To exploit that flexibility, the DSO generates a delta price, denoted by ∆λ α t,ξ D , by formulating a control problem for the loads in the affected cluster.In this study, a PI controller is used to generate delta prices for each cluster in accordance with an effective voltage metric (e.g., average voltage deviation of each cluster).In order to avoid extreme prices, a price cap, ∆λ α ξ D , is imposed, which also represents the upper limit of price reaction.It means that the pool of consumers cannot provide additional flexibility beyond this value due to load characteristics [33].The dynamic prices are then submitted to the nodes with voltage issues.The load flexibility service continues for a certain amount of time (30 seconds in this study) until the source of the voltage disturbance disappears.

C. Flexibility Modelling
In this section, suitable models are proposed to estimate the aggregate consumers' price-response from TSO's and DSOs' standpoints.Since TSO and DSO deal with two different pools of consumers (in size, type and response time), we use specific models for each of them.A power function is used to model consumers' flexibility at the distribution level in response to delta prices.For the TSO, however, an MILP formulation is preferred to develop an ANN-based controller and to quantify the actual flexibility obtained from the consumers for simulation purposes.Please note that in practice, however, the actual load variations can be estimated by the aggregate measurements at the distribution and transmission substations.
1) Consumers' price response model at TSO level: In AS4.0 framework, it is assumed that consumers will minimise their daily cost of electricity upon receiving the delta price using EMS at their premise.Detailed theoretical background, assumptions, and the parameters of the 29 end-users' categories that are considered in this paper are discussed in [22].The proposed MILP formulation accounts for the rebound effect that occurs when providing flexibility [34], and it constitutes a major improvement with respect to our previous model in [22].The MILP problem is formulated as follows: x h,j + w h,j + v h,j = 1 ∀h, j, (2g) ∀h, j, α, (2j) The objective function in Eq (2a) represents the cost of electricity for end-users' category j within a day (i.e., τ H =24 h).The electricity price contains a flat price, λ base (retailer electricity price covering fixed costs and taxes), and a timevarying price, ∆λ α h,ξ T , that is generated by the TSO.The electricity consumption is given by a baseline consumption, P base h,j , and the overall flexibility provided is ∆P α,F h,j,ξ T from end-users' category j at time h for regulation type α (i.e., α = u for a decrease in consumption, and α = d for an increase in consumption).Eq. (2b) enforces the up-and downramp-rate limits, r α j , for category j; Eq. ( 2c) and (2d) define minimum and maximum load flexibility that can be provided by category j.In this study, the minimum and maximum load for category j at time h, i.e., P min h,j and P max h,j , are obtained from historical aggregate data for that category at time h.In this equation, u α h,j is the flexibility status variable for category j at time h for regulation type α.The parameter a α h,j,ξ T represents the willingness of each consumer in category j to adjust load at time h for flexibility type α.Consumers' willingness depends on the price they receive (among other factors such as temperature and day of the week).More details about the modelling of consumers' willingness is presented in [22].We assume that a α h,j,ξ T is computed by: where a α j,ξ T is the maximum price responsiveness of category j for flexibility type α and max ∆λ α h,ξ T is the maximum value of the price set received.Eqs.(2e-2h) enforce the load RE for category j.The total amount of flexibility provided by consumer type j until time t can either be zero (when up-and down-regulation perfectly counterbalance), positive or negative.These three cases are modelled by Eqs.(2e)-(2f) through binary variables w h,j , v h,j and x h,j , and the large and small constants M and , respectively.Eq. (2g) guarantees that only one of these binaries can be nonzero at time t.Eq. (2h) requires that a certain increase or decrease in consumption shall be compensated at least once within a given time period R j .Eq. (2i) ensures that only one type of flexibility (i.e., up-or down-regulation) is provided by category j at time h; Eqs.(2j) and (2k) represent the flexibility activation and deactivation for category j at time h, where y α h,j and z α h,j are starting and stopping binary variables of category j at time h, respectively, for flexibility type α.Eq. (2l) enforces a limit on the number of times that category j can be activated in a day, where n α j is the number of times that a flexibility resource can be activated for end-users' category j for flexibility type α.Eqs.(2m-2n) refer to the minimum (d α j ) and maximum (d α j ) duration for which the load response can be sustained.Eqs.(2o-2p) are the integrality constraints.The values of the parameters used in the MILP are provided in [22].
2) Consumers' price-response model at the DSO level: The composition of the loads changes from one area to another from a DSO' s perspective.Therefore, using the MILP model at the DSO level will become computationally overwhelming as it will require specific model for each area.Moreover, it needs a profound knowledge of the load composition at each node or area, which is not available.Therefore, an alternative solution is proposed for price response estimation at the DSO level, based on the following aggregate price response (APR) function: In Eq. (4b), ∆P α,F t,ξ D is modelled as the product of three terms, consisting of the baseline consumption P base t , the willingness parameter, a α t,ξ D , and a price ratio.a α t,ξ D represents the flexibility of consumers, which varies between 0 and 1.It can be a function of weather conditions, and load/day type, whose values are provided in [22].A certain price response is achieved only when the price signal is bigger than a threshold price, i.e., ∆λ α ξ D , [35].Also, according to [36], the response saturates beyond a certain price signal, i.e., ∆λ α ξ D .In Eq. (4b), parameter γ represents the willingness of consumers to respond to different prices.It is assumed bigger than one as consumers will be more willing to provide flexibility when the absolute value of the price signal is higher [22], [33].
The block-diagram shown in Fig. 5 summarises the simulation model of the entire system that was shown in Fig. 3.As it can be seen from the figure, some assumptions are made: • From the timescale point of view, two sets of simulation models are designed in this paper: hourly (h ∈ N 24 ) and second-by-second (t ∈ τ ).The former timescale is used in consumers' reaction modelling, as it is the only way to account for the consumers RE.Therefore, the ANN model, the MILP model in Eq. (2a)-(2n), and the LP formulation in Eq. (1b)-(1d) are hourly for an entire day.
In the hourly models, prices, ∆Λ α ξ T , are generated and submitted every hour and the disturbance is represented by ω ξ T .The latter timescale, i.e., second-by-second, is used to run simulation for frequency (TSO model) and voltage regulations (DSO model), where the frequency simulation is continuous and PF runs discretely.In this timescale, prices ∆λ α ξ T and ∆λ α ξ D are generated by TSO and DSO, respectively, and submitted every second (i.e., ∆t = 1 second), and the system disturbances are ω ξ T and ω ξ D .In order to solve hourly functions (e.g., MILP, Check the frequency deviation. Issues?  price neutrality function and ANN) within the dynamic simulation model at hour h, as shown in Fig. 5, the price/disturbance is placed in the hourly vector for hour h.While the hourly operation was inevitable because of limited data availability, all algorithms can be performed in higher resolution depending on data availability.As it is shown in Fig. 5, values of the hourly vectors from hour 0 to hour h−1 are known, while values of the future (i.e., from h + 1 to 24) are estimated by prediction.

Yes
• Required flexibility from the consumers throughout the day at the TSO level is estimated with hourly resolution, denoted by This value is updated by the LFC model for hour h, and the new vector will be used as the input in the ANN model.
• It is assumed that delta prices are estimated for the entire day, which is defined in hourly basis and denoted by ∆Λ α ξ T = ∆λ α t,ξ T ∈ R ∶ t ∈ τ .When running simulations at hour h, only present and future time steps are generated by the ANN, while the previous time steps are given by historical values.
• A certain external power disturbance is imposed on the system every ∆t = 30 seconds during dynamic simulations, denoted by ω ξ T = ω t,ξ T ∈ R ∶ t ∈ τ .Only a portion, i.e., χ, of the ω ξ T reaches the DSO level, i.e., ω ξ D .Therefore, the DSO's load is modified according to the ω ξ D disturbance at each iteration.From the figure, it can be seen that consumers' response to delta prices issued by an SO affect the operation of other SOs.This has been modelled properly in the proposed framework.In this study, the DSO and TSO solve their control problems simultaneously.

IV. SIMULATION STUDIES
In this section, simulation studies are carried out to assess the validity of AS4.0 mechanism under different power disturbances.The LFC model is implemented for the Danish transmission system consisting of two areas of 3 GW peak demand each.Actual data from the Elforbrugspanel project [37] is used for the TSO level MILP model.In our simulations, we solve the PF every ∆t = 1 sec.However, depending on the voltage dynamics and regulatory standards at the distribution level, it is possible to update PF results faster or slower.Frequency and/or voltage regulation is initiated if the deviation exceeds a certain threshold.In order to show the impact of load REs on the performance of AS4.0, simulations are repeated for two hours, i.e., h = {5:00, 15:00}.The daily required flexibility (∆P α,C ξ T ) and prices (∆Λ α ξ T ) at the TSO level are generated randomly.Other general simulation parameters are given in Table I, where parameter χ is only needed for simulation purposes.Although imbalance in generation and demand can be caused by multiple sources simultaneously, the original source of the disturbance is irrelevant to the operation of AS4.0 in practice.Simulation models are implemented in MATLAB [38] and GAMS 24.9.1 [39], and the optimisation problem is solved using GUROBI 8.1.0[40].The PF problem at the DSO level is solved using the MatPower 6.0 [41] package in MATLAB.The experiments were carried out on an Intel(R) Core(TM) i7-2600 CPU 3.40GHz processor with 16 GB of RAM.

A. AS4.0 Operation at the TSO Level
In this sub-section, AS4.0 performance and the ANN training will be analysed for frequency regulation at the TSO level.In Table II, simulation parameters of the TSO operation are given.The ANN model is trained using 5000 sets of daily delta prices generated by random uniform distribution.Each delta price set is bounded by the dead-band and saturation price values, as discussed in sub-section III-C2, and has a null sum over the day.The ANN is trained using MATLAB Neural Net Fitting toolbox [42].
1) Artificial neural network performance: To define the optimal ANN structure (i.e., number of neurons in the hidden layer and training sample size), a sensitivity analysis is executed.The results are reported in Table III along with mean squared error (MSE) and correlation coefficient [43] for comparison.Typically, the number of neurons in the hidden layer is between the size of the input and the output [44], and it was changed between 10 to 24 for the sensitivity analysis.Moreover, larger training samples, if providing better statistical representation of the underlying system, can improve ANN performance.The number of samples are varied from 1000 to 5000 in this study.It is clear from Table III that larger training samples and 24 neurons led to the best performance.However, despite the outstanding performance of the ANN, a small modelling error exists (i.e., R=0.98 and MSE=0.01),which indirectly represents the lack of perfect knowledge of the consumers' behaviour.In other words, if we assume that the MILP solutions are the realised flexibility from the consumers in real-time operation, ANN model drifts away from true values by a small amount, as expected in practice.The existence of controller (i.e., LQR) at the TSO level, however, guarantees obtaining frequency regulation over time.
2) Frequency regulation: Table IV shows the system's frequency deviations at the end of each disturbance at steady state.The values are reported for the two areas: Area T 1 in which CGUs provide secondary regulation services, and Area T 2 , where flexibility is provided through AS4.0.Simulations are performed twice; once with CGU-based AS in area T 2 and the second time with AS4.0 in area T 2 , the results of which are compared in Table IV.Overall, the results show that AS4.0 mechanism always outperforms CGU-based AS, reducing the frequency overshoot up to 60%.This is because of the faster response of load flexibility to price signals.From the table, it can be noticed that availability of the consumers' flexibility depends on the time of the day, which depends on the values of P min h,j , P max h,j and P base h,j as well as the RE.The dynamic performance of frequency regulation is shown in Fig. 6.It is clear from the figure that the frequency regulation performance is superior in AS4.0 mechanism in comparison with the CGUbased AS in terms of settling time and overshooting.
3) Price response: In Fig. 7, the delta prices and consumers' reactions are shown for the same simulation study at the TSO level in hour 15:00.From the figure, it can be seen that the TSO obtained 268 MW flexibility from the load demand (in a system with 3 GW peak load) by submitting a positive delta price of 0.84 DKK/kWh.On the other hand, the TSO could manage to increase load consumption by 211 MW through a delta price of -0.82 DKK/kWh.

B. AS4.0 Operation at the DSO Level
In this sub-section, the performance of AS4.0 at the DSO level is examined for voltage regulation.Related parameters for the simulation model at the DSO level are presented in Table V.The PI controller coefficients, i.e., K p and K i , are selected in a way to achieve fastest response without oscillation and large overshoot by trial and error.In the APR function, γ = 2 represents conservative consumers that only respond to large delta prices.1) Voltage regulation: In Fig. 8, voltages at different nodes are shown over time.It can be seen that voltages at several nodes violate the lower limit (i.e., 0.95) at the beginning of each disturbance.However, the delta prices offered by the DSO manage to mitigate the issues in less than 10 seconds in cases.Moreover, the figure shows that the voltage violations are not the same in the two different hours, i.e., h = {5:00, 15:00}, because of the different consumers preferences during the day, as discussed in sub-section IV-A1.In Fig. 9, the number of nodes with voltage issues are plotted along with the frequency response of the system.It is observed that i) the number of buses with voltage issues decreases in time and ii) the frequency evolution in time shows that the DSO operation does not compromise the TSO operation for frequency regulation.Therefore, independent and simultaneous operation of TSO and DSO is indeed plausible without jeopardising the system stability.From Fig. 9, it can Nodes Nodes nodes Fig. 9. Number of nodes with voltage violations along with the system's frequency at hour 15:00.also be seen that the number of nodes with voltage issues does not increase when the system frequency is higher than 50 Hz.This is due to the fact that the nodal voltages of the original model are close to the lowest admissible levels.Since the upper voltage limit is set to 1.05 p.u., nodal voltages never reach that limit even in the worst light loading conditions in our simulation studies.Nevertheless, the proposed formulation at the DSO level can correct both upper and lower voltage violations.
2) Price response: In Fig. 10, the delta prices and corresponding consumers' response are provided for the two clusters at the DSO level.When a voltage violation occurs, the PI controller starts generating a price signal that keeps increasing until the voltage issues are resolved within the cluster.The delta price will be maintained until the power disturbance disappears or another disturbance hits the network.During this time, the PI controller generated a positive delta price that increased to 0.57 DKK/kWh to obtain 202 kW of decrease in consumption to regulate voltage in those buses.This operation did not have any negative impact on the rest of the system.
Note that the changes in consumers' demand caused by delta prices are intended to counterbalance the original disturbance, which initiated the imbalance between generation and demand in the first place.Therefore, the varying consumers' behaviour improves system stability by mitigating the external disturbance.To that end, Figs.7 and 10 show the amount of flexibility that is achieved at the transmission and distribution levels by a given set of delta prices to step-wise disturbances in the system.Also, please note that the overall change in demand is much smaller than the total load in the system.Moreover, it can be seen in Fig. 10 that the delta prices never reach the pre-defined ceiling and floor prices at the DSO level.Nevertheless, saturated delta prices might happen in larger distribution networks than the one considered in this study.In such cases, an anti-windup PI controller [45] can be used instead of the PI controller to prevent saturation under such rare events.

V. CONCLUSIONS
This paper provides a control-based solution for the provision of AS from consumers, which is called AS4.0.In this alternative approach, SOs at different levels of the grid submit time-varying prices to the pool of consumers at their territory to address different operational issues.Consumers receive price signals by their EMS and react to minimise their electricity cost.The proposed AS mechanism is explained and appropriate simulation models and estimation algorithms are developed to examine the proposed framework.At the transmission level, price-response of consumers is modelled using an MILP formulation while accounting for loads' RE.Then, an ANN model is developed based on the MILP problem to generate appropriate prices to achieve the required flexibility.At the distribution level, aggregate price response of consumers is modelled through an APR function and appropriate delta prices are generated by a PI controller.Simulation results prove that both TSO and DSO are able to resolve operational issues through AS4.0 approach simultaneously, and the performance of frequency regulation is significantly better through AS4.0 compared to the conventional AS provision.While the simulation results in this paper are promising, further studies using high-resolution data, larger power system models, and real-world implementation of the proposed framework in an islanded microgrid setting are required.Moreover, possibility of conflict and competition between TSO and DSO in obtaining flexibility from the load demand should be further investigated and appropriate coordination methods should be developed.Finally, further analytical and simulation studies are needed to gain insight into the precise conditions under which system stability can be guaranteed under the proposed control architecture.
Transmission and distribution systems.F Demand flexibility resources.α Type of regulation, i.e., up-(i.e., α = u) or down-(i.e., α = d) regulation.B.Sets: T Set of time with second-to-second resolution, indexed by t.H Set of hourly time periods, indexed by h.J Set of end-users' categories, indexed by j.Ξ Set of system operators' levels, indexed by ξ.C.Parameters: ω t Injected power disturbance at time t [W].ω t,ξ Power disturbance measured at time t at level ξ = T , D [W]. a α h,j,ξ T The coefficient of consumers' willingness at TSO level for end-users' category j and regulation type α at time h [p.u.]. a α t,ξ D The coefficient of consumers' willingness at DSO level for regulation type α at time t [p.u.]. a α j,ξ T Maximum willingness coefficient of consumers at TSO level from end-users' category j for regulation type α [p.u.].

j
for regulation type α at level T of end-users' category j [kW/h].n α Activation times of flexibility provision at level T from end-users' category j for regulation type α. d α j , d α j Minimum and maximum duration of AS provision by end-users' category j when activated to provide regulation type α [h].R j Maximum duration of rebound effect for endusers' category j [h].λ base Baseline electricity price [DKK cent/kWh].∆λ α ξ , ∆λ α ξ Minimum and maximum time-varying electricity price (called delta price) for regulation type α at level ξ = T , D [DKK cent].P base h,j

Fig. 7 .
Fig. 7. Delta prices and the corresponding response from consumers at the TSO level at hour 15:00.

Fig. 10 .
Fig. 10.Delta prices and corresponding flexibility at the DSO level at hour 15:00.
Delta price generated by system operator at level ξ = T , D for regulation type α at time t [DKK cent/kWh].
kW].P max h,j , P min h,j Maximum and minimum electricity consumption for end-users' category j at time h [kW].T Modified delta price ensuring daily price neutrality at TSO level at time t for regulation type α [DKKcent/kWh].L Accumulated price error from perfect neutrality [DKKcent/kWh].
ξT 1 Fig. 4. LFC model of a two-area power system at the TSO level: Conventional model in black and red, AS4.0 model in black and blue.one of the two control areas.The conventional secondary loop is replaced by demand's contribution to frequency regulation in area T 2 , to evaluate AS4.0 performance independently.In the modified LFC model, ∆P α,C t,ξ T 2 is the required control effort for frequency regulation in area T 2 .The TSO generates delta prices based on ∆P α,C t,ξ T 2

TABLE II TSO
PARAMETERS IN THE SIMULATIONS STUDY.

TABLE III SENSITIVITY
ANALYSIS FOR ANN MODEL STRUCTURE.

TABLE IV PERFORMANCE
BENCHMARKING FOR AS4.0 AND CGU-BASED AS.
Frequency profile of the system in T2 area at hour 15:00.(a) Overall frequency.(b) Zoomed-in part to better visualise the dynamics.

TABLE V DSO
PARAMETERS IN THE SIMULATIONS STUDY.