Day-Ahead Scheduling and Online Dispatch of Energy Hubs: A Flexibility Envelope Approach

Energy hub is a micro energy system in industrial park or residential area. It makes full use of renewable resources through multi-energy integration. Proper coordination among generation units, battery and thermal storage devices provides additional flexibility to cope with volatile renewable power. Due to the lack of accurate predictions, the operator uses an interval estimation for day-ahead scheduling and makes online decisions in response to real-time observations without future renewable output values. This paper proposes an envelope-based approach which offers a thorough coordination between day-ahead scheduling and real-time dispatch. In the day-ahead stage, a flexibility envelope for admissible renewable power and action envelopes for controllable devices are determined from a convex quadratic program. In the real-time operation stage, online actions for generation facilities and storage devices are determined by a partial-quantile policy based on the current renewable output and those envelopes. Since the flexibility envelope intentionally caters to the interval estimation of renewable output, the online decisions would be less myopic. Test results demonstrate that the proposed method has an optimality gap smaller than 5% compared to the hindsight optimum, which is comparable to the model predictive control (MPC) method with exact predictions in the next 4–5 hours.

Day-Ahead Scheduling and Online Dispatch of Energy Hubs: A Flexibility Envelope Approach Songjie Feng , Graduate Student Member, IEEE, Wei Wei , Senior Member, IEEE, and Yue Chen , Member, IEEE Abstract-Energy hub is a micro energy system in industrial park or residential area.It makes full use of renewable resources through multi-energy integration.Proper coordination among generation units, battery and thermal storage devices provides additional flexibility to cope with volatile renewable power.Due to the lack of accurate predictions, the operator uses an interval estimation for day-ahead scheduling and makes online decisions in response to real-time observations without future renewable output values.This paper proposes an envelope-based approach which offers a thorough coordination between day-ahead scheduling and real-time dispatch.In the day-ahead stage, a flexibility envelope for admissible renewable power and action envelopes for controllable devices are determined from a convex quadratic program.In the real-time operation stage, online actions for generation facilities and storage devices are determined by a partial-quantile policy based on the current renewable output and those envelopes.Since the flexibility envelope intentionally caters to the interval estimation of renewable output, the online decisions would be less myopic.Test results demonstrate that the proposed method has an optimality gap smaller than 5% compared to the hindsight optimum, which is comparable to the model predictive control (MPC) method with exact predictions in the next 4-5 hours.

NOMENCLATURE
Yue Chen is with the Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong, SAR, China (e-mail: yuechen@mae.cuhk.edu.hk).
Color versions of one or more figures in this article are available at https://doi.org/10.1109/TSG.2023

I. INTRODUCTION
T HE EXTENSIVE use of fossil fuels causes massive carbon emissions and climate change.In order to achieve sustainable development, countries around the world have been promoting the wide deployment of renewable energy resources [1], [2].Electricity and heat are two main energy carriers at the demand side.Energy hub is a promising framework for residential or industrial energy systems to achieve efficient use of renewable resources through multienergy integration [3].Since the waste heat from producing electrical power can be reused for heating purposes, combined heat and power generation enjoys higher efficiency [4].Furthermore, the battery and thermal storage devices in an energy hub provide additional flexibility to cope with volatile renewable power [5].The operation of energy hub has attracted wide attention.Existing research on energy hub operation can be classified into three categories, depending on how the uncertain factors are modeled.c 2023 The Authors.This work is licensed under a Creative Commons Attribution 4.0 License.
For more information, see https://creativecommons.org/licenses/by/4.0/Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
The first class focuses on day-ahead planning and relies on two-stage optimization techniques, such as stochastic optimization [6], [7], robust optimization [8], [9] and distributionally robust optimization [10], [11].Stochastic optimization models uncertainty via sampled scenarios and minimizes the expected cost.Robust optimization uses a pre-specified set to bound the fluctuation of uncertainty and minimizes the worst-case cost.Distributionally robust optimization assumes the probability distribution is inexact and minimizes the cost in the least-favored probability distribution, and hence inherits advantages from both stochastic and robust optimization approaches.However, two-stage optimization assumes that the second-stage decision is made after all uncertain parameters have been observed, this is not the case in energy hub operation, because in real-time dispatch, actions in a period must be determined based on the observed renewable output up to that period without knowing the output in future periods, which is called casuality or non-anticipativity [12], [13].For such problems with sequentially revealed uncertainty, the real challenge is how to dispatch generation and storage devices online without using future renewable power information.
The second class employs a dynamic programming framework and resolves the non-anticipativity issue.In dynamic programming, the future impact of the current dispatch decision is taken into account in the cost-to-go functions, which are trained offline using historical data.Then, online decisions can be determined based on the current observation of uncertainty and the Principle of Optimality by Bellman.Reinforcement learning is a model-free dynamic programming approach and has been applied in power grid operation [14].Because the physical operating constraints are not explicitly modeled in the neural network based actor, reinforcement learning requires sufficient interaction with a simulator to train a usable dispatch policy.Among various model-based dynamic programming methods, stochastic dual dynamic programming (SDDP) is the most widely used one [15], [16].SDDP leverages the linear structure of the dynamic programming problem, whose value functions are known to be convex and piecewise linear, and constructs under-estimators via forward and backward procedures.If the probability distribution of uncertainty is not known, robust dual dynamic programming (RDDP) [17] is a good alternative.RDDP uses an interval of uncertainty as the input, which is easy to obtain.RDDP has been used in the operation of residential energy systems [18] and largescale power grids [19].A common difficulty in dynamic programming based approaches is the construction of cost-togo functions.Although the optimal value functions of linear dynamic programming programs are known to be convex and piecewise linear [15], they may contain a large number of pieces, which brings a significant computation burden, especially for problems with multiple energy storage units or a long operating horizon.In addition, the cost-to-go functions must be carefully initialized; otherwise, the subproblems in the forward iteration step may be unbounded.Some improved dynamic programming based methods are also proposed in recent years.The hybrid approximate dynamic programming method in [20] and [21] which combines model predictive control and dynamic programming has better performance due to the exploitation of ultra-short-term forecast.
Lyapunov optimization [22] is a prevalent method to design online policies via queue stability.By defining a virtual queue for energy storage level, a drift-plus-penalty cost is minimized, balancing instance cost and queue length which may impact the future.This method is applied in the online operation of remote power plants [23] and energy hub systems [24].Since Lyapunov optimization uses neither forecast nor historical data, the online policy can be regarded as an adaptive greedy algorithm and may be somewhat myopic if the uncertainty changes drastically over time, although the optimality gap of Lyapunov policy can be theoretically quantified.
The importance of coordinating day-ahead scheduling and real-time dispatch has also been recognized in the existing work.Existing literatures on joint scheduling of both dayahead and real-time stages can be mainly divided into two categories.The first category adopts a following-the-reference strategy.The day-ahead problem determines reference trajectories of controllable units based on forecast information, and the real-time actions endeavor to follow reference profiles.For example, integration of day-ahead scheduling and realtime dispatch for an integrated energy system is studied in [25]; reference trajectories are determined from the dayahead problem based on wind power prediction, and real-time dispatch is formulated as a rolling-horizon problem.A penalty term is added into the dispatch objective to avoid an excessive deviation from the day-ahead plan.Integrated scheduling and dispatch of the multi-energy system is discussed in [26], the reference is a plan of hydrogen storage level, which bounds the actual hydrogen storage level within a neighborhood of its periodical targets.The performance of the following-thereference strategy largely depends on the forecast quality.When the forecast has large errors, following the reference may not be a good choice.
The second category realizes the integration of day-ahead scheduling and real-time operation through multistage robust optimization.In [27], the unit on-off status is determined in the day-ahead stage; the real-time adjustment is assumed to be affine functions in the forecast error, which is known as affine policy, and the coefficients of affine functions are also optimized in the day-ahead unit commitment problem.In real-time dispatch, when actual renewable output is known, the incremental output of generators is arithmetically calculated via affine policy, constituting a closed-loop control.The affine policy is also applied to multi-period economic dispatch [28].As an explicit policy, the affine approach ensures non-anticipativity through an affine relation between dispatch actions and uncertain parameters.However, it could be suboptimal and may sacrifice system flexibility [29].The implicit policy imposes time-varying lower and upper bounds on decision variables that couple the constraints in multiple time periods [30].For example, in [31], time-varying bounds of thermal unit output are determined in the day-ahead stage, so that real-time dispatch is decoupled over time, while the feasibility of inter-temporal constraints is naturally met without the need of knowing future realizations of uncertainties.Above methods are mainly used in large-scale power systems, where conventional thermal and hydro units still account for the majority of installed generation capacity, so there is adequate flexible resource to compensate for volatile renewable output.However, in the renewable-dominated energy hub, we assume generation and storage capacities are limited, and renewable energy accounts for the majority of energy demand.So there is not enough backup power.Such a fact makes the robust optimization strategy not very suitable for the operation of renewable-dominated energy hubs, because the robust model would be infeasible in most cases, and thus no operation policy can be provided.
This paper aims to develop an easy-to-use method that integrates day-ahead scheduling and online dispatch even if there is neither adequate historical data nor forecast of renewable power.The proposed method is inspired by a recent work in [32], in which the aggregated flexibility of a distribution network is proposed.However, renewable generation is treated as a deterministic source in [32].Similar idea is adopted in [33] and the flexibility offer is computed via distributed optimization.We further extend the flexibility concept in [32], [33].The contribution of this paper is twofold.
1) An envelope-based approach is proposed for integrating day-ahead scheduling and real-time dispatch of energy hubs.In the day-ahead stage, a flexibility envelope for renewable generation and action envelopes for controllable devices are determined from a convex quadratic program.The flexibility envelope caters to the trend of supply/demand volatility.In the real-time operation stage, online actions for generation facilities and storage devices are determined by a partial-quantile policy based on the current observation of renewable power subject to the constraints offered by action envelopes, so as to improve online adaptability to real-time data and reduce the myopia of online decisions without future information.There are two significant revamps compared to [32], [33].In the dayahead stage, the uncertainty of renewable output is considered, leading to a completely different view on renewable resources and formulation of flexibility envelope optimization.In the real-time stage, the quantile policy in [32], [33] determines the actions of all controllable devices through a weight parameter and two bounds.In the proposed method, the actions of generation devices, which incur operation costs, are optimized; only cost-free storage operation strategies are determined by the quantile.So the latter enjoys better economic performance.The proposed method directly applies to cases in which renewable power, electric load demand and electricity price are all uncertain, while most existing works consider either price uncertainty or power uncertainty.
2) The feasibility of the non-anticipative quantile policy is proven.For the ease of discussion, ideal energy storage models are used to develop the proposed method.When nonideal storage models are considered, we expound how to implement storage dispatch without simultaneous charging and discharging, and thoroughly analyze the impact of energy loss during charging and discharging on the proposed method.Such a treatment provides useful insights on controlling nearly ideal energy storage (cycling efficiency is close to 1), such as battery and thermal storage, without imposing strict complementarity between charging and discharging in the optimization problem.The rest of this paper is organized as follows.The configuration of energy hub and the operation problem are formally stated in Section II.The envelope-based approach for integrating day-ahead scheduling and online dispatch is developed in Section III.Results of numerical tests are reported in Section IV.Finally, conclusions are drawn in Section V.

II. SYSTEM AND PROBLEM DESCRIPTION
This section gives the configuration of the energy hub and the motivation for integrating day-ahead scheduling and online dispatch.We suggest a general framework that unifies the operation problem under uncertainty.It underpins a fair comparison of different policies via simulation with the same manner of real system operation.

A. Configuration of Energy Hub
The basic architecture of an energy hub is shown in Fig. 1.It consumes electricity and natural gas to meet electric power and heat demands.At the electrical side, power is generated from the renewable plant and co-generation unit, or purchased from the main grid.At the thermal side, heat is produced by the co-generation unit and a boiler.Battery and thermal storage endow the system with flexibility to compensate for the fluctuation of renewable energy and demands.To reduce the carbon emissions, we assume that renewable energy accounts for the majority of demand; the capacity of co-generation unit is less than half of the peak demand; the capacities of energy storage units are also limited.This is a salient feature of renewable-dominated energy hubs in the low-carbon society.
Three special points in Fig. 1 are explained as follows: 1) Renewable-system interface point: the green dot in Fig. 1.This interface point divides the system into an uncertain part which includes renewable generation and a controllable part which consists of generation/storage facilities and loads.
2) Generation-demand splitting point: blue (electrical side) and red (heat side) dots in Fig. 1.Such points split the electrical and thermal systems into the generation side and the demand side.The generation side contains the co-generation unit, boiler, renewable plant and the power grid.The demand side consists of energy storage devices and loads.
3) Grid-connection point: the purple dot in Fig. 1.This point represents a feeder or a transformer, through which the energy hub connects to the power grid.
The energy hub studied in this paper is an abstraction of a multi-energy system at the industrial-park level or residential level, which is a classical concept in [34] and [35].Such systems are small in scale, and network constraints are typically not the main bottleneck.The main difficulty of operating a renewable-dominated energy hub is to maintain power balance without renewable power forecasts under limited generation and storage capacity.

B. Problem Statement
We make the following general assumptions: 1) Renewable power is free.The natural gas has a fixed price known in advance.The electricity price from the main power grid could be fixed or time-of-use tariff which is known in advance, or a real-time price which changes over time and is unknown day-ahead.In the latter case, we say that electricity price is uncertain in the day-ahead stage.In realtime operation, the current price is known, based on which dispatch actions are determined.
2) An interval estimation of the renewable power over the next day (called the uncertainty envelope) is available.It can be a confidence interval prediction of renewable output, or simply the envelope of historical trajectories of renewable power, or can be set by experiences.
3) The electric load can be either deterministic or uncertain.For the former case, the load profile is known in the dayahead stage.In the latter case, the uncertainty envelope is estimated for the renewable output minus load forecast error.Compared to electricity, heating system has large thermal inertia, and appliances such as warmer are typically not very sensitive to the small changes in temperature, unlike the power system where the frequency is sensitive to power imbalances.This means that heat can be balanced in a relatively longer timescale, so we neglect the uncertainty of heat demand.
Based on the requirements in real applications, the general framework of energy hub operation consists of: 1) Day-Ahead Scheduling: The input includes component data and the uncertainty envelope; because of the volatility of renewable output, no fixed strategy can be given a priori, and the day-ahead decisions are not concrete control commands or reference trajectories.Instead, day-ahead strategy ought to offer real-time dispatch a global view to reduce myopia.In this work, the day-ahead scheduling outcomes are necessary parameters {S t } T t=1 in real-time dispatch policies {μ t (•, S t )} T t=1 ; each μ t is a function of S t and the uncertain parameters.
2) Real-Time Dispatch: The input is the latest observation of the uncertain parameter, denoted by w t , and the action to be deployed is a t = μ t (w [t] , S t ), where w [t] = [w 1 , . . ., w t ] is the observation history up to the current period t.The policy μ t can be an explicit function or implicitly determined from a certain optimization problem.
Some existing methods also belong to this framework.Take the affine policy approach in [27] for an example, μ t (w [t] , S t ) is restricted to be affine functions in w [t] , and the gain matrices are S t ; a simplified affine policy in [27] is memoryless and can be expressed by μ t (w t , S t ).For the implicit policy approach in [30], S t contains time-varying bounds of state variables.Deep reinforcement learning uses neural network to approximate the memoryless policy, and S t denotes the parameters of the neural network; if an infinite horizon model is used, the optimal policy is stationary, which means that the parameters do not change over time, and the policy can be expressed as μ(w t , S); the Lyapunov driftplus-penalty technique in [23], [24] also gives a stationary policy.In the proposed method, the actions of generation facilities are given by an optimization problem with time varying parameters S t and w t in the current period, and energy storage devices are dispatched according to an explicit policy.Hence, the policy has a form of μ t (w t , S t ); it is memoryless but non-stationary.
3) Objective Function: The operating cost is a function of (a 1 , . . ., a t ), denoted by t f t (a t ).The operation problem aims to minimize the expectation E w [ t f t (μ t (w [t] , S t ))] by designing parameter S t and policy μ t .
The above framework meets practical requirements: the day-ahead scheduling plan and the real-time dispatch are seamlessly integrated; the rough tendency of the uncertainty envelope captured in the day-ahead problem helps circumvent myopic real-time actions; according to the data used in policy μ t (w [t] , S t ), the real-time action a t is non-anticipative, and thus casuality is respected.
Note that this is only a conceptual framework.Since the distribution of w is not known, the expectation in the objective function is never computed or evaluated in the proposed method.To achieve an economic operation, we technically divide the operation into a day-ahead stage which determines the parameters {S t } T t=1 in the policy {μ t (w [t] , S t )} T t=1 and a realtime stage which deploys actions based on the newly observed price and net demand information.
There are two difficulties in solving the operation problem. 1) Unlike solving a stochastic optimization or robust optimization problem, in which the outcome is a numeric solution, the output here is a function μ t (w [t] , S t ) in each period.There is no systematic method for optimization over functions.A good policy μ t must consider the particularity of the problem.
2) Although energy storage devices offer flexibility in system operation, their capacities are limited.The use of storage at present influences how it can be used in the future.However, the renewable output in the future is unknown, imposing another difficulty on problem solution.
Indeed, it is difficult to find the ideal optimal policy which gives the identical result compared to the hindsight problem, which is the deterministic problem where uncertain parameters are exactly known in advance.We aim to find an acceptable policy that has a satisfactory performance.We coordinate decisions from three aspects: first, an appropriate mechanism that coordinates the scheduling plan and dispatch policy; second, temporal coordination of policies over time; third, spatial coordination among system components.

III. ENVELOPE-BASED OPERATION METHOD
In this section, component models will be presented first, followed by the day-ahead scheduling problem which provides Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.flexibility envelope and action envelopes; finally, the real-time dispatch policy is given.

A. Component Models 1) Co-Generation
Unit: We consider extractioncondensing co-generation unit.The feasible operating region is a polytope [3], as shown in Fig. 2. The region is constrained by the maximum/minimum fuel generation line AB/CD, and the maximum heat extraction line BC.The electric output p g t and thermal output q g t can be expressed by the convex combination of extreme points (H where weight coefficients π k,t determine the operating point (p g t , q g t ) in the polyhedral feasible set; the fuel cost is where a, b, c, d, e, f are constant coefficients, and C g t is known to be a convex function in both inputs, as e is typically small compared to other coefficients.The co-generation unit has fast ramping capacity so the ramping constraint is omitted.
2) Boiler: The boiler consumes natural gas and produces steam or hot water.The thermal output q b t satisfies where Q b max /Q b min are the upper/lower limits of heat output.The cost of boiler is a linear function where θ is a constant coefficient.

3) Energy Storage Devices:
The dynamics of storage levels in ideal battery (E bs t ) and thermal storage (E hs t ) are given by: where t is the duration of period t; the net charging power p bs t /q hs t (positive for charging and negative for discharging) and storage levels are bounded variables subject to: where P bs max /P hs max and E bs max /E hs max are power and energy capacities of battery/thermal storage; In practice, the minimum storage level is greater than zero to avoid over depletion.We intentionally offset this point to zero in (3c), which means that we only consider usable intervals of storage devices.For ease of exposition, lossless energy storage model is used in model (3).Real battery and thermal storage units are nearly ideal with a cycling efficiency close to 1. Nevertheless, using a lossy energy storage model in (3) does not change the proposed method.In the proof of main results, we also consider lossy energy storage.In a later section, we will expound how to control non-ideal energy storage in real-time dispatch circumventing simultaneous charging and discharging.
The controllable components endow the system with a certain degree of flexibility to respond to imbalanced power.In an energy hub, the output of co-generation plant and boiler in each period is independent.However, since the energy capacities of battery and thermal storage are finite, the usage of battery or thermal storage in period t impacts how it can be used in future periods.To overcome this difficulty, we resort to a flexibility envelope; it is the envelope of all renewable power trajectories, so that for any trajectory the system can be dispatched through an online policy without forecast.The day-ahead scheduling problem allocates flexibility over time, and the cost is optimized in the real-time stage.The reason for such a decomposition is that prices may be unknown in the day-ahead stage, so cost is optimized after the actual price is revealed in the real-time stage.Nonetheless, the proper allocation of flexibility can help the use of renewable energy and reduce the amount of energy purchased from the power grid, which also contributes to the cost reduction.As flexibility scheduling requires the information of renewable power trajectory, this must be done in the day-ahead stage.

B. Day-Ahead Scheduling
In the energy hub, let p d0 t and q d0 t be the total demands of electricity and heat at the respective demand points: where p L t /q L t is the fixed electric and heat demand in period t.Since storage charging strategy is adjustable, p d0 t and q d0 t are controllable.The power balance condition at the renewablesystem interface point is where p 0 t is the need for renewable power.By motivation, the flexibility envelope consists of the intervals of {p 0 t } T t=1 and reflects the ability of the system to track the volatile power in the uncertainty envelope.
To formalize the above idea, let p g = {p g t } T t=1 and q g = {q g t } T t=1 represent vectors of electric and thermal outputs of the co-generation plant, respectively.Similarly, define p bs , q hs and q b for energy storage units and boiler.Vector x = p g , p bs , q g , q b , q hs Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.collects all the decision variables.The uncertainty envelope is where w l t /w m t is the minimum/maximum renewable power in period t estimated day-ahead.The lower/upper trajectory w/ ŵ is We associate two sets of variables in vectors x and x with the lower trajectory [p 0 t ] T t=1 and upper trajectory [p 0 t ] T t=1 of the flexibility envelope which should be a subset of the uncertainty envelope W.
To increase the ability of the system to mitigate uncertainty, we have to widen the flexibility envelope F as much as possible while keeping the width evenly distributed over all time slots.To this end, we build the following problem: max x satisfies (1a), (2a), ( 3), ( 4), (5) (7g) x satisfies (1a), (2a), ( 3), ( 4), (5) (7h) In the objective function (7a), the width of the flexibility envelope, quantified by the weighted sum of (p 0 t − p0 t ) in each single period, is to be maximized.The convex quadratic term can reduce the variance of the widths, making the flexibility evenly allocated over time, and the parameter ω = 10 −3 .The role of the quadratic term is analyzed in Appendix-B.
As p0 t and p0 t are the lower and upper bounds of flexibility envelope, (7b) must hold.Because p 0 t is the desired renewable power, we actually design the shape of the flexibility envelope to cater to the shape of uncertainty envelope, which is done in (7c), where 0 < α < β ≤ 1 are adjustable parameters.These two constraints require that the lower trajectory p0 and the upper trajectory p0 of flexibility envelope reside in the interval [w l t , αw The same explanation holds for the thermal energy storage in (7e).However, inequalities in (7e) have opposite directions compared to those in (7d).For a co-generation unit whose heat output is proportional to the electric output, when the renewable output is high, the battery tends to charge more power and the co-generation unit needs to generate less power and heat, so the thermal storage has to release heat.The rule also applies to the extraction-condensing unit although its heat and power output range is more flexible.
In (7d)-(7f), the upper trajectories of battery storage level Êbs t and charging power pbs t correspond to the upper trajectory of renewable generation p0 t .The rationale is that when the renewable generation is abundant, the system tends to generate less electricity and charge more electricity into the battery storage.Therefore, p0 t is associated with pbs t and Êbs t .Finally, (7g) and (7h) define component operating constraints corresponding to the upper trajectory ŵ and lower trajectory w, respectively.In (7g) and (7h), equation ( 4) is used to provide the definitions of p d0 t and q d0 t .The components (p d0 t , pd0 t , qd0 t , qd0 t ) at the optimal solution offer important information that prevents real-time dispatch from being myopic.
By solving the day-ahead scheduling problem (7), we obtain the flexibility envelope (p 0 , p0 ), action envelopes of storage devices (p bs , pbs ), (q hs , qhs ), and action envelopes of generation devices (p g , pg ), (q g , qg ) and (q b , qb ).They are collected in the parameter set for designing the online dispatch policy.
Compared to stochastic programming models, the day-ahead problem ( 7) is a convex quadratic program that contains only two scenarios, so its size is small.Compared to robust models, problem (7) is deterministic, while robust optimization methods often entail sophisticated duality transformation and may require solving challenging bilinear programs.Moreover, the uncertainty set must be properly given.This may be difficult for a renewable-dominated energy hub.Problem (7) could be infeasible if the uncertainty envelope W is too wide and α and β are not well chosen, implying a lack of adequate flexibility.To increase flexibility, we need larger generation and storage capacities, which pertain to the planning problem and are beyond the scope of this paper.At the algorithmic level, we can adjust the values of α and β to make problem (7) feasible.
The above discussion assumes that uncertainty stems from renewable generation.If the electric load is also uncertain, we concentrate the uncertainties of renewable output and load and form an equivalent uncertainty envelope.Specifically, assuming that: where p L0 t is the deterministic part and p L t is uncertain.In the power balance condition (5), renewable power and electric demand are additive.Their time series for the next 24 hours are not known in the day-ahead stage, but their current values can be observed and used in real-time dispatch problem.In Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
where p 0 t − p L t is treated as a new uncertain variable, for which the flexibility envelope is constructed.The components at the lefthand side of (10) are uncertain, while the components at the righthand side of (10) are either controllable or constant.

C. Quantile Policy for Real-Time Dispatch
Based on {S t } T t=1 obtained in day-ahead scheduling stage, a casual quantile policy for real-time dispatch is established.The flowchart is given in Algorithm 1.
In brevity, if the renewable generation w t is greater than the upper bound or smaller than the lower bound of the flexibility envelope, then in the former case the excessive renewable power is curtailed, and in the latter case the insufficient power is bought from the power grid.Otherwise, if w t resides inside the flexibility interval [p 0 t , p0 t ] in period t, we have a quantile λ t ∈ [0, 1], which is used to determine the outputs of cogeneration unit and boiler, as well as the charging strategies of battery and thermal storage.In the quantile policy, no future information on renewable generation is needed, so this policy is non-anticipative, which is an appealing feature.
Proposition 1: If renewable power trajectory w ∈ F, then the quantile policy provides a sequence of viable actions.
Here viability means that the operating constraints of all controllable facilities are satisfied.This is straightforward for co-generation unit and boiler whose constraints are decoupled over time.In particular, storage charging/discharging power and storage levels also obey the constraints in (3).The proof is given in Appendix-A.Compared to [32], [33], we also discuss non-ideal storage devices whose charging/discharging efficiency is slightly smaller than 1.Details can be found in Subsection F.
In Algorithm 1, when w t > p0 t , the extra renewable power is curtailed.Clearly, this strategy is not optimal, because we can make use of the extra renewable power by reducing the output of co-generation unit.To improve the cost performance of online dispatch policy, we notice two facts: 1) Costs originate from the natural gas consumption of the co-generation unit, the boiler, and purchasing electricity from the power grid.Actions at the demand side, which refer to the operation of energy storage devices, are free of charge.
2) The challenge for constraint satisfaction in the online setting stems from the time coupling constraints of energy storage and the lack of future information about uncertainty.There is no inter-temporal constraint at the generation side.
The above two facts motivate an improved partial-quantile policy which minimizes the cost of generation devices while preserving non-anticipativity.The key is to modify the control actions at the generation side and retain the quantile policy for energy storage devices.

D. Partial-Quantile Policy for Real-Time Dispatch
Recall the definition of controllable electrical/thermal demands in (4) and the quantile policy, we give the following proposition: Proposition 2: The policy gives a sequence of viable actions for battery and thermal storage devices for any λ e t ∈ [0, 1] and λ h t ∈ [0, 1].Here, viability means that the dispatched power and storage level of battery and thermal storage satisfy the constraints in (3).The proof follows a way similar to that in the Appendix-A, except for considering battery and thermal storage separately.
The quantiles separate the generation side and demand side.At the generation side, the generation dispatch and energy purchase strategies can be determined by solving a convex optimization problem: The objective (12a) is a convex quadratic function, the former two terms are costs of co-generation unit and boiler, as in (1b) and (2b); the third term is the payment for purchasing power from the power grid, where γ t is the electricity price.For constant and time-of-use pricing, γ t is a known constant.For real-time pricing, γ t in the current period is known; unknown prices in future periods do not appear in problem (12).Constraint (12b) collects the operating constraints of co-generation unit and boiler in (1a) and (2a).The first inequality in (12c) restricts the amount of electricity that can be purchased from the power grid, which is bounded by P gd max , the capacity of the feeder or transformer.The second inequality in (12c) limits the value of the dispatched renewable power p r t .Constraints (12d)-(12e) restrict the power and heat generation within the adjustable ranges [p d0 , pd0 ] and [q d0 , qd0 ] determined in the day-ahead problem.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
4: The charging strategies of storage devices are determined through quantile policy (11).
Fluctuations of renewable power and load are treated in different ways.Excessive renewable power can be curtailed, which is modeled in (12c).In this constraint, the dispatched renewable power p r t can vary from 0 to the real-time renewable output w t , which is not directly constrained by the upper bound of flexibility envelope p0 t .Although no penalty cost is given for renewable spillage, since renewable energy is free in (12), the system always prioritizes the use of renewable energy and reduces generator output when w t exceeds p0 t .As a result, renewable energy is not curtailed unless the generator has reached its minimum output and power imported from the grid is zero.Load shedding is not an option, and insufficient supply can be purchased from the power grid, unless the feeder capacity P gd max is not large enough.The flowchart of real-time dispatch is given in Algorithm 2. In step 3, λ e t ∈ [0, 1] and λ h t ∈ [0, 1] clearly hold due to constraints (12d)-(12e).The viability of storage dispatch actions is guaranteed by Proposition 2. Because only dispatched power of energy storage is calculated by quantile, while generator output is optimized in (12), we call this partial-quantile policy.As the partial-quantile policy can make better use of renewable energy through constraint upper bound in (12c), as explained above, the partial-quantile policy enjoys a better economic performance.

E. Overall Framework
The overall framework of the proposed method is summarized in Fig. 3.In the day-ahead stage, the input is the prediction range of renewable power [w l t , w m t ] T t=1 , i.e., the uncertainty envelope W. We conduct (7) to widen the flexibility envelope F while catering for the shape of W. The day-ahead scheduling provides the optimal flexibility envelope [p 0 t , p0 t ] T t=1 and action envelopes of devices.These operating ranges are collected in S t defined in (8) and passed to intraday operation.In real-time dispatch, the generation-side dispatch actions {p g t , q g t , q b t , p gd t , p r t } are obtained by solving problem (12), and the actions of energy storage units are determined from (11) and (13a).Since the bounds in S t vary over time and have been optimized in the day-ahead problem, adding these bounds in the real-time problem can prevent myopic decisions without forecast.

F. Dispatching Non-Ideal Energy Storage
When the energy loss during charging and discharging is taken into account, the storage level dynamics are revised to: where η c /η d and σ c /σ d are charging/discharging efficiencies of battery and thermal storage, respectively.p bs,c t /p bs,d t and q hs,c t /q hs,d t are charging/discharging power of battery and thermal storage in period t, which are constrained by: 0 ≤ p bs,c t ≤ P bs max , 0 ≤ p bs,d t ≤ P bs max , ∀t 0 ≤ q hs,c t ≤ P hs max , 0 ≤ q hs,d t In the day-ahead problem, energy storage model (3a) and (3b) are replaced with (14a) and (14b), and the day-ahead problem (7) remains a convex quadratic program.Although we do not use binary variables to impose strict complementarity between p bs,c t /p bs,d t and q hs,c t /q hs,d t , this does not impact the implementation even if they are not complementary because they are not real dispatch actions.The day-ahead problem only provides flexibility envelope and action envelopes.In realtime operation, when the quantile λ e t and λ h t are determined from (13a), the charging/discharging power of battery and thermal storage are: which obey constraint (3c), as stated in Proposition 2.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
Although charging power and discharging power in (15a) and (15b) may not be strictly complementary, but we can choose to either charge or discharge energy storage based on the result.Take battery for an example, if p bs,c t and p bs,d t in (15a) are positive at the same time, we let where p c t and p d t are actual charging and discharging power of battery.This strategy prevents simultaneous charging and discharging, but the actual storage level is slightly different from that computed from (15c).
When p bs,c t ≥ p bs,d t , the discrepancy E bs t is: , the discrepancy is: In both cases, the discrepancy is positive, implying a higher storage level than expected.A slightly higher storage level is desired, because excessive renewable power can be freely curtailed, but insufficient supply must be bought from the power grid which incurs a certain cost.So we accept such a small discrepancy.For Lithium-ion battery whose efficiency is 97% [36], even if p bs,c t and p bs,d t in one third of all the 96 periods are not complementary, the maximum error of storage level for a complete cycle is at most 3.6% and typically much lower.This is acceptable in practice.Furthermore, even when the battery is fully charged, the output of co-generation unit or boiler can be slightly reduced so the feasibility of the strategy can also be guaranteed.The real-time operation cost will not increase due to the discrepancy.The analysis also applies to thermal storage.Since the day-ahead scheduling is conducted everyday with an updated storage level, such a discrepancy will not accumulate to the next day.

IV. CASE STUDY
We consider the energy hub for an energy-intensive industrial zone.The peak demands of electric and heat power are 445MW and 343MW, respectively.The duration of each period is t = 15 minutes, so there are T = 96 periods across the day.The installed wind and solar generation capacities are 800MW and 400MW, respectively.We adopted non-ideal models for battery and thermal energy storage units.Their capacities are 200MW/800MWh and 100MW/800MWh, respectively.The charging/discharging efficiencies for battery and thermal energy storage are 97% and 98%, respectively.Fig. 4 shows the predicted/actual output of renewable generation and deterministic power and heat demands.Uncertain demands are considered on the electric side, where the real-time electric load can fluctuate around the fixed load curve.Totally 100 scenarios of renewable and load power trajectories are created and used in Monte Carlo simulation.The complete system data is available at [37].The computation platform is a laptop with Intel i7-9750H CPU and 16GB RAM.
We first standardize the routine of comparison, which comes down to evaluating the statistical performance of dispatch policies.The regret ratio, which quantifies the gap between the online policy and the hindsight optimum, is introduced.Then, the rival methods for comparison are presented, and numerical results are given.

A. Streamline the Comparison
The regret ratio is introduced to reflect the optimality gap between a given online policy and the hindsight optimum.Suppose we have the exact renewable output curve {w t } T t=1 of the entire day, the deterministic dispatch problem is cast as where v 0 is the optimal value.We can solve problem (18) at the end of a day when {w t } T t=1 becomes the observation history, so v 0 is called the hindsight (ex-post) optimum.
However, this is an ideal case, and the hindsight optimum cannot be achieved in practice, because w t is observed periodby-period in real-time dispatch, and any causal dispatch policy μ t (w [t] , S t ) can only use observation history up to period t.The proposed partial-quantile policy can be expressed as μ † t (w t , S t ), where the history up to period t − 1 is not used, and S t given in (8) includes the lower and upper bounds of flexibility and action envelopes.
To test the statistical performance of a given causal policy μ t , Monte Carlo simulation is conducted.The flowchart is given in Algorithm 3. In our tests, the daily operation cost and the renewable curtailment rate are two performance indices to be investigated, which are expected to be as small as possible.

Algorithm 3 Monte Carlo Simulation
Initiation: Prepare renewable output scenarios Observe renewable output w t ; predict renewable output in the next τ periods, S τ t = {w t+1 , . . ., w t+τ }. 2: Solve a τ -step look-ahead optimization problem 3: Deploy the optimal solution in period t, which constitutes the MPC policy μ p t (w t , S τ t ).
The regret ratio of cost quantifies the optimality gap, which can be calculated as: Because any causal policy is non-anticipative, the optimality gap must be positive.

B. The Method for Comparison
Dynamic programming is a popular method for sequential decision-making.Since the input of the proposed method is a rough interval of renewable power, we choose RDDP in [17] which uses a similar input for comparison.We use the multi-parametric programming method in [18] to initialize the value functions, so that the subproblems in forward pass have bounded optimums.Different uncertainty sets are tested.
In the industry, rolling horizon optimization, which is also known as model predictive control (MPC), is a widely used policy for managing uncertain systems.The flowchart of MPC is provided in Algorithm 4. Noted that MPC requires forecasts on the renewable output, in the Monte Carlo simulation, we assume MPC knows the exact values of w t in the prediction windows with different lengths and neglect forecast errors.This is favored by the MPC method.

C. Benchmark Case
Using the uncertainty envelope in Fig. 5, we obtain the dayahead scheduling strategy.The method in Appendix-B gives α = 0.7229 and β = 0.9157.So α = 0.75 and β = 0.9 are adopted in the day-ahead problem (7).The uncertainty envelope (yellow) and the flexibility envelope (green) are illustrated in the same figure.It is observed that the flexibility envelope caters to the shape of the uncertainty envelope, and becomes wider when the renewable output is higher in the noon.Although the flexibility envelope cannot cover the entire uncertainty envelope, this is the best that can be achieved with the capacity configuration of energy storage devices.
To exhibit real-time operation strategy, we conduct Algorithm 2 with a scenario in which renewable output is moderate.The trajectories of the controllable demands p d0 and q d0 at electrical side and thermal side, respectively, and storage net charging powers are given in Fig. 6 and Fig. 7.The dispatch actions of generation devices are shown in Fig. 8.The net charging powers of storage devices always stay in the action envelopes because they are determined by the quantile.The controllable demands also reside in the envelopes due to constraints (12d)-(12e).However, the dispatch strategies of co-generation unit and boiler are given by the solution of problem (12), so the action envelopes of generation devices are not used, which actually unleashes their flexibility.
The economic performance of the proposed method is compared with MPC.We assume the prediction is always accurate, and change the length of prediction window, implying that the results of MPC are optimistic and unlikely to be achieved in practice.In this test, the electricity price at the main grid is Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.I.When τ = 4, MPC is myopic and postpones the charging of energy storage; however, when renewable power declines and demand rises in the evening, there is not enough capacity, resulting in operation infeasibility.With the increase of τ , MPC performs better.As for the proposed method, since the flexibility envelope is optimized in the day-ahead stage, it achieves a regret ratio as small as 4.68%, which is better than MPC with a 16step look-ahead window (or exact renewable power prediction for the next 4 hours), although no prediction is employed in the partial-quantile policy.This observation demonstrates the importance of integrating day-ahead scheduling and online dispatch.The proposed method utilizes renewable energy more efficiently with a curtailment rate of 9.89%, which is much smaller than that of MPC.
RDDP method is also tested.Since neither historical data nor the probability distribution of uncertainty is available, the uncertainty set of renewable energy can only be estimated based on day-ahead prediction w m t .We denote -RDDP with the uncertainty set given as [ w m t , w m t ] T t=1 , where 0 ≤ ≤ 1.When = 1, RDDP becomes deterministic.A larger leads to a smaller uncertainty set.As a result, the cost-to-go function may not be well-approximated if w t steps out of the uncertainty  I, As reduces from 0.9 to 0.7, the uncertainty set grows closer to the uncertainty envelope and the cost performance improves.When = 0.7, the regret ratio is 5.27%, which is comparable to MPC with a 16-step lookahead window.Since the upper bounds of uncertainty set are the same, the renewable spillage rate does not change much.However, when = 0.6, the uncertainty set grows too large for RDDP: since the feeder of the energy hub has a finite capacity, the support from the power grid has an upper limit.When renewable generation varies in so large intervals, it is impossible to guarantee a feasible operation for all possible scenarios in the uncertainty set.In this case, no operation strategy can be offered by RDDP.However, this can be a result of an overly conservative estimation of uncertainty.In contrast, even if the real renewable generation is not in the flexibility envelope, the proposed partial-quantile policy still gives dispatch actions as long as one exists.

D. Impact of Time-Varying Electricity Price
To investigate the impact of uncertain electricity prices at the power grid, we take the real-time price data in March 13, 2023 from the PJM market [38], varying between $15.16/MWh to $47.33/MWh, as shown in Fig. 9; the average is $27/MWh, which is used in the previous tests.The sequential observation of electricity price is taken into account in problem (12).In MPC, we assume the prediction of electricity price is also exact, although this is very challenging.For RDDP, the worst price realization is always the upper trajectory of the uncertainty set, so we use a fixed price curve by multiplying the price curve in Fig. 9 by 1.2, and then the RDDP problem can be solved in the same way as in the fixed price case.Results are shown in Table II.The regret ratio of the proposed method in this particular day slightly increases to 4.70%.To achieve a similar regret ratio, MPC needs exact predictions on renewable power and electricity price for the next 4.5 hours (or 18-step look-ahead).Price uncertainty has a more significant impact on RDDP.The regret ratio of 0.7-RDDP increases to 6.22%, which is comparable to 14 ∼ 16 step look-ahead MPC.This is because RDDP trains optimal value functions to address the worst-case realization of price and renewable power.As a result, RDDP inclines to purchase more energy from the power grid or produce more energy from  generation devices to maintain relatively high storage levels, and the value functions trained offline cannot be updated in the real-time stage.However, in the proposed method, the real-time dispatch problem (12) utilizes newly observed price and renewable output.This makes the proposed method less affected by the price uncertainty.
The amount of electricity purchased by different methods under fixed and real-time prices are also shown in Fig. 9.With accurate renewable generation data, the offline method can satisfy demand with little energy transaction with the power grid.In the online setting, limited information about uncertainty is available; MPC tends to import power at the end of the day; the proposed method allocates flexibility evenly across the day, and more electricity is bought during periods 20 ∼ 40 when the price is relatively low, because price is considered in problem (12).However, RDDP purchases more electricity during the same periods when the demands are high, as it has to tackle the worst case scenario.This test validates that the proposed method can better adapt to uncertain electricity price.

E. Impact of Energy Storage Capacity
To reduce carbon emissions, the installed capacities of co-generation unit and boiler are limited.Hence, system flexibility is mainly affected by the capacity of energy storage devices.In this test, we examine flexibility impact of storage capacities.Here, flexibility refers to the ability to track volatile renewable output.The parameter is used to quantify flexibility, where Cap is the total installed capacity of renewable generation.We also introduce parameter to reflect the ratio between volumes of flexibility envelope and the grand uncertainty envelope in which the lower trajectory is always 0. Clearly, 0 ≤ ρ ≤ 1 due to constraint (7c).
The battery storage at the electrical side has a more significant impact on the flexibility envelope.From Fig. 7 we can see that E bs max is the main bottleneck.We set E bs max = 800 • υ and change υ from 1 to 6 while keeping remaining parameters unchanged.Results are shown in Table III.The flexibility envelope is illustrated in Fig. 10.When υ = 1, about one quarter of the grand uncertainty envelope is covered, and κ is 4.41%.As E bs max grows larger, κ increases to 17.17%, which is sufficiently large because the capacity factors of wind and solar resources are 17.23% and 18.68% for the data we used.Generally, the battery achieves satisfactory renewable utilization when υ = 2.

V. CONCLUSION
This paper proposes an envelope-based framework to integrate day-ahead scheduling and real-time dispatch of energy hub systems.It is advantageous from three perspectives: first, the required information is moderate: neither historical data nor forecasts of renewable generation are needed; the only Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
input is an uncertainty envelope, a rough estimation of the fluctuation range of renewable power which is easy to obtain.Second, the day-ahead scheduling and real-time dispatch are thoroughly coordinated, circumventing myopic online decisions without using forecast data.Third, the proposed method directly applies to cases in which renewable power, load demand and electricity price are all uncertain.Simulation results show that the proposed method achieves a regret ratio of around 5% compared to the hindsight optimum, which is similar to MPC with exact prediction data for the next 4 ∼ 5 hours.The proposed framework is effective in the general framework of energy hub and can be easily extended to other applications such as operating a microgrid and a virtual power plant without a network model.

A. Proof of Proposition 1
To prove the feasibility of quantile policy, we consider two kinds of constraints.
1) Time-Decoupled Constraints: This type of constraint is linear inequalities in variable x t and independent of those variables in other periods.They can be expressed as: where A t and b t are constant matrix and vector.Since x and x satisfy (A.1).Hence, x given by the quantile policy satisfies all time-decoupled constraints, because:

B. Parameters Selection 1) Choice of α and β:
In the day-ahead scheduling problem (7), we intend to design the shape of the flexibility envelope to cater to the shape of uncertainty envelope.In ideal case, the uncertainty envelope should be covered by the flexibility envelope.However, the width of flexibility envelope depends on the capacities of co-generation unit and energy storage units.Due to the investment cost, it is not economical to deploy very large energy storage units to cover the entire range of uncertainty.So we have to 1) shape the flexibility envelope, making it resemble the uncertainty envelope; 2) place the flexibility envelope in an appropriate location in the uncertainty envelope so as to cover possible fluctuations as much as possible.
Therefore, in (7c), the flexibility envelope is required to cover a subset of the uncertainty envelope, which is [αw m t , βw m t ] T t=1 .We need this interval to prevent over-large flexibility in some periods while sacrificing the flexibility in the remaining periods.2a), (3) x satisfies (1a), (2a), (3) x and x satisfy (7d), (7e), (7f) where pr t and pr t are the minimum and maximum dispatched renewable power.Variables x and x correspond to the two extreme cases of renewable power, respectively.In practice, a larger α or a smaller β can lead to a less tight constraint for (7c), which enables more flexibility.However, parameters should not deviate too far to guarantee the shape of the flexibility envelope.
2) Choice of ω: Parameter ω appears before the quadratic term in the objective function (7a) of day-ahead scheduling.The role of this term is to minimize the variance of the envelope width (p 0 t − p0 t ) in different time periods.The rationale of the quadratic term can be described as follows: let z t = p0 t − p0 t .Then, the total flexibility across all periods is T t=1 z t .We expect flexibility to be evenly distributed as much as possible, so we minimize the following variance: Because the total flexibility of the energy hub is mainly determined by the capacities of generator units and energy storage.Since the capacities are given, the maximum of summation T t=1 z t does not change much, so minimizing variance can be implemented through minimizing the first term T t=1 z t , which leads to the objective function in (7a).The first term in the objective function aims to maximize the total power flexibility and ω weights the importance of variance reduction.If ω is too large, the primary goal of maximizing flexibility is compromised; if ω is too small, the variance is hardly minimized.In the objective function (7a), if power is represented by per unit value, an appropriate ω is about 0.4 ∼ 0.5, regardless of other system parameters, because the objective function is stagewise additive and contains only power variables p0 t and p0 t .In the case study, the base value of power is 400MW, and ω = 10 −3 converted to actual value.

Fig. 6 .
Fig.6.Action envelopes and real actions of controllable demands.

Fig. 7 .
Fig. 7. Action envelopes and real actions of energy storage.

Fig. 9 .
Fig. 9. (a) Real-time electricity price; (b) Transaction with power grid under fixed price; (c) Transaction with power grid under real-time price.

Algorithm 1
Quantile Policy μ t (•, S t ) in Period t 1: Observe the renewable output w t .

TABLE II COMPARISON
RESULTS UNDER UNCERTAIN ELECTRICITY PRICE

TABLE III IMPACT
OF BATTERY CAPACITY ON FLEXIBILITY Fig. 10.Flexibility envelope with different battery capacities.
= A t λ t xt + (1 − λ t )x t λ t A t xt + (1 − λ t )A t xt ≤ λ t b t + (1 − λ t )b t = b t , ∀t2) Time-Coupling Constraints: The feasibility analysis is complicated by the existence of energy storage dynamics which couple the variables in two consecutive periods, since no future data is available during real-time dispatch in the online setting.For both battery and thermal storage devices, the storage level in period t satisfiesE t = E 0 +From the day-ahead problem, we obtain the sequences of {p t } T t=1 and {p t } T t=1 satisfying (A.3).As stated in (7d) and (7e), for battery storage, pt is always greater than pt ; for thermal storage, pt is always greater than pt .For brevity, we define: p * t = min{p t , pt } and p t = max{p t , pt }.Hence, pt specified by the quantile policy satisfies:Taking the summation over multiple periods yields: Ideal Energy Storage Case: The above proof can be applied to the case with lossy energy storage whose charging/discharging efficiency η c /η d is strictly smaller than 1.The storage dynamics are given byE t = E 0 + In such circumstances,Proposition 1 still holds, because the storage level in constraints (7d) and (7e) satisfies The proof follows the same way by replacing p τ with η c p c τ − p d τ /η d .From the above prove we can see that the constraints Êbs t ≥ Ěbs t and Êhs t ≤ Ěhs t in (7d) and (7e) are redundant if the energy storage is lossless; they are necessary if non-ideal energy storage is taken into account.
* t ≤ pt = λ t pt + (1 − λ t )p t ≤ p t , ∀t Parameters α and β can be obtained from the following problem based on the upper trajectory w m t :