Operating Reliability Evaluation of Power Systems With Demand-Side Resources Considering Cyber Malfunctions

,

The rapid development of Internet of Things technologies has resulted in complex interaction between cyber systems and physical systems [1]. Smart grids stand as a typical application of Internet of Things technologies, in which advanced information technologies have created a strong connection between power cyber systems and power physical systems [2]. Thanks to the rapid employment of advanced information and communication technologies, demand-side resources (DSRs) like that air conditioners and electric vehicles, can be aggregated to provide reserve capacity [3] for enhancing the operation of power systems with wind power. DSRs are usually aggregated as aggregators to coordinate their response and conclude bilateral contracts [4] or bid into power markets [5]. For example, interruptible load can provide reserve capacities to power grids through bilateral contracts with reasonable economic compensation to the customers or the bids of demand response customers in the energy market [6].
However, malfunctions of cyber systems due to cyber attacks and communication latency [7] might result in losing controllability of DSRs and even the entire power systems. For example, the cyber attack that occurred to several power distribution companies has resulted in the blackout of Ukrainian power grids in 2015, which was the first blackout ever caused by a cyber attack [8]. Meanwhile, the characteristics of volatility and intermittence for high penetration of wind power [9], as well as stochastic behavior of DSRs [4], increase operating pressures of power systems. Therefore, it is imperative that the time-varying reliability of power systems with wind power [10] and DSRs should be thoroughly investigated and quantitatively assessed.

B. RELATED WORKS
Research effort has been devoted to the reliability evaluation of power systems with cyber malfunctions. In [11], cyber attacks in wind farm energy management systems were described as a modified Bayesian attack graph model and simulation method was adopted to analyze reliability of the power system. Validity evaluation of cyber links in active cyber physical distribution systems was proposed in [12] and the system reliability was evaluated using simulation techniques. The sequential simulation method was used to analyze the impacts of cyber systems on microgrids without considering cyber attacks [13], [14]. Although simulation methods are flexible for evaluating reliability of systems with complex operation conditions, only imprecise results are provided in these works [15]. Thus, an analytical method is proposed for accurate operating reliability assessment of power systems considering cyber malfunctions.
Considering the stochastic processes of power systems with wind power and DSRs considering malfunctions, multi-state models can be effective in representing system performance [16], [17]. However, the number of system states increases exponentially as the number of system components increase [9]. Hence, the number of system states can be comparatively huge even for a small power system composed of several wind turbines and DSRs. Significant efforts are needed for developing the stochastic process model for power systems.
To obtain accurate results of power system reliability, analytic methods including Markov process [18], universal generating function (UGF) [19] and Lz transform [17], [20] have been extensively employed. However, the traditional Markov process model might suffer from the curse of dimensionality considering the large number of system states [21].
Since the system state enumeration can replace extremely complicated combinational algorithms and reduce the computational burden [22], a systematic UGF method [23] was proposed to algebraically demonstrate performance distributions of the entire multi-state systems with steady-state probabilities. Lz transform method [20], as an extension of traditional UGF, has been developed to comprise time-varying probabilities of various states for assessing dynamic reliability of multi-state systems. The effectiveness of Lz transform for evaluating power system dynamic reliability has been verified in [17]. However, the integration of DSRs and cyber malfunctions due to cyber attacks and communication latency are not considered.
Considerable research has been conducted for the reliability of systems with cyber malfunctions. In [24], novel cyber-physical attacks occur to transmission lines were analyzed for detecting fake line-outage positions. Cyber attacks were modelled using stochastic failure model to analyze cascading failure process considering interdependency between the coupled power networks and cyber networks in [25]. Cyber attack rates were investigated considering four attack strategies in [26] to explore the interaction between attack and defense. Communication delays for islanded microgrids were considered in [27] to find corresponding effects on system control. In [28], the effect of communication delays was determined on the convergence of distributed frequency regulation. A cyber-physical modeling for power grid infrastructures with cyber-induced or cyber-enabled disruptions of physical components was proposed in [29]. However, limited works have been done to analyze the effects of cyber malfunctions on the operating reliability of power systems with DSRs and wind power.

C. CONTRIBUTIONS
Motivated by practical desire for quantitative assessment of power systems with DSRs considering cyber malfunctions, in this paper, the Lz transform approach is developed for the reliability evaluation of multi-state power systems with DSRs considering cyber malfunctions to avoid extremely complicated combinational algorithms. In the proposed framework, multi-state models for stochastic reserve capacity from DSRs and generation systems with wind power are respectively proposed, considering cyber malfunctions due to cyber attacks and communication latency. The multi-state model for a typical hierarchical decentralized control infrastructure in demand side is embedded. Reliability indices based on load curtailment by conducting optimal power flow for various system states are utilized for reliability assessment. The contributions of this paper can be summarized as follows: 1) A novel operating reliability evaluation framework for multi-state power systems with DSRs considering cyber malfunctions is proposed for avoiding increasing complexity caused by multiple system states. An analytic method based on Lz transform technique is developed to achieve dynamic system reliability instead of traditional simulation approach.
2) The multi-state model for reserve capacities of DSRs considering reliability model of ICT infrastructure and stochastic characteristics of DSRs is developed, where the reliability model of a typical hierarchical decentralized control infrastructure is extensively analyzed with the consideration of cyber attacks and communication latency. 3) Time-varying reliability indices of the proposed power systems is quantitative evaluated based on optimal power flow for multiple system states considering cyber malfunctions of both DSRs and generation systems.

D. ORGANIZATION OF THE PAPER
In the proposed operating reliability evaluation framework, three sections are elaborated as illustrated in FIGURE 1. First, the multi-state model for reserve capacities of DSRs are proposed considering reliability of a typical hierarchical decentralized control framework and uncertainties. Then, multi-state model for power generation systems with wind power and conventional generation is demonstrated with the consideration of cyber malfunctions. Finally, operating reli- Overview of the operating reliability evaluation framework for power systems with DSRs considering cyber malfunctions. VOLUME 8, 2020 ability evaluation is introduced with the calculation of reliability indices based on optimal dispatch. The remainder of the paper is organized as follows. Section II proposes the multi-state reserve capacity model for DSRs considering cyber malfunctions of cyber attacks and communication latency as well as uncertainties from DSRs. Section III presents the multi-state model for power generation systems with both wind power and conventional generation. The operating reliability evaluation based on optimal dispatch is developed to achieve minimal load interruption in Section IV. A modified IEEE RTS is adopted to depict the validity of the proposed approach in Section V. Section VI provides conclusions of this paper.

II. MULTI-STATE RESERVE CAPACITY MODEL FOR DSRS CONSIDERING CYBER MALFUNCTIONS A. RELIABILITY MODEL FOR THE CYBER SYSTEM WITH A HIERARCHICAL DECENTRALIED CONTROL FRAMEWORK IN DEMAND SIDE
To incorporate the control of DSRs, a typical cyber infrastructure with a hierarchical decentralized control framework is usually adopted as presented in FIGURE 2. For the control structure, a control center and multiple local controllers (a number of Q i ) in different load sectors are embedded.
For cyber systems in demand side, the developed control strategy consists of two decision making layers including supervisory and device layers in FIGURE 2. During a control period, the control center in the supervisory layer is responsible for sending control signals to distributed local controllers for the response of DSRs; then, DSRs in different load sectors in the device layer receive these signals from local controllers. However, cyber systems might suffer from FIGURE 2. A hierarchical decentralized control framework of the cyber system in demand side for bus i . malfunctions including cyber attacks and communication latency. For example, hackers might intrude into the control center causing the complete failure of the cyber system and losing controllability of DSRs. Moreover, if communication latency occurs, DSRs cannot immediately receive control signals, which would result in the late response of DSRs. These kinds of cyber malfunction might seriously affect system reliability.
For the load sector q i at bus i, the reliability models for the cyber system consists of two layers from the control center in the supervisory layer to the local controller q i and from the local controller q i to the load sector in the device layer.
In this paper, cyber attacks are presented as a typical binary-state model. Once cyber attacks occur to a cyber system, the cyber system is in complete failure state without functioning denoted by ''0''. If no occurrence of cyber attacks, the cyber system is in perfect functioning state denoted by ''1''. Therefore, for the time t, the Lz transforms for cyber attacks of the supervisory layer from the control center to the local controller q i and the device layer from the local controller q i to the corresponding load sector can be formulated in (1)-(2), respectively. z is a complex variable providing a comprehensive method for the system state enumeration and substituting complicated combinational algorithm.
Moreover, for the time t, taking the supervisory layer for example, if communication latency occurs for t i,q CL i with a probability p i,q CLD i (t), the functioning of the cyber system in the time (t + t i,q CL i ) is affected; if no occurrence of communication latency, the cyber system will not be influenced by this latency. Therefore, the Lz transforms for both layers are presented in (3)-(4), respectively.
Considering both cyber attacks and communication latency, the Lz transforms for two layers are formulated in (5)-(6), respectively.
For the load sector q i at bus i, the reliability model of its cyber systems for both layers has four forms with communication latencies in both layers, the supervisory layer, the device layer, and no latency. The corresponding Lz transforms can be formulated using series operator s in see (7)-(10) as shown at the bottom of the this page, respectively.
The Lz transform for the reliability model for cyber systems with communication latencies in both layers is presented in (7).
The Lz transform for the reliability model for cyber systems with communication latency in only supervisory layer is presented in (8).
The Lz transform for the reliability model for cyber systems with communication latency in only supervisory layer is presented in (9).
Specifically, if no communication latency occurs which indicates t i,q CL i = t i,q LS i = 0, the corresponding Lz transform for a cyber system in this scenario is formulated in (10).

B. UNCERTAINTIES FROM STOCHASTIC CHARACTRISTICS OF DSRS
Since DSRs can actively participate in the operation of power systems, considering stochastic features of DSRs, especially physical characteristics of interruptible loads, for example, the thermodynamic model for thermostatically-controlledloads, the reserve capacity provided by DSRs can be modeled by a multi-state model.
As presented in FIGURE 3, the multi-state model of available reserve capacity for DSRs considering stochastic characteristics form the physical characteristics of DSRs and participation levels of customers.
Let  states K DSR i is determined by the maximal reserve capacity and the accuracy of multi-state model. An appropriate value of K DSR i is supposed to be determined for balancing the accuracy and the complexity. Let p i,j DSR i , j DSR i = 1, · · · , K DSR i be the probability of the reserve capacity in the state j DSR at bus i. Therefore, the Lz representation of the reserve capacity of DSRs is formulated in (11).
Moreover, although the reserve capacity of DSRs is obtained considering their physical characteristics, the reserve capacity is also related to the participation level of customers in demand side. For example, some customers may refuse to respond to interrupt their air conditioners in a hot summer day to keep their comfort which will certainly influence the reserve capacity of DSRs. Moreover, customers' willingness to provide reserve capacity is also dependent on incentive mechanism for conducting demand response. Considering these uncertainties, the participation level of customers can be demonstrated as a multi-state model where level states and transition rates can be derived from historical database. In Considering both original reserve capacity from DSRs and participation level of customers, the Lz transform of the available reserve capacity from DSRs can be obtained utilizing the multiply composition operator m as following.

C. MULTI-STATE MODEL FOR DSRS
The actual amount of reserves from DSRs is dependent on the load amount which can be interrupted and the participation level of customers, as well as the well-functioning work of cyber systems in demand side. Considering the effects of cyber systems, taking (7) for example where communication latencys in both layers are considered, the actual available reserves from DSRs can be defined using the multiply composition operator m in the following polynomial.

III. MULTI-STATE MODELS FOR POWER GENERATION SYSTEMS WITH CYBER MALFUNCTIONS
Considering the interconnection between cyber systems and physical systems in power generation, the influence of cyber malfunctions due to cyber attacks and communication latency on wind power and conventional generating units are modeled in this section. FIGURE 4 presents the reliability model for power generation systems with cyber malfunctions.

A. MULTI-STATE MODEL FOR WIND FARMS WITH CYBER MALFUNCTIONS
The power output of a wind farm is determined by its wind turbines and corresponding cyber systems.

1) MULTI-STATE MODEL FOR A WIND TURBINE
For a wind turbine, the power output is definitely related to the stochastic wind speed and its physical conditions. Markov process model for wind speed has been extensively utilized in the reliability analysis considering the time-varying characteristics of wind speed. During the system operation period, the state probabilities of future wind speeds are not only related to the current wind speed but also dependent on the possible future wind speeds. In the Markov process model, the multi-state wind speeds and state transition rates are presented, which can be obtained by statistical analysis of operational data.
Let S i,j WS i , j WS i = 1, · · · , K WS i be the wind speed for bus i at time t which is a stochastic variable. The number of discrete states K WS i is determined by the accuracy of multi-state model. In the Markov process model for wind speed, differential equations are utilized for solving the state probability p i,j WS i as formulated in (15), under the initial conditions in (16).
According to the corresponding relationship [9] between wind power and wind speed, the power outputs under different wind speeds can be represented as Hence, considering the stochastic characteristics of wind speed, the Lz transform for wind speed at bus i can be formulated in (17). (17) Moreover, the power output of a wind turbine also relies on the reliable operation of its physical components. If the wind turbine fails, there is no power output. Otherwise, if it functions well, the output is equal to its rated power. A binary-state model considering random failures for the reliability of the wind turbine u is usually adopted as formulated in (18), where superscripts ''1'' and ''0'' indicate the wind turbine is in well-functioning state or complete failure state, respectively.
Therefore, considering both of wind speed and reliability of a wind turbine, the Lz representation for the output of wind turbine u is formulated in (19) utilizing the multiply operator m .
For a wind farm at bus i consisting of various wind turbines, the Lz transform of power output of the wind farm can be presented utilizing parallel operator p in (20).
Moreover, considering the cyber attacks and communication latency of cyber systems in wind power generation systems, the Lz transform for the reliability model of wind power generation systems at bus i is formulated in (21).
· z 0 (21) Therefore, the available power generation of a wind farm at bus i considering random malfunctions of cyber systems can be represented utilizing the multiply operator m in the following polynomial.

B. MULTI-STATE MODEL FOR CONVENTIONAL GENERATION SYSTEMS WITH CYBER MALFUNCTIONS
For conventional generating units, random failures or degradation process of generating units can reduce the available generating capacity completely or partially. For stochastic characteristics of generation, continuous state process is usually difficult to model. Therefore, multi-state characteristics for generation of a traditional generating unit v can be VOLUME 8, 2020 presented using Lz transform as following. Note generation capacities of different states can be obtained by statistical analysis from historical data of a generating unit operation.
For the conventional generation system at bus i, the multistate model for the generation system based on Lz transform using parallel operator p is presented in (24).
Considering the cyber influences on traditional generation systems, cyber attacks and communication latency are considered as well. The corresponding Lz representation is formulated in (25). Then, the reliability model of available generation for the traditional power generation system is formulated in (26).

C. MULTI-STATE MODEL FOR POWER GENERATION SYSTEMS WITH CYBER MALFUNCTIONS
Considering multi-state models for wind power and conventional generating units, the multi-state model for power generation systems with cyber malfunctions is formulated as the following polynomial using parallel operator p .

IV. RELIABILITY EVALUATION OF POWER SYSTEMS WITH DSRS CONSIDERING CYBER MALFUNCTONS A. RELIABILITY INDICES
After obtaining Lz transforms for generation systems and reserve capacity of DSRs, load curtailment for the system can be obtained by conducting dispatch operation based on an optimal power flow operator opf . The Lz transform of load curtailment at but i of a N -bus power system with K states is formulated as the following polynomial.
The optimal power flow composition operator in (28) is defined as an optimization model to determine load curtailment for system state j at bus i, which is formulated in (29)- (33).
The objection function in (29) is to minimal the total load curtailment for the system state j at time t, which is subject to the following constraints.
Power balance constraints: Generation limits: Load curtailment constraints: Line flow constraints: P j (t) = [P j 1 (t), · · · , P j N (t)] T and L j (t) = [L j 1 (t), · · · , L j N (t)] T are the vectors of equivalent power generation and load for the state j at time t. Four reliability indices for the entire power system and a bus including the expected energy not supplied (EENS), and loss of load probability (LOLP) are utilized. Reliability indices for both of bus i and the entire system for an operation period τ are provided in (34)-(37).

B. RELIABILITY EVALUATION PROCEDURE
The procedure for evaluating time-varying reliability of the power systems with cyber malfunctions is elaborated as follows.
Step 1: Initialization of system parameters, including the initial states for wind power generation, conventional power generation, DSRs, cyber systems.
Step 2: Calculation of time-vary probabilities of different states for wind power generation, conventional power generation, reserve capacity of DSRs, cyber systems and participation level of customers.
Step 3: Determination of Lz transforms for actual available reserve capacity of DSRs considering cyber malfunctions in demand side using (1)- (14).
Step 5: Calculation of the Lz transform for determining load curtailment at each bus using (28)- (33).

V. SYSTEM STUDIES
The modified IEEE RTS [30] is adopted for demonstrating the proposed models and techniques. A large wind farm consisting of 250 wind turbines with a rated power of 2MW is added at bus 21 [9]. The Markov process model for the power output of a single wind turbine introduced in [31] is used for the studies. The MTTF and MTTR of a wind turbine are assumed to be 3650 and 55 h, respectively [9]. Five 575-MW coal thermal generating units modelled as a four-state Markov process [32] are installed in the system. The failure rate and repair rate including failures caused by cyber attacks of cyber systems in generation system are 8 * 10-4/hour and 0.02/hour [33], respectively. The failure rate and repair rate of cyber systems in demand side are 1/960 per hour and 0.025 per hour, which are a little larger than those in the generation system. The system total demand L is 2850MW.
There is a proportion a of interruptible loads can serve as DSRs to provide operating reserves in the study. The distribution of these DSRs at each bus is proportional to the base load in these buses. A four-state model for reserve capacity of DSRs is adopted where state transition rates are presented in TABLE 1. The multi-state model for participation level of customers presented in [4] is adopted. The system operation period is 100 hours. The initial states for wind turbines,  To validate the accuracy of the proposed method, Monte Carlo simulation technique with 0.1 million sampling size is also adopted in Scenarios A' and B' to compare the results   with those obtained by the proposed method. Computer programs for the proposed method and simulation method were developed in Matlab R2018b and were implanted on a laptop with a 1.80 GHz processor. The results of the proposed method have high computational accuracy, where the average error of the proposed method and simulation is relatively small at 0.37% .  TABLE 3 lists EENS for bus 6 and the system as well as corresponding relative differences for the entire operation period of Case 1. The relative difference comparing Scenario A and D is up to 11.33% which fully proves reliability enhancement of DSRs. TABLE 4 presents computation time of the proposed method and simulation method for different scenarios, denoting that the computation time of the proposed method is much shorter than that of simulation method.

B. CASE 2: MALFUNCTIONS OF CYBER SYSTEMS
The systems in different scenarios with and without cyber malfunctions are investigated in Case 2, where a proportion with 20% DSRs is adopted. Scenario A denotes the system without the consideration of cyber malfunctions where there is no occurrence of cyber attacks and communication latency. Scenario B considers only cyber attacks and without communication latency. Cyber malfunctions with cyber attacks and two hours delay are investigated in Scenario C.
Time-varying LOLP of bus 6 and system EENS for Case 2 are respectively demonstrated in FIGURE 6 and     DSRs is utilized. Besides signal latency time and response time of DSRs, the time when the DSRs are committed is also considered. These three types of time are aggregated as the time for providing reserve capacities as presented in TABLE 6 for different scenarios. FIGURE 7 illustrates the time-varying LOLP of bus 6 and TABLE 7 presents system EENS and relative differences for the operation period for Case 3. Scenario A where DSRs are immediately committed is the base one with relatively small system EENS. The LOLP in FIGURE 7 immediately   decreases when the reserve capacity of DSRs is performed. The relative differences of EENS increase with the increase of the time using reserve capacity of DSRs.

D. CASE 4: DIFFERENT INITIAL CONDITIONS
In order to explore the impacts of initial states on system reliability, scenarios with different initial conditions are provided, where a proportion of 20% DSRs and immediate utilization of DSRs are assumed. TABLE 8 presents different initial states of wind power generation and reserve capacity from DSRs. Scenario A is the base one where the initial states of wind power generation and reserve capacity of DSRs are all the best states with the largest capacity. In Scenario B, the wind power is initially in the worst state with zero power output. In Scenarion C, reserve capacity of DSRs is initially in the worst state. Both of wind power and DSRs are all while initially in the worst states in Scenario D. FIGURE 8 and TABLE 9 present system time-varying LOLP and EENS, respectively. It can be observed that a change of system initial conditions may lead to a large variation of system reliabilities. For example, the LOLP at t = 10 in Scenario D is 0.03995, which increases about 31.46% comparing with Scenario A. Moreover, with the increase of system operation time, the system LOLP in different scenarios becomes very close. The system steady-state LOLP of in different scenarios of Case 4 is the same and evaluated as 0.0924, since the steady-state reliability assessment does not consider the initial conditions of operating states, which is considerably important for time-varying reliability assessment.

VI. CONCLUSION
This paper proposes a reliability evaluation framework based on Lz transform for power systems with DSRs considering cyber malfunctions due to cyber attacks and communication latency. Multi-state models for DSRs and generation systems are proposed considering cyber malfunctions. System reliability is investigated by conducting optimal dispatch to calculate load curtailment. Operating reliability indices of four cases in the modified IEEE RTS are quantitatively presented, where three findings are summarized as follows.
1) By comparing results and computation time of the proposed method and simulation method, the effectiveness of the proposed method is clearly validated. 2) Reserve capacity from DSRs can definitely enhance system operating reliability, which is affected by proportions of DSRs, consideration of cyber malfunctions, actual committed time for DSRs and initial system conditions. In the case studies, system EENS improves up to 11.33% with the increasing proportions of DSRs; system EENS considering cyber malfunctions is about 18.73 times to that of the system without cyber malfunctions; LOLP immediately decreases when reserve capacities from DSRs are performed; the relative difference of system EENS between the systems which are initially in the best state and the worst state is 2.93%.
3) The quantitative assessment proposed in this paper instead of qualitative analysis can provide time-varying reliability values for decision making during the operation of practical power systems.