Cyber-Attack Detection and Countermeasure for Distributed Electric Springs for Smart Grid Applications

With increasing installations of grid-connected power electronic converters in the distribution network, there is a new trend of using distributed control in a cyber layer to coordinate the operations of these power converters for improving power system stability. However, cyber-attacks remain a threat to such distributed control. This paper addresses the cyber-attack detection and a countermeasure of distributed electric springs (ESs) that have emerged as a fast demand-response technology. A fully distributed model-based architecture for cyber-attack detection in the communication network is developed. Based on a dynamic model of ES with consensus control, a local state estimator is proposed and practically implemented to monitor the system. The estimator is fully distributed because only local and neighboring information is necessary. A countermeasure for the distributed ESs to ride through the cyber-attack and maintain regulatory services in a microgrid is demonstrated successfully. Experimental results are provided to verify the effectiveness of the proposed cyber-attack detection method and its ride-through capability.


I. INTRODUCTION
Power-electronics-intensive microgrid is an efficient way to achieve high-performance power distribution with increasing penetration of renewable power [1]. In a recent review paper on cyber security in smart grid [2], grid-tied power converters are classified as (i) grid-feeding, (ii) gridforming and (iii) grid-supporting units. Grid-forming units play a role in regulating the voltage and frequency of the power grid. Grid-feeding units feed energy into the grid. Grid-supporting units offer other auxiliary functions such as power quality enhancement, stability support, ride through and economic dispatch. High-frequency power electronics offers a bottom-up approach to smart grid technology. In a 2020 review article [3], electric spring (ES) is quoted as an example of high-frequency power electronic device for providing electric voltage support, storing electric energy, and damping low-frequency oscillations. Electric spring is a smart grid technology originally introduced as a demandside management method for achieving instantaneous power balance for microgrid and power grid with substantial penetration of intermittent renewable energy sources. Importantly, recent research shows that ES is a power-electronic unit that could have the triple functions of being grid-forming (for regulating mains voltage and frequency [4]), grid-feeding (for feeding solar energy into the grid [5]) and grid-supporting (for providing auxiliary services such as power quality enhancement [6], power imbalance reduction [7] and power system resilience [8]).
With the rapid development of the digital control and communication technologies, supervisory informationbased control of microgrid power electronics converters has been widely studied [9]. Compared with communicationfree control, communication-based control methods introduce new features such as improved performance of the global average voltage regulation and current sharing [10]. Among different communication-based control methods, distributed control can offer good control performance with very sparse communication links between neighbors [11]. While the extra cyber layer provides a channel for the flow of information and opens the door for distributed control in emerging power electronics-intensive microgrid and power grid, it makes the system vulnerable to potential cyber-attacks. Therefore, the cybersecurity of distributed power electronics systems connected through a cyber layer is an emerging research topic that deserves attention and investigation [12].
According to [2] and [13]- [16], cyber-attacks may take place at the: (1) hardware devices: the attacker directly attacks the onboard sensors and change the sensed data before it goes into microprocessor; (2) transmitted data: the attacker can violate the information transmitted over the communication links. Data attacks can be divided as privacy attacks and false-data injection attacks (FDIAs) [16]. In this paper, we consider false-data injection attacks. Indeed, direct attacks on the hardware devices is technically difficult because the attacker must have direct access to the hardware devices. In a privacy attack, the attacker aims to steal the data that smart meters send to the power market and dig out end user's privacy information [9]. This scenario is of great importance, as FDIAs could distort the control information transmitted in the cyber layer and inject wrong data into control loops. Compared with privacy attack, the FDIA is a much bigger threat to the distributed power electronics systems because FDIA could mislead the power electronics devices to make wrong decisions. Related power electronics devices may compete against, instead of cooperating with, one another. In extreme cases, this fault may make the system unstable or damage the electrical equipment. An example is the communicationbased cooperative voltage and frequency control of multiple inverters in an islanded ac microgrid. Wrong decisions made by inverters could shift the operating voltage and frequency from their respective nominal values and cause serious consequences. Many research efforts have been devoted to the control design of multiple inverters, but its cybersecurity is not yet well studied.
Because of the fast dynamic of power electronics, traditional cybersecurity methods such as encryption and authorization are too slow to cope with the fast power electronics control loops. Hence, it is necessary to establish an attack detection architecture to provide cybersecurity for power electronics systems. Recent research of power system cybersecurity mainly focused on the generation and transmission level with large-scale centralized supervisory control and data acquisition (SCADA) system [10], [15]. Cyber protection mechanism in distribution network with power electronics converters needs urgent attention.
Potential FDIA could change the information in the consensus control and alter its final control output to mislead the distributed power electronics systems. Due to the limited communication ability in such a distributed control framework, good cyber-attack detection methods should rely only on the local information and data provided by neighbors. Studying a distributed controlled inverterbased microgrid, reference [12] numerically analyses the FDIA effects on the system performance. A stable region is defined where the attack is not serious enough to make the system unstable but only worsens the power-sharing performance. Reference [17] presents a simulation study on the effects of the FDIAs in a distributed controlled DC microgrid. The detection method uses Hunger tool in MATLAB to insert extra detection marks in control signals. Reference [18] uses the consensus algorithm features to find the attacked device and proposes a resilient cooperative control for DC microgrid [18]. This detection method is effective when less than half of the devices are attacked. This control will not eliminate the attacked data but uses an elastic coefficient to suppress the attack effect. References [12], [17] and [18] are based on simulation studies only. Hardware implementation issues for cyberattack detection and countermeasures need more investigations.
The main contributions of this paper include (i) the design and practical evaluation of a fully distributed modelbased detection architecture against cyber-attacks in the communication network between subsystems which are physically interconnected and regulated by a distributed consensus protocol, and (ii) a countermeasure to maintain normal services of a group of distributed ESs under cyberattack.

A. THE PHYSICAL AND CYBER LAYERS IN A NETWORK OF ES
Consider an islanded microgrid comprising a weak ac power source and a cluster of loads. The physical layer is shown in Fig.1. The ac power source in this weak grid is fed with intermittent renewable energy sources. Such a weak power grid can be emulated in hardware setup by programming a pre-recorded mains voltage profile of a renewable energy source in the programmable power source to create a time-varying voltage fluctuation along the distribution line. The cluster of loads consists of a mixture of smart loads and critical loads. Critical loads here This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2022.3145015, IEEE Access VOLUME XX, 2021 10 refer to sensitive electric loads that require a well-regulated ac mains voltage. Each smart load consists of an ES connected in series with a noncritical load. The noncritical load is one that can tolerate certain voltage variations without causing consumer inconvenience. The ES plays the significant role of regulating the local line voltage and adaptively adjusting the power consumption of each noncritical load to instantaneously balance power supply and demand.
In this study, an additional communication network is added to explore new control features of distributed ESs. As verified in [4], consensus control can coordinate different ESs for the voltage regulation purpose and guarantee the power-sharing ability. Under extreme cases when the communication network collapses, the cyberphysical system can roll back to the simple physical system with only local droop control [4]. Fig. 2 represents the cyber-physical model of multiple ESs. The solid black lines show the physical electric lines between neighboring ESs. The blue arrows represent the two-way communication links between local controllers in the neighborhood. A potential cyber-attack may occur in the communication network.
Assume that there are n ESs in the system and each ES is considered as an agent. Graph G = (V, E) is a graph with nodes set V = [1, 2,…, n] and edges E  VV. In this study, the nodes are the distributed ESs and the edges represent the communication links among different agents. The adjacency matrix A = (aij)nxn has non-negative elements. aij = 1 if and only if there is a communication link between node i and j. According to [11], the graph G should be a connected undirected graph that guarantees consensus convergence.

B. THE DYNAMIC MODEL OF ES
Without loss of generality, Fig. 3 shows the layout of the i-th ES (ES-i) which is connected to its neighbors: ES-l and ES-k. The dynamics of ES-i should be studied with the considerations of the ES circuit, the noncritical load and the connected lines. The ES is implemented by a half-bridge power inverter connected in series with a resistive noncritical load (Rnc).
As shown in Fig.4, a decoupled single-phase d-q framework can be developed by choosing the local smart load current (ISL) as the referenced d vector. The complete dynamics of ES-i can be described by the following set of equations (note: subscript i has been dropped for notational simplicity): 1 where Ves is the output voltage of the ES, IL is the inductor current, and Iil, Iik are the line currents. Vin is the voltage fed to the LC filter. Vsi, Vsl, and Vsk are respectively the line voltage of ES-i, ES-l, and ES-k. Vnc is the voltage across the noncritical load. Parameters Cf, Lf, RL are the values of the filter capacitor, inductor, and inductor's series resistance, respectively. Lil, Ril, Lik, and Rik are the parameters of the distribution line connecting ES-i to its neighbors k and l, respectively. We define XL and XC as the inductive and capacitive reactance, respectively, i.e.: 1 ,  v es vin This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.
In general, each ES physically connected to ES-i shall be included in group N.
Combining (1), (2) and (3), the continuous-time statespace formulation for the ES-i is characterized as: where i x is the state of the ES, ui is the control input, ξi is a vector containing the interconnections terms between an ES and its neighbors (the physical connection), and yi is the output vector. The vectors are explicitly defined as: [ , , , , , , , ] [ , ] All matrices are defined in the Appendix.

C. CONSENSUS CONTROL OF ES
The consensus control of distributed ESs has been studied in [2]. The control objectives for ESs can be summarized as voltage/frequency regulation and accurate proportional reactive/active power-sharing. In this paper which focuses on the ES cyber-attack detection problem, the control loop of ES has been simplified to the objectives of voltage regulation and reactive power-sharing. Fig.5 shows the block diagram of ES consensus control.
Assuming a lossless operation, the ES voltage on the d axis is set to zero by the reference, resulting in an ES voltage perpendicular to the noncritical load current. This indicates that the ES compensates only reactive power for the voltage regulatory service. Hence, the voltage regulation and consensus power-sharing loops are designed to control the variables on the q axis. The reactive power Qesi is defined as: With the objective of sharing reactive power compensation in a distributed scenario, it is necessary for the neighboring ESs to communicate among one another. Each ES will receive the following set of data from its neighboring ES-j: The wireless communication links over which information is transferred are marked as the blue arrows (as the inputs to the power calculator) in Fig. 5. Note that the transmitted data shall be in the AC form and can be decoupled in the local d-q frame. Because the real-time value of each ES's reactive power does not change with the reference d-q frame, it is possible to compute ES-j's reactive power by calculating the d/q components of dji with respect to the local reference of ES-i: From the received data, ES-i is then able to calculate the reactive power, allowing for the consensus input to be computed. The reactive power values of both neighbor and local ES are then fed into the consensus algorithm. A compensator value δ is generated by the following equation: where c Q is the coupling parameter between the voltage and reactive power in the regulator. The error signal is added to the voltage reference Vref to adjust the reference point. The control input in the q-axis is given by where si V is the mains bus voltage, q P K and q I K are the proportional and integrator gains for the q axis, respectively. Similarly, d i u can be written as: where d P K and d I K are the proportional and integrator gains regulating the d-axis voltage, respectively. The input in (5) is the real input voltage Vin which can be derived by multiplying q i u and d i u with a fixed factor. This factor is related to the dc voltage and modulation gain. As proved in [4], the proposed consensus control can guarantee the main bus voltage regulation and reactive-power sharing performance in steady state.

Remark 1.
Note that the subscript ji is used rather than j to explicitly emphasize that the communicated data vector is being transmitted from ES-j to ES-i.
To simplify the analysis, the following assumptions about the communication network are clarified:  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.  (4)), as computed by ESi: This interconnection term will be used for cyber-attack detection.
The usage of a large group of distributed ESs in the power grid could be associated with a wide range of electric loads [6][7], PV panels [5] and energy storage [21] in the power network. Consequently, the collective stabilizing effects of distributed ESs are less dependent on the power factors of the individual electric loads in practice. The ratings of the ESs are typically less than 15% of those of the noncritical loads [22]. Thus, ES technology is an economical and distributed way to stabilize the power grid.

D. CYBER ATTACK ON ES
The introduction of communication in the control architecture for a network of ESs possibly exposes the ESs to cyber-attacks. As shown in Fig. 5, potential cyber-attacks may tamper with the information package dji. As a result, the local ES will get the wrong reactive power value of its neighbor and the expected control performance cannot be achieved.
To model the cyber-attack behavior, the action of the attack on the transmitted information can be formalized by defining an additional variable: containing the data as received by ES-i, and where () ji t  is a function defined by the attacker, unknown to the ES controller, with the following characteristics: 0, : 0, with Ta > 0 the time instance at which the attacker starts influencing the communicated data. Furthermore, r i  in (12) is redefined with the attacked information r ji d . Given the possible presence of a cyber-attack in the communication infrastructure, it is necessary to design a monitoring strategy to tackle the following "attack detection" problem: Problem 1 (Attack detection). Given ES-i possibly subject to attacks, design an attack detection module Di to verify whether: i.e., whether an attack is active on any communication channel into ES-i or not. The design of the attack detection module will be the subject of the following Section.

III. CYBER ATTACK DETECTION
To detect cyber-attacks in the communication between neighboring ESs, it is necessary to equip each ES with a local monitoring tool (called a diagnoser). The diagnoser consists of a distributed state estimator (from which a residual is generated) and a detection algorithm. In this section, a discretization method for the system model is described first. Then, the distributed estimator and the detection method are explained.

A. MODEL DISCRETIZATION
The design of the attack detection module is implemented on a digital controller. Based on the statespace model of the ES, the discretized dynamic model with the additional unstructured and unknown disturbance terms is expressed as: where i w and i  are the process and measurement noise with 0 0 i  , and is independent of the noise terms, for all 0 k  . Remark 2 (PWM input voltage). The input to the ES is defined as a PWM input voltage. Thus, Vin is a highly nonlinear, switching input. Thus, to improve the performance of the state estimator (to be defined in the following section), we suppose that Vin is an unknown input. This, on the one hand, binds us to the use of those state estimators that are decoupled to the input ui, on the other it allows us to remove the error caused by approximating Vin as a sine wave.

B. DISTRIBUTED STATE ESTIMATOR
As anticipated in Remark 2, a state estimator for which Vin is an unknown input can be used to decouple the estimation error from the nonlinearities introduced by the PWM input voltage. Here we exploit the use of the unbiased Kalman Filter (KF) proposed in Error! Reference source not found., which states that the following two conditions must be satisfied: i. Pair ( , ) Through matrix analysis, it can be shown that ii A , i B , and i C as defined in (4) and (21) meet these conditions.
To remove any possible bias from the estimation error (i.e., the difference between the estimated parameters and the real values of the electrical components), we augment the ES's state by defining x [ ] taken to be a constant output bias vector, with the following dynamics: Note that the conditions i. and ii. remain satisfied by the matrices defined in (22a) and (22b). The following equations define the dynamics of the state estimator unbiased by ui:

A I K C I -B M C I L C (24)
The estimation error is defined as i i i xx  =− and the residual is given by i i i r y y =−. Fig. 6 shows the block diagram of the state estimator, as defined in (23). Therefore, the proposed Kalman filter can estimate ES-i's state with input ui unknown to the estimator.
Given their definition in Error! Reference source not found., matrices Mi(k) and Ki(k) guarantee that the estimation error does not depend on the switching input ui. Mi(k) is designed at each time step k such that the following proposition holds: The dynamics of estimation error i ε and residual can be derived from (22) and (23):

C. ATTACK DETECTION STRATEGY
From the definition of the residual and its covariance [12], the cyber-attack detection strategy is presented here. Under normal conditions, both the estimation error and the residual of the unbiased Kalman Filter approach asymptotically zero-mean Gaussian processes. However, after the onset of a cyber-attack, which is a non-zero deterministic signal, the residual will not be zero mean, but rather: ( where i H is a block matrix which is nonzero in correspondence to the attacked information from the neighbor j. Hence, for 0, 0 a ji i r   . Given the change in the mean of the residual after an attack, it is possible to exploit well-established change detection algorithms available in the literature Error! Reference source not found.. These algorithms exploit the stochastic properties of the residual to discriminate between the following hypotheses: where 0 i is the null-hypothesis, i.e. the diagnostic supposes the system is not under attack. If 1 i is chosen, then an attack is thought to be present on the communication link between ES-i and its neighbors, i.e., detection occurs.
In this work, we exploit a detection scheme similar to that described in [12]. We start by introducing an auxiliary variable holds, 1 i is thought to be active. Otherwise 0 i is chosen. Wi is the length of a window over which i T is computed.

Remark 3
The choice of the detection threshold () i k  is fundamental to the performance of the detection scheme. If the threshold is selected to be too high, (32) will not hold except in the most extreme cases. If it is selected too low, it may lead to a high probability of false alarms, i.e., saying () i k  is active while there is no attack. While this aspect is out of the scope of this paper, readers interested in it should refer to [18] and [18] and their citations therein for methods of threshold selection. Note, specifically, that there exist methods to define () i k  such that certain properties are guaranteed by the detection scheme, e.g., user-defined false-alarm rates.
The flowchart in Fig. 7 presents the proposed cyberattack detection method. The method starts by initializing the offline ES state-space model with circuit parameters. The model is then discretized by (17). A Kalman filter can be constructed as a local estimator based on (23) and (24). Then, the distributed state estimator can be implemented as a module in the ES controller which requires only local and neighboring data. At each sampling step, the residual signal ri(k) is calculated and saved in a FIFO (first-in, first-out) buffer. The cyber-attack module then compares , () ii T r k with the threshold () i k  to detect the presence of an attack. If a cyber-attack is detected, a protection mechanism will be triggered. The consensus control will ignore the attacked neighboring data and will use the previous consensus data during the attack. After the attack disappears, consensus control output can be updated with normal data. This mechanism can ride through the cyberattack smoothly and preserve power-sharing ability during the attack.
continuous state-space model (4

A. EXPERIMENT SETUP
The proposed cyber-attack detection method is validated in an experimental test. The hardware setup involves a 110V AC microgrid with three distributed ESs as shown in Fig. 8. A programmable power source (California Instruments CSW550) is used to emulate a power system with changing renewable power. Three ESs are located along the distribution line, and each is connected in series with a resistance load. Lead-acid battery cells with Model No. LC-R127R2NA (and ratings of 12V, 7.2 Ah/20 hr) are used in the experiment. The three controllers and communication networks are implemented in the dSPACE system. For the control layer, three virtual control blocks are established in the dSPACE. Each controller collects local ES sensors' data and sends real-time PWM signal to the ES hardware. According to Assumption 2, the communication links are ideal and simulated in dSPACE. The cyber layer has a chain topology which is the same as the physical layer.
Each ES is equipped with a specially designed cyberattack detection module, as shown in Fig. 9. The diagnoser consists of a state estimator and a cyber-attack detection strategy as presented in Section III. The designing process of the detection module follows the steps shown in flowchart Fig. 7. Implemented in dSPACE, the diagnoser collects real-time local sensor measurements and communication data. Working parallelly with the local controller, the detection module generates a detection signal and triggers the alarm in case of any cyber-attack. Here we choose ES-2 as the test subject and ignore the duplicated results on different ESs. Potential cyber-attacks may change the data sent from ES-1 to ES-2 and ES-3 to ES-2 as marked in red in Fig.8. The parameters of the experimental setup are listed in TABLE I.

B. EXPERIMENTAL RESULTS: DETECTION OF CYBER-ATTACK
The proposed cyber-attack detection method is verified by experimental results, as shown in Fig. 10. A cyber-attack occurs in the communication link between ES-1 and ES-2 at t=13.13s. The attack changes the ES-2's received data Ves1 (ES-1 voltage) to a sinusoidal signal with a constant 30V amplitude (RMS value 21.21V). The attacked signal has the same phase with the real Ves1 signal in the test. Before the attack, the line voltages Vs1, Vs2, Vs3 are well regulated at 110V (RMS value) and Ves2=30V(RMS). The constant attack is then injected into the consensus control loop to change the consensus equilibrium of the system. As protection operation is not considered yet, the attack will keep influencing the system dynamic until the end. As shown in Fig. 10(b), ES-2 voltage drops immediately after the attack and moves slowly to a new equilibrium around 21.21V RMS. Consequently, all ESs reduce the output power and cannot support the system voltage adequately. This leads to a drop in the line voltage as shown in Fig.  10(a). Fig. 10 which is designed such that the overall probability of false alarm is 5% [12]. Note that instead of the original residual signals, the auxiliary variables , () ii T r k are used. In the test, we replace line current I21 and I23 by local net current ISL assuming only the net current is measurable, as

C. Experimental results: Countermeasure to tackle cyber-attack
Strategies to tackle cyber-attacks on distributed ESs depend on the persistence of the attack. For non-persistent attacks (e.g., within a few seconds), the original control data can be retained for the distributed ESs to ride through the attacks via consensus control. Tests based on the setup in Fig. 8 have been conducted. Fig. 11 shows the practical measurements of the nodal voltages (Vs1, Vs2 and Vs3) of the distributed ESs. The consensus control successfully maintains voltage regulations for the local voltages during the attack and also resumes normal services after the attack. Fig. 12 shows the experimental results when the attack is persistent. If the persistent attack lasts longer than the ridethrough period based on the previous control data, resuming consensus control with false data cannot lead to normal services as shown in Fig. 12. In this case, individual inbuilt droop control in each ES can take over so that normal voltage regulation can be achieved, despite that responsibility sharing among the distributed ESs will be abandoned temporarily. As mentioned previously in [4] and [20], the advantage of distributed ESs is that ESs can operate individually and collectively with droop control even if there is no cyber-layer for consensus control. But the availability of the consensus control in the cyber layer provides the extra advantage of allowing the distributed ESs to share responsibility in providing their voltage and/or frequency regulatory functions in the power grid.

V. CONCLUSIONS
In this paper, a distributed cyber-attack detection architecture for ESs in a microgrid and a countermeasure are proposed and practically evaluated. A local attack diagnoser composed of a state estimator and detection algorithm is designed for each ES. The residual signal and threshold are designed to perform the detection strategy. The protection mechanism will be triggered to smooth the dynamics during cyber-attacks. A practical evaluation of this distributed detection architecture is presented with experimental results. The proposed method can be applied in principle to detect cyber-attack in distributed control of grid-connected power electronic equipment. A strategy of countermeasure based on the persistence of the attack has been developed and implemented. It is confirmed that retention of the control data before the attack can allow the distributed ESs to ride through non-persistent cyber-attack with consensus control. However, if the cyber-attack persists, the ESs can revert to their individual droop control to continue voltage and/or frequency regulatory services. This important feature of distributed ESs provides extra robustness to power system stability. With the urgent need to increase renewable energy generation of intermittent nature to combat climate change, the consensus control of distributed ESs could offer a drastic solution to increase wind and solar power generation without causing power system instability.