Resilient AC Microgrids Against Correlated Attacks

Multi-inverter AC microgrids increasingly rely on local embedded controllers and distributed communication networks to meet operational requirements, which makes the microgrids vulnerable to physical and cyber attacks. Conventional resilient control strategies generally assume that the attack signals are bounded and uncorrelated. In this paper, we study the ramifications of allowing the antagonistic inputs to be unbounded and correlated. We consider a two-layer hierarchy for networked multi-agent systems with two opposing teams on different directed communication graphs: a control protagonist team with cooperative multi-inverter microgrids and an attack antagonist team with interacting attackers. We consider three types of unbounded attack injections launched from the antagonist layer, namely, coordinated and correlated attacks on the sensor measurements, as well as generally unbounded attacks on actuator commands and communication channels. We propose a fully distributed control framework to guarantee uniform ultimate boundedness for the secondary frequency regulation and voltage containment of AC microgrids against malicious attacks. The proposed results are validated on a modified IEEE 34-bus test feeder system, which is emulated in a controller/hardware-in-the-loop environment.


I. INTRODUCTION
Multi-inverter AC microgrid systems increasingly rely on distributed control paradigms with information exchanged among local controllers of inverters on a sparse communication network [2]. Multi-agent consensus and containment results are employed to reach frequency regulation [3] and voltage bounds [4], respectively. The communication network among inverters poses a vulnerability [5], [6], particularly as individual inverters lack a global perspective with limited information from their neighbors. Malicious attackers could simultaneously launch attacks, at times in a coordinated fashion, on the sensors, actuators, and/or communication channels to undermine microgrid performance.
The associate editor coordinating the review of this manuscript and approving it for publication was Qiang Li .
There are generally two main approaches to dealing with cyberattacks in power grid systems. The first approach is to detect the compromised inverters [7], [8], [9], [10], which could then be removed or overcome. This approach could undermine the graphical connectivity and jeopardize the consensus protocol fundamental to distributed control approaches. Hence, a restriction on the upper bound of the number of compromised inverters is generally required. Alternatively, distributed resilient control protocols preserve an acceptable level of performance by mitigating the propagated impact of external disturbances and noises [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21]. The main idea is to devise local distributed control approaches to enhance the self-resilience of the microgrids against malicious attacks instead of detecting and removing/overcoming the corrupted agents. For example, the researchers in [16] have utilized a cyber-resilient distributed control strategy VOLUME 11, 2023 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ FIGURE 1. A two-layer networked MAS hierarchy with two opposing teams: a control protagonist layer consisting of a networked multi-inverter microgrid and an attack antagonist layer.
based on the frequency error observations from the neighbor agents. An observer-based resilient control strategy is proposed in [17] in which neighboring frequency and active power's signal errors are integrated into the conventional distributed control strategy as compensatory terms. However, these distributed strategies depend on secure additional data transmission, which an intruder can attack. In [18], a resilient distributed optimal frequency control considers an auxiliary network system with the original cooperative control system. Note that the existing distributed resilient control methods for microgrids generally deal with different types of attacks separately. A unified resilient control framework is needed to address the simultaneous attacks on sensors, actuators, and communication channels. Moreover, with few exceptions, e.g., [22] and [23], existing results mostly deal with bounded injections, which could fail for unbounded attacks purposely devised to maximize the damage [24]. Last but not least, conventional attack-detection methods generally work for independent and non-interacting bad measurements. With certain knowledge of the system modeling, the intelligent attackers may launch coordinated attacks to introduce correlated false data into the system state measurement so that it looks consistent with the attack-detection mechanism, and hence successfully bypass it [25], [26]. The vulnerability and impact assessments of such unobservable and correlated attack injections, in the context of state estimation in power systems, have already been studied in [27], [28], and [29].
We consider the simultaneous and coordinated attacks on multi-inverter microgrids, where the correlation among sensor attacks is formulated using a layer of the communication network. In particular, we consider a two-layer networked multi-agent systems (MAS) hierarchy with two opposing players, a control protagonist team of networked cooperative multi-inverters, and a coordinated attack antagonist team, on two different communication digraphs, as illustrated in Fig. 1. The antagonists are modeled such that their decisions on sensor attack injections on the protagonist layer are made in a feedback manner, being highly correlated with each other and following two unbounded commanding antagonists. With limited knowledge of the microgrids configuration, i.e., the number of inverters, the malicious attackers could systematically and efficiently construct and launch correlated sensor attack signals to the control protagonist layer to change the sensor measurements of the microgrids. Moreover, the malicious attackers could launch generally unbounded attacks on the distributed communication channels and input control signals of individual inverters. These could severely deteriorate the synchronization mechanism of the AC microgrid system. Note that although unbounded attacks can be detected, isolating compromised agents under such simultaneous and coordinated attacks could lead to communication network failure. The local resilient controller design makes the microgrid resilient to such attacks without the need to detect and isolate the compromised agents. The major contributions of this paper are: • A two-layer networked MAS hierarchy is introduced, consisting of the control protagonist team with cooperative multi-inverter microgrids and the antagonist team with interacting attackers. We consider three kinds of unbounded attack injections launched from the antagonist layer, i.e., attack injections on the communication links among inverters and/or between the inverters and leaders, actuator attacks on the control input signals of local inverters, and the correlated attacks on the sensor measurements. The unbounded attacks for AC and DC microgrids were studied in [22] and [23], respectively, the intact state measurements, however, were needed in the local controller design. In practice, local sensors could also be compromised by malicious attackers. Compared with [22] and [23], simultaneously correlated and interacting attacks on sensors are also addressed in this paper.
• A distributed resilient secondary control framework mitigates the effects of simultaneous and unbounded attacks on sensors, actuators, and communication channels. This method is refined to guarantee uniformly ultimately bounded (UUB) convergence for both frequency regulation and voltage containment without the need for any global information.
• The proposed resilient control framework is validated in a controller/hardware-in-the-loop (CHIL) setup for an AC microgrid under different attack scenarios.
The remainder of this paper is organized as follows: Section II offers preliminaries on graph theory, communication networks, and notations used. Section III presents the conventional cooperative secondary control protocols for AC microgrids. Section IV introduces the unbounded attack models and formulates the resilient secondary control problem for AC microgrids. The distributed resilient controller design is presented in Section V and verified in Section VI using a CHIL setup. The conclusion is drawn in Section VII.

A. GRAPH THEORY
Suppose that the interactions among the multi-inverter protagonist layer and the antagonist layer are represented by time-invariant weighted digraphs G p and G a , respectively.

B. COMMUNICATION NETWORK
There are N inverters and two leader nodes on the protagonist layer G p . The upper (lower) leader node launches the upper (lower) reference value to the neighboring inverters. Likewise, there are N following attackers and two leading attackers on the antagonist layer G a . g p ik is the pinning gain from the k th (either the upper or the lower) leader to the i th inverter on the protagonist layer. Likewise, g a ik is the one connecting the k th commanding attacker and the i th following attacker on the antagonist layer. g p ik > 0 (respectively, g a ik > 0) for a connection between the k th leader (respectively, k th commanding attacker) and the i th inverter (respectively, i th following attacker); Otherwise, g p ik = 0 (respectively, g a ik = 0). G p k = diag g p ik and G a k = diag g a ik . Note that G p and G a can be different.

C. NOTATIONS
σ min (X ) and σ max (X ) are the minimum and maximum singular values of matrix X , respectively. diag{·} denotes the block diagonal matrix. The Kronecker product is shown by ⊗.

III. CONVENTIONAL COOPERATIVE SECONDARY CONTROL
As illustrated in Fig.2, in a cooperative control structure for the i th inverter, the secondary control level acts as an actuator and provides the setpoints of the frequency and voltage terms to the decentralized primary droop control. The P − ω and Q − v droop characteristics are formulated as where P i and Q i are, respectively, the active and reactive powers. ω i and v i are the operating angular frequency and the direct component of the terminal voltage. ω n i and V n i denote the droop setpoints fed from the secondary controller. m P i and n Q i are P − ω and Q − v droop coefficients selected as per inverter's power ratings. Differentiating the droop characteristics (1) and (2) giveṡ where u f i and u v i are auxiliary control inputs. The distributed leader-follower containment approach is applied to accomplish the secondary frequency regulation and voltage containment for AC microgrids, by using the relative measurements from the neighboring inverters and leaders: where c fi , c vi ∈ R > 0 are the coupling gains.
Using (6) and (6), we rewrite (3) and (4) aṡ respectively, where ω n k = ω k +m P i P i and V n k = v k + n Q i Q i . Due to the coupling between the active power (respectively, reactive power) of each inverter and its angular frequency (respectively, voltage magnitude), the control protocols (10) and (10) ensure the synchronization of the local frequency and voltage in the steady state [3]. Thus, to synchronize both ω i and m P i P i (respectively, v i and n Q i Q i ), we can directly synchronize ω n i (respectively, V n i ). Next, we give the preliminaries on secondary voltage containment control with two leaders. A similar discussion can be applied in the context of secondary frequency control. For convenience, we denote V n i as V i hereafter.
Definition 1: The objective of the secondary voltage containment control is to make the local voltage of each inverter converge within the range of the two constant voltage references issued by the leaders, i.e., Then, the global form of (10) becomeṡ where . We then introduce the following global voltage containment error Assumption 1: There exists a directed path from (at least) one leader to each inverter on the protagonist layer G p . Moreover, d p i > 0 for each inverter, i.e., there exists (at least) one neighbor for the i th inverter.

IV. UNBOUNDED ATTACK MODELING AND THE FORMULATION OF ATTACK RESILIENCY
Since frequency regulation with a single reference value is a special case of voltage containment with two reference values, for brevity, in the following, we only present the problem formulation and convergence results for the secondary voltage control.

A. MODELING THE UNBOUNDED ATTACKS
We consider a two-layer networked MAS hierarchy with two opposing teams, where the malicious attackers lie on an antagonist layer, and the cooperative inverters lie on a protagonist layer, as depicted in Fig. 1. Local inverters confront correlated sensor attacks launched by the corresponding antagonists. Moreover, the malicious attackers launch generally unbounded signals to corrupt the communication channels and the control input channels of local inverters.
The intelligent antagonists try to deteriorate the performance of the AC microgrids by following two unbounded commanding attackers and launching coordinated sensor attacks to bypass certain bad data detection schemes. It should be noted that our proposed method can deal with any linear/nonlinear antagonist layer for sensor attacks with consensus-based interaction dynamics. For convenience, we give the following simplified dynamics of the i th attackeṙ where δ s i ∈ R and δ s k ∈ R are the states of the i th following attacker and the k th commanding attacker, respectively. δ s k (t) can be any unbounded signals satisfying boundedδ s k (t). It is seen that the decision on sensor attack injection of each following attacker is made in a real-time and feedback manner and highly correlated with the neighboring attackers.
Remark 1: With limited information on the microgrids system configuration, i.e., the number of inverters, N , the intelligent attackers could systematically and efficiently construct and launch correlated sensor attack signals in (14) to change the sensor measurements of the microgrids in arbitrary ways. Note that, as indicated in (14), although the two communication networks in Fig. 1 share the same number of agents, the communication topology of these two networks could be totally different.
We consider three kinds of unbounded attack injections launched from the antagonist layer, namely, correlated attacks to the sensors, general unbounded attacks to the communication channels, and control input channels. Hence, (10) under attacks becomeṡ whereV i denotes the corrupted sensor measurement of V i . V i,j andV i,k are the compromised delivered measurements ofV j andV k at the i th inverter, respectively. In particular, we describe these compromised measurements as where δ s i and δ s k denote the sensor attack at the i th inverter and the k th leader, respectively. δ c ij denotes the injection to the communication link from the j th inverter to the i th inverter, and δ c ik denotes the communication channel attack from the k th leader to the i th inverter. It is seen that the attackers can launch unbounded attacks on the AC microgrid by corrupting the local sensor measurements as reflected in δ s i and δ s k (sensor attack), intercepting the communication channels as shown in δ c ij and δ c ik (communication channel attack), and distorting the local input control signal as shown in δ a i (actuator attack).
Assumption 2:δ a i ,δ c ij , andδ c ik are bounded. Remark 2: The signals with excessively fast-varying values could be easily detected. In the event that the attacker does inject an attack signal with an infinite magnitude of the rate of change, the cooperative microgrids can incorporate a defensive mechanism to detect and reject such injection. Hence, one can suppose that Assumption 2 holds. Since the intentionally injected attacks could be unbounded, the bounded noises and/or disturbances that are unintentionally caused can also be addressed using the resilient controller to be designed.

B. ATTACK-RESILIENT PROBLEM FORMULATION
We first analyze the synchronization performance for the antagonist layer. Define the global synchronization error on the antagonist layer as where x (t) ∈ R N , A ∈ R N ×N is Hurwitz, and µ (t) ∈ R N is bounded and piecewise continuous for all t ≥ t 0 . Then, for any x (t 0 ), x (t) is bounded. Proof: Since A is Hurwitz, for any M = M T 0, there exists P = P T 0 such that PA + A T P = −M . One can pick the following Lyapunov function candidate where its time-derivative is given aṡ Using Sylvester's inequality, ∀ x ≥ σ max (P) µ σ min (M ) , we obtaiṅ Since µ is bounded, from the LaSalle's invariance principle [31], x is bounded by σ max (P) µ σ min (M ) . Lemma 3: Consider the attackers' dynamics in (14). Then, the sensor attack δ s i in (16) is unbounded if there is a directed path from at least one commanding attacker to the i th following attacker on the antagonist layer. Proof: Use (14) to obtaiṅ Use (17) and (22) to obtaiṅ Using Lemma 1, we similarly obtain that k∈L a k is positive-definite, hence − k∈L a k is stable. From Lemma 2, the global synchronization error η in (17) is UUB. Hence, we obtain that δ s i stays in the small neighborhood around the range spanned by δ s k . Since δ s k is unbounded, δ s i is also unbounded. This completes the proof. Next, we study the vulnerability assessment of the conventional secondary voltage control approach under unbounded and correlated attacks. Rewrite (15) aṡ which can be considered as the overall attack information gathered at the i th inverter due to the network propagation. Since δ s i , δ s k , δ c ij , δ c ik , and δ a i are unbounded, the conventional secondary control fails in preserving the system stability and achieving the voltage regulation. It is necessary to design a secure and attack-resilient control method to guarantee closed-loop stability and voltage containment performance. The following result is needed to formulate our problem.
Definition 2 ( [32]): Signal x(t) is UUB with the ultimate bound b > 0 if there exists a constant c > 0, independent of t 0 ≥ 0, and for every a ∈ (0, c), there exists τ = τ (a, b) ≥ 0, independent of t 0 , such that The resilient voltage containment problem is introduced.

Definition 3 (Resilient Voltage Containment Problem):
Consider a two-layer hierarchical communication network. The resilient voltage containment problem is to design local input u v i in (4) such that e v in (13) is UUB under the unbounded actuator and communication channel attacks and the correlated sensor attacks described in (15) and (16). That is, the local voltage term converges within a small neighborhood around the range of the two voltage references.

V. FULLY DISTRIBUTED RESILIENT DESIGN
Consider the following measurable error term whereV i is the estimation of the uncorrupted voltage term andδ s i is the estimation of the sensor attack. To cope with the correlated sensor attacks and unbounded actuator and communication channel attacks, we present the following overall resilient control framework whereˆ i is the estimate of i in (25). The observers (28), (29), and (30) serve to compute the compensational signalˆ i to be used in (27). Figure 3 shows an inverter-based AC microgrid, where the proposed distributed resilient voltage containment control framework is also illustrated. A cyber layer with two opposing hierarchical communication networks is spanned among inverters. The sensor and actuator channels on the multi-inverter protagonist layer are attacked by the antagonist layer. Define the following global containment error of the voltage estimation whereê T . Next, we analyze the convergence result using the proposed resilient method.
Theorem 1: Given Assumptions 1 and 2, and using the resilient control protocols consisting of (27), (28), (29) and (30), e v in (13) is UUB, i.e., the attack-resilient voltage containment problem is solved. Cyber-physical AC microgrids: Cyber layer with two hierarchical communication networks, control layer including the primary control and the attack-resilient secondary voltage containment control, and the physical layer including inverters, distribution lines, and sources/loads. The attack-resilient secondary frequency regulation is also considered, but not shown here.
Proof: We first study the convergence result ofV i . Combing (28) and (31) yieldṡ Next, we prove thatṼ i and θ are both bounded. Using (25), (27), and (28), and since θ i =Ṽ i +δ s i , we obtaiṅ where l (14) and (29) yieldṡ where where and From Lemma 3 and Assumption 2, i and i are bounded. For inverter i, consider γ i as a bounded disturbance. In the following, we prove that A i in (39) is Hurwitz. From Assumption 1, one has −c vi d Then, to prove that A i is Hurwitz, we only need to prove that i is Hurwitz. Introducing U = 1 −1 0 1 yields which is Hurwitz. Hence, i is Hurwitz. Furthermore, A i is Hurwitz. Using Lemma 2, we obtain that ξ i is bounded. Therefore,Ṽ and θ i =Ṽ i +δ s i are both bounded. Hence,ê v is also bounded. Note that Hence, e v is also UUB. That is, the attack-resilient voltage containment problem is solved.
Remark 3: As seen, compared with [22] and [23], simultaneously correlated attacks on sensors are also addressed in this paper. The upshot is the intact state measurements are not available in the local controller design; instead, the measurable error term θ i is used in (28) to build the state estimation observer. Moreover, an additional observer is developed in (29) to estimate the sensor attack signals. These result in the more complicated augmented error dynamical systems shown in (37).

VI. CONTROLLER/HARDWARE-IN-THE-LOOP EVALUATION A. MULTI-INVERTER AC MICROGRID TESTBED
We evaluate the proposed controller on an IEEE 34-bus feeder system, islanded from the bulk grid at bus 800 and modified to include four inverters and two leader references, as shown in Fig. 4. The power distribution network and inverter data are chosen as in [3] and [33], respectively, with slight modifications. Each inverter is connected to the feeder lines using a Y-Y, 480 V/24.9 kV, 400 kVA transformer. The nominal reference frequency is 376.99 rad/s. All four loads in Fig.4 Fig. 4, the antagonist layer includes four following attackers and two commanding attackers with their dynamics shown in (14). Note that δ s k (t) could be any unbounded signals with

B. ATTACKS ON COMMUNICATION LINKS
The performance of the proposed resilient controller is compared against the conventional cooperative controller under unbounded attacks on communication links. Initially, both controllers are at the steady state, as presented in Fig. 5. The frequency is at its nominal reference value. The voltage of the secondary control was held at the nominal value whereas, with resilient control, the voltages are held between the upper and lower bounds. At t = 5s, an unbounded communication attack took place on communication links 1 → 2 and 1 → 4 with an attack value of δ c 21 = δ c 41 = 0.01t and δ c 21 = δ c 41 = 1t for frequency and voltage signals, respectively. As seen in Fig. 5(a), under unbounded communication attacks, the conventional secondary controller fails to remain stable. On the contrary, the proposed attack-resilient control  achieves the UUB regulation on the frequency term and maintains the magnitudes of the bus voltages within a small neighborhood spanned by the upper and lower bounds.

C. ATTACKS ON ACTUATORS
The control input signals of inverters 1 and 3 are under unbounded attack injections at t = 5s with δ a 1 = δ a 3 = t and δ a 1 = δ c 3 = 10t for frequency and voltage signals, respectively. From Fig. 6(a), it is clear that the conventional secondary control is unstable for the actuator attacks, whereas the resilient controller shows only a very slight deviation from the nominal reference value. Similarly, the voltages are held within a small neighborhood around the upper and lower bounds, as shown in Fig.6(b).

D. ATTACKS ON COMMUNICATION LINES, ACTUATORS, AND SENSORS
The resilient controller is evaluated under consecutive attacks on communication, actuators, and sensors. Initially, the inverters are at the steady state employing the resilient control paradigm. The communication link attacks, described in Section VI.B, and the actuator attacks, discussed in Section VI.C, are initiated simultaneously at t = 5s. The correlated sensor attacks launched from the antagonist layer are initiated at t = 15s for both frequency and voltage measurements. Figure 7 shows that, even under all these unbounded attacks, the proposed resilient controller guarantees the system stability and achieves the UUB convergence result for both frequency regulation and voltage containment objectives.

VII. CONCLUSION
We have introduced a two-layer hierarchy for the networked control of AC microgrids with two opposing layers, i.e., a multi-inverter protagonist layer and an antagonist layer with interacting attackers. One launches simultaneous and correlated sensor attacks and generally unbounded injections to the communication channels and control input channels of the AC microgrid layer. A distributed resilient secondary control architecture is proposed to maintain microgrid stability and preserve the UUB convergence result for the frequency regulation and voltage containment control objectives. The proposed control technique has been verified for a modified IEEE feeder system in a CHIL environment. Future work will focus on relaxing Assumption 2 to address generally unknown and unbounded attack injections.

ACKNOWLEDGMENT
The technical content has been approved for public release under DCN# 43-10272-22. Any opinions, findings, and conclusion or recommendations expressed in this article are those of the author(s) and do not necessarily reflect the views of the Office of Naval Research. This work draws from an unpublished chapter in the Shan Zuo's article-based Ph.D. dissertation [1] and has not been published before.