Resilient Distributed Optimization Against Cyber-Attacks

This letter presents a novel resilient distributed optimization algorithm for a network of agents. It guarantees that the local estimates of the agents converge close to the optimal solution of the original optimization problem, even in the presence of unknown but bounded attacks on both the agent’s local computation and communication network. To this end, a virtual state variable is introduced, which also enables each agent to identify the compromised communication links in real-time and in a distributed manner. The distributed algorithm imposes no restrictions on the maximum tolerable number of attacks, and it does not require high network connectivity and trusted/secured agents or edges. A numerical example is provided to illustrate the results.


I. INTRODUCTION
D ISTRIBUTED optimization has received significant in- terest over the last decade, driven by its numerous potential applications, including sensor networks [1], intelligent transportation system [2], multi-robot coordination [3], power systems [4], and machine learning [5].In a distributed optimization problem, a network of agents which share the same decision variable cooperate over a communication network to minimize a global objective function.This global objective function is the summation of local objective functions of individual agents.Distributed optimization is highly desirable over the centralized framework due to several key advantages, including its scalability, robustness to a single point of failure, and the potential to preserve the privacy of the agent's local information, such as their local objective function.However, the reliance on a communication network exposes the networked agents to cyber-attacks [6].In particular, an attacker could inject malicious signals into the agent's local computation and/or the communication links, driving the solutions of the agents away from the global minimizer and, in the worst case, preventing the local estimation of the agents from converging [7].Therefore, it is crucial to ensure the resilient operation The work of A. Gusrialdi was supported by the Research Council of Finland under academy project decision no.330073.The work of Z. Qu was supported in part by US Department of Energy's awards DE-EE0007998, DE-EE0009028, DE-EE0009152, and DE-EE0009339.
Zhihua Qu is with the Department of Electrical and Computer Engineering, University of Central Florida, Orlando, FL 32816 USA (email: qu@ucf.edu).
of the distributed optimization algorithms in the presence of unknown cyber-attacks.
Resilient distributed optimization in the presence of cyberattacks has received increased attention in the last few years.The line of work [8], [9] focus on designing resilient distributed optimization algorithms against adversarial agents.These agents, whether faulty or under attacker's control, transmit incorrect information to neighboring agents.For example, the work [8] proposes an algorithm removing extreme local estimates of F agents at each iteration, converging to a convex hull of non-adversarial agents' local minimizers.However, this assumes agents know the maximum number of adversarial agents and requires high network connectivity.In contrast, the work [9] introduces a method based on trusted agents forming a connected dominating set, ensuring convergence to a final solution within the convex set of minimizers but faces challenges in practical scenarios.Notably, the work in [8], [9] do not distinguish between adversarial agent presence and communication network attacks.As a result, the convergence of local estimates to the optimal solution of the original problem in the absence of adversarial nodes and with or without compromised communication links, is not guaranteed.The work [10] proposes a distributed reputation-based strategy aimed to identify and isolate adversarial agents that transmit the same incorrect information to all their neighbors.However, it requires increased communication burden and its effectiveness hinges on the selection of an appropriate threshold value.Moreover, there is a risk that the system may become unstable before the adversarial agents are successfully identified.
This paper focuses on a scenario in which both the communication network and the update of the agent's local estimate (agent's local computation) are subjected to unknown attacks.In contrast to the work in [11], [12], which focus on denial-ofservice attacks on the communication network, this paper aims to design a resilient distributed optimization algorithm against false data injection (FDI) attacks.A novel filtering strategy is proposed in [13], ensuring that the local estimates of the agents converge to the original optimization problem under FDI attacks on the communication network.However, this strategy comes with certain limitations, including restrictions on the maximum number of links that can be attacked and constraints on the network's connectivity.Moreover, it is assumed that at least one outgoing link of each agent is secure.A resilient distributed optimization based on trusted edges is proposed in [14], ensuring that the local estimates converge near the optimal solution.However, it requires that at least one edge for every agent be secure or trusted and that these edges are also known to all nodes within the network.In practice, this requirement is challenging to realize.Finally, the authors in [15] propose a resilient distributed optimization algorithm that does not impose any restrictions on the maximum number of links that can be attacked.However, it is assumed that the adversary can only inject the same quantity of attacks into the links connected to an agent.Moreover, the work does not take into account attacks on the update of the agent's local estimate, that is the agent's local computation.
This paper presents a novel resilient distributed optimization algorithm which ensures the local estimates of agents converge closely to the optimal solution of the original optimization problem, even under unknown but bounded attacks on both the agent's update of its local estimate and communication network.To achieve this, a virtual state variable is introduced, which also enables each agent to identify compromised communication links in a real-time and distributed manner.Specifically, the main contributions of the proposed resilient distributed optimization algorithm are listed as follows: • In contrast to the settings in [13]- [15], we consider a scenario where both the agent's update of its local estimate and communication channel are subjected to FDI attacks.Moreover, the adversary is able to insert different injections into each communication link.• Unlike the strategies in [8], [9], the proposed strategy ensures that all local estimates converge closely to the optimal solution of the optimization problem.• In contrast to the strategies in [8], [9], [13], [14], the proposed strategy imposes no restrictions on the maximum number of tolerable adversarial edges and agents.Moreover, it requires neither high network connectivity nor the existence of trusted/secured agents or edges.• Unlike the method in [10], the distributed attack identification method in this paper does not require a threshold value and two hop information.Most importantly, it ensures the system's stability during the identification.The paper is organized as follows.After formally formulating the problem in Section II, the proposed resilient distributed optimization algorithm together with the distributed identification strategy are presented and analyzed in Section III.The proposed algorithm is demonstrated via a numerical example in Section IV.Concluding remarks are presented in Section V.

II. PROBLEM FORMULATION A. Notations and Preliminaries
The Euclidean norm of a column vector x ∈ R n is denoted by ∥ x ∥, and x T is the transpose of a vector x.We use I n to denote the n × n identity matrix.
The gradient of a differentiable function f : , where ∂f ∂xi represents the derivative of f with respect to x i when all other component of x are fixed.A function is said to be locally Lipschitz if for every x o ∈ R n , there exits a neighborhood A differentiable function f is strictly convex if and only if Next, we provide a background of graph theory that we will use in the sequel.Let G = (V, E) be an undirected graph with a set of nodes V = {1, 2, • • • , n} and a set of edges E ⊂ V × V.An edge (i, j) ∈ E denotes that node j can receive information from node i.Since graph G is undirected, we have (i, j) ∈ E ⇔ (j, i) ∈ E. The neighbor set of node i is defined as N i = {j|(j, i) ∈ E, j ̸ = i}, and |N i | is the cardinal number of set N i .The undirected graph G is connected if there exists no isolated nodes in the graph.A Laplacian matrix L associated with an undirected graph G is defined as All the eigenvalues of L are nonnegative.Furthermore, if the graph G is connected, the null space of L associated with G is one-dimensional and spanned by the vector 1 n .

B. Problem Statement
Consider a network of n agents, where every agent has a local (private) objective function f i : R → R. The agents aim to distributedly solve the following optimization problem: where the functions f 1 , • • • , f n and the solution to (2) satisfy the following assumption.
Assumption 1.The functions f 1 , f 2 , • • • , f n are strictly convex and continuously differentiable.Furthermore, the gradients ∇f i for all i ∈ {1, 2 • • • , n} are locally Lipschitz.In addition, the set of optimal solutions to the problem ( 2) is non-empty, and that the minimum of the sum of all the functions is finite.
Remark 1.The assumption on ∇f i to be locally Lipschitz can be found in different real-world examples such as localization using networked cameras [16] and demand response applications in power distribution systems [17].
Let x * ∈ R denote the optimal solution to (2).Furthermore, let us introduce a stacked column vector estimate about the optimal solution x * .The problem in (2) can then be written in an equivalent form as minimize It is worth noting that if the optimal solution is p-dimensional, then x * ∈ R p .In this case, the above formulation and subsequent discussion can be analogously carried out by defining Next, let us assume that the agents can communicate between themselves whose communication network topology is modelled as graph G which satisfies the following assumption.
Assumption 2. Graph G is undirected, static, and connected.< l a t e x i t s h a 1 _ b a s e 6 4 = " 6 4 M a p t 8 s t H s J X z 6 K u 6 A u f C 2 j / n s = " > A A A B 6 n i c b V B N S 8 N A E J 3 U r 1 q / q h 6 9 L B b B U 0 l U q s e C F 4 8 < l a t e x i t s h a 1 _ b a s e 6 4 = " t o P 9 n d 9 T h 9 s p d S E 2 In order to obtain the solution to (3) each agent updates its local estimate x i according to the following update rule where u i (t) is the update of agent i's local estimate (agent i's local computation), ℓ i (t) denotes the local information (which depends, e.g., on x i (t)) of agent i, and y j (t), which will be defined later, is the information that agent j sends to its neighboring agent i.
In practice, the updates of the agents' local estimates and the communication channels are vulnerable to cyber-attacks.In this paper, we consider FDI attacks on both the agent's update of its local estimate and communication channels as illustrated in Fig. 1.The adversary aims to destabilize (4) or to prevent the agents from achieving optimality as will be illustrated in Example 1. Specifically, the attack on agent i's update of its local estimate can be modelled as where u a i (t) is the compromised update of the agent's local estimate under unknown injection δ ui (t).Furthermore, agent i may not receive the true information from its neighboring agent j, that is the possibly corrupted information that agent i is receiving from its neighboring agent j takes the following form where δ ij (t) is the malicious signal injected into the communication link (j, i) ∈ E. In contrast to prior works [8], [9], [13], [14] which limit the number of attacks while leaving their magnitude unrestricted, we do not impose any restrictions on the number of attacks.However, we assume that the attacks have a bounded magnitude, i.e., the injections satisfy the following assumption.
Assumption 3. The injections δ ui (t), δ ij (t) and their derivatives are all uniformly bounded.
Remark 2. Assumption 3 is reasonable in practice since physical signals typically have known ranges, for instance, in several studies of power system applications [18], [19].Furthermore, an injection of unbounded magnitude can be easily rejected using a threshold check [20].That is, given the proposed update rule, which will be described later, the agent utilizes the knowledge of the ranges of physical state Fig. 2: Distributed optimization algorithm (9) under cyberattacks.The attacker was able to prevent the agents from estimating the optimal solution x * = 1.
x i (t), auxiliary variable z i (t) and the gain β to estimate the ranges of y j and u i .Subsequently, it verifies and disregards the (potentially corrupted) received information y a ij (t) if it falls outside the estimated range.Remark 3. As an example, the bounded injections δ ui (t) (and similarly δ ij (t)) can take one of the following forms: 1) Uniformly bounded injections: That is, 2) Finite-gain injections: Injection δ ui (t) is generated by exogenous finite-L 2 -gain dynamics of the state x(t), that is, δ ui (t) satisfies the following differential equation δui (t) = g(δ ui (t), x(t)).
This can be interpreted as the input-to-state (i.e., from x(t) to δ ui (t)) stability of the attack dynamics.
The update of local estimate (4) in the presence of unknown attacks can then be written as Example 1.The authors in [21] proposes a distributed PI algorithm to solve (3) whose update rule (4) is given by ) where z i (t) is an auxiliary state of node i.Here, the local information is and the information sent by node j is y j (t) = {x j (t), z j (t)}.An adversary could destabilize (9), as illustrated in Fig. 2, by inserting bounded injections into the local computation and communication links designed in Section IV.
The objective of this paper is to develop the update law in (4) so that in presence of unknown but bounded cyberattacks, the local estimate of the agents x i (t) converge closely to the optimal solution x * of (3), i.e., for any ϵ > 0, lim

III. MAIN RESULTS
In order to make all the agents' local estimates achieve (10) in the presence of unknown but bounded attacks, we design the update rule (4) as follows: where local information ℓ i (t) = {ℓ i,1 (t), ℓ i,2 (t)} are given by and the information that agent j sends to its neighboring agent y j (t) = {y j,1 (t), y j,2 (t), y j,3 (t)} are given by y j,1 (t) = x j (t), y j,2 (t) = βz j (t), y j,3 (t) = z j (t) + βx j (t).
(12) In the above, the scalar gain β > 0 will be designed for ensuring resiliency.Note that in contrast to the physical state x i (t), the auxiliary variable z i (t) does not have any physical meaning and thus it is also called as a virtual state variable.The local estimate's update rule (11) in the presence of unknown attacks can then be written as where the adversary may insert different injections δ ij,1 (t), δ ij,2 (t), δ ij,3 (t) into different information being exchanged given in (12).
Next, by defining vectors where L denotes the Laplacian matrix associated with the communication network topology G, matrix In the following it will be shown that the update rule ( 14) ensures (10) and the proposed resilient distributed optimization algorithm also enables the agents to distributively identify the compromised communication links.

A. Analysis of Resilient Distributed Optimization
Theorem 1 below is the main result of this paper and shows that the objective (10) is achieved under update law (14).
Theorem 1.Consider the dynamics (14) where the local objective functions f i and its gradient satisfy Assumption 1, the communication graph G associated with Laplacian matrix L satisfies Assumption 2, and the injections d(t), d ′ (t) satisfy Assumption 3.Then, for any given ϵ > 0, the objective in (10) is achieved under all large values of β.
T , z e , d e , and d ′e denote the equilibria of system (14), respectively.At the steady state of ( 14 Substracting ( 15) from ( 14) and defining the error states the error system becomes (16) Next, consider the following Lyapunov function Taking the derivative of V along ( 16) yields Furthermore, as the injections and their derivatives are uniformly bounded, the term x i dominates all the other terms in V1 and thus it can be concluded that V1 < 0 for any sufficiently large gain β > 0. Therefore, we have x(t) → 0, that is x(t) → x e for any sufficient large gain β > 0.
The next step is to prove that x e converges to x * 1 n as the value of β increases.The second equation in (15) yields Hence, we have x e → c1 n for some c ∈ R and for all sufficiently large values of β.Next, multiplying the first equation in (15) from the left by 1 T n and substituting x e i = c results in We can write the above equation as Observe that n i=1 ∇f i (c) → 0 as the gain β increases.Furthermore, since n i=1 f i (x) is strictly convex we have n i=1 ∇f i (x * ) = 0 where the optimal solution x * to (2) is unique.Comparing the above two equalities, it can be concluded that c → x * and thus x e → x * 1 n .Remark 4. One can determine the gain β in practice through the following approaches: (i) initiate with a considerably large β based on the worst estimate of the attacker's injections; (ii) in cases where estimating the worst injection magnitude is challenging, commence with a lower beta to guarantee the boundedness of local estimates, subsequently increasing beta until achieving satisfactory performance.

B. Distributed Attack Identification on the Communication Links
Another feature of the proposed resilient control algorithm is its ability to identify in real-time and distributed fashion whether the communication link (j, i) ∈ E is compromised.The idea is that for agent i to estimate the physical state x j (t) using the possibly compromised information y a ij,1 , y a ij,2 , y a ij,3 received from agent j and are defined as y a ij,γ (t) = y j,γ (t) + δ ij,γ (t), j ∈ N i , γ = 1, 2, 3 where y j,γ (t) is given in (12).To this end, using the compromised information y a ij,2 (t), y a ij,3 (t) and for a given communication link (j, i) ∈ E, node i can estimate the neighboring physical state x j (t) according to Next, we propose the following test criterion for node i to detect if the information received from node j, i.e., the communication link (j, i) ∈ E, is being corrupted at time t.
Detection test: The following proposition summarizes the result on distributed attack identification on the communication link.
Proposition 1.Given the detection test criterion (18) and a communication link (j, i) ∈ E. We have the following results.Remark 5.In order to launch stealthy attacks, the adversary must know or learn the structure of y j,1 , y j,2 , y j,3 in ( 12).However, it is very challenging for the adversary to learn the structure in (12) as they (including the gain β and the auxiliary state z i (t)) are considered as local information for each agent.Remark 6.After identifying the compromised links distributively, one can then remove those links and if necessary reconfigure the communication network topology, e.g., using the distributed algorithm proposed in [22], to ensure the network's connectivity.This strategy becomes practical in situations where increasing β to any values is not permissible.

IV. AN ILLUSTRATIVE EXAMPLE
In this section, the efficacy of the proposed resilient distributed optimization algorithm is demonstrated and evaluated via a numerical example.Consider a network of four agents whose network topology is shown in Fig. 1.The agents aim to solve distributively the optimization problem (3) with the local objective function of each agent given by To this end, each agent implements the resilient distributed optimization algorithm ( 14) under cyber-attacks where an adversary inserts injections to the updates of the local estimates of agents 2, 4 and also to the communication links between agents 1 and 2, 3 and 4, 2 and 4. Specifically, the injections d(t), d ′ (t) are launched according to where matrices S 1 = −0.5In , S 2 = −0.3In , and matrix B is randomly chosen.The simulation results for different gain values β = 20, 150, and 500 are shown in Fig. 3.It can be observed from Fig. 3a that at a lower β value, despite the bounded nature of the agents' local estimates, the consensus is not achieved and their values differ from the optimal solution x * = 1.As the gain β increases, the agents' local estimates reach consensus and their values approach x * = 1, as depicted in Figs.3b, 3c and further elucidated in the proof of Theorem 1.In addition, the agents' local estimates converge faster to the equilibrium point with an increase in the gain β.
Finally, each agent implements the detection test (18) to identify which communication links are compromised.As an example, agent 1 is able to identify that the communication link (2, 1) ∈ E is compromised while the link (3, 1) ∈ E is attack-free, as illustrated in Fig. 4.

V. CONCLUSION AND FUTURE WORK
This paper presents a novel resilient distributed optimization against unknown but bounded attacks on both the agent's update of its local estimate and communication network.It is shown that the proposed distributed algorithm ensures that local estimates of all the agents within the network converge The local estimates converge to a small neighborhood of the optimal solution x * = 1 as β increases.closely to the optimal solution of the original optimization problem.Furthermore, the proposed framework also enables each agent to identify in a real-time and distributed manner the compromised communication links.A numerical example demonstrates the efficacy of the proposed strategy.In the future we aim to investigate the resilient distributed optimization which involves the agent's physical/system dynamics.In addition, the issue of noisy local estimates will also be addressed.

3 < 4 Fig. 1 :
Fig. 1: A network of four agents communicating with each other to solve an optimization problem distributively where agent i has a local objective function f i .An attacker inserts bounded injections into both the communication links and local computation of the agents.

Fig. 3 :
Fig.3: Trajectories of the local estimates x i under update rule(11) for different values of β and in presence of cyber attacks.The local estimates converge to a small neighborhood of the optimal solution x * = 1 as β increases.
(18)f.If the equality(18)does not hold, it means that at least one of the values of δ ij,1 (t), δ ij,2 (t), δ ij,3 (t) is not equal to zero, i.e., at least one of the information being transmitted is corrupted.If equality(18)holds, that means either all the values of δ ij,1 (t), δ ij,2 (t), δ ij,3 (t) are equal to zero (i.e., there is no attacks on link (j, i))