TIMPANY - deTectIon of Model Poisoning Attacks usiNg accuracY

Nowadays, Federated Learning has widely been adopted for data security in the Industrial IoTs. With Federated Learning, local Industrial IoTs devices download the current machine learning model and update it on their own local Industrial IoTs devices. Then, local Industrial IoTs devices transmit these locally trained models back to the Industrial Server. The Industrial Server aggregates all the locally trained models into a single consolidated and enhanced global model. On one side, Federated Learning secures the data; on the other side, Federated Learning itself is vulnerable to one subtle yet severe attack: the model poisoning attack. Model poisoning attack is difficult to detect, especially in Industrial IoTs applications, for two reasons: a) neither the Industrial Server nor the local Industrial IoTs devices in Federated Learning is capable of identifying poisoned local models, and b) every iteration of Federated Learning consists of many Industrial IoTs devices, and therefore, verification of every single device is computationally expensive. Thus, this study proposes an effective and efficient framework for deTectIon of Model Poisoning Attacks usiNg AccuracY (TIMPANY). TIMPANY is the first detection framework for the model poisoning attack that utilizes accuracy as a detection measure. We performed theoretical analysis of TIMPANY with other detection solutions (for model poisoning attack) concerning communication and computational efficiency, security, and detection accuracy. Our thorough theoretical comparative analysis showed that TIMPANY efficiently addresses these open research challenges that previous studies failed to address. In our thorough experimental analysis, error analysis from the first iteration shows that TIMPANY results in 0% error, leading to a True Positive Rate and accuracy of 100% with 0% False Positive Rate. Thus, TIMPANY outperformed some of the existing detection solutions for model poisoning attacks against Federated Learning. We conclude that TIMPANY is effective and efficient against model poisoning attacks in Federated Learning, even for resource-constrained Industrial IoTs devices widely used in various industrial applications.


This paradigm of data generation violates the indepen-
Several studies have proposed to overcome the challenges 58 of expensive communication, system heterogeneity, and sta-59 tistical heterogeneity. For example, in [8], [10], [28], the various Industrial IoTs applications [18]- [23], as discussed 66 in Section II. 67 Therefore, in this study, we particularly tailored a detection 68 framework for model poisoning attacks against a Federated 69 Learning, which is viable for the industrial environment. Our 70 proposed framework is a novel privacy-preserving frame-71 work, TIMPANY, that utilizes accuracy as a detection mea-72 sure. 73 The acronyms and abbreviations used throughout this 74 study are mentioned in Table 1.

75
A. CONTRIBUTIONS 76 The main contributions of this study can be summarized as 77 follows. 78 1) The proposed framework, TIMPANY, analyzes the 79 local model weights for model poisoning attacks by 80 itself at the server side as clients have to send their 81 respective local model weights directly to the server. 82 Thus, clients' summarized weights are secure from 83 any other client or party in the FL. Hence, TIMPANY 84 maintains the privacy and security of every single client 85 within the FL setup.

86
2) TIMPANY follows vanilla FL and analyzes local 87 model weights at the server side. Thus, TIMPANY 88 does not require additional computation and commu-89 nication cost to detect poisoned local model weights. 90 Therefore, TIMPANY is a viable framework for any 91 environment, even for a resource-constraint IoT-based 92 industrial environment.

93
3) TIMPANY does not rely on clients of the FL for 94 analysis and evaluation of local model weights. There-95 fore, TIMPANY does not misevaluate the local model 96 weights. As a result, TIMPANY has significantly high 97 TPR and accuracy of 100% while having noticeably 98 low FPR of 0%.

99
The rest of the paper is organized as follows: Section 100 II describes some related work and some of the existing 101 research gaps. Section III presents our proposed solution 102 along with its detailed methodology. Section IV highlights 103 the experiments and evaluations of our proposed solution. 104 Finally, Section V concludes this paper along with the future 105 work and directions.

107
In [11]-[13], the authors have devised model poisoning 108 attacks employing a boosting mechanism and altering op-109 timization strategy for stealthy model poisoning. In [14] 110 and, [15], the authors conducted a thorough study involving 111 recent advances, open challenges and problems posed by FL. 112 In [16], the authors have used model poisoning attacks to 113 introduce backdoor attacks in FL. Their results showed that 114 attacks are possible with only 0.01% access to the devices. In 115 [17], the authors conducted a study on weight poisoning at-116 tacks (also known as model poisoning attacks) on pre-trained 117 models. The authors have discussed the defence and illus-118 trated the effectiveness of these for exposing the backdoors. 119 In [18], the authors proposed local model poisoning against 120 Byzantine-Robust FL. In [19], authors have demonstrated 121 model poisoning attacks through data poisoning, specifically, 122 label flipping poisoning attacks with which authors can also 123 negatively impact the global model.

124
In [18], the authors have also discussed generalized two 125 data poisoning defences against their proposed attacks based 126 on the largest error loss and negative impact. However, these 127 defences have limited success. The authors, in [19], presented 128 the dimensionality reduction-based defence mechanism ca-129  [23], the authors presented a stochastic quantization-based 160 detection framework for Byzantine clients. Their proposed 161 solution puts additional computation both on a server as 162 well as the client side in terms of clients' selection and 163 computation of pairwise distances.

164
In summary, those proposed detection solutions lack in 165 multiple aspects, thus leading to various main research gaps 166 and challenges. The first and foremost aspect is a need of 167 such a verification process for poisoned local model weights 168 which should be efficient and effective, i.e., the process 169 consumes less computation and communication cost while 170 maintaining the attacks detection accuracy (no/less miseval-171 VOLUME 4, 2016

180
In this study, we have proposed TIMPANY, a detection 181 framework for model poisoning attack as shown in Figure   182 1. It is the first framework that utilizes the accuracies of the 183 local model weights to identify and detect model poisoning 184 attacks in FL. Moreover, TIMPANY effectively overcomes 185 all the research gaps and challenges posed by previous stud-186 ies, as highlighted in Section II. Also, the working mecha-187 nism of TIMPANY enables TIMPANY a viable model poi-188 soning detection framework for various environments, even 189 for resource-constraint applications.
190 Figure 2 illustrates the functionalities and workflow of the 191 entire detection process in a module diagram form of Figure 192 1. Following steps further describe the mechanism of the 193 entire detection process.
where w m are the updated weights of each participant in Then, the server employs TIMPANY to identify and detect 219 poisoned local model weights.

220
Step 3a Step 3c: Evaluation of accuracy using quality control 233 charts 234 Thereafter, using quality control charts 1 , TIMPANY sets 235 the T based on accuracies computed using local model 236 weights. To compute the quality control charts, TIMPANY 237 , 238 where acc i = accuracies obtained from the local model 239 weights. Then TIMPANY computes CL as CL = X, UCL 240 as U CL = X + α * σ and LCL as LCL = X − α * σ where 241 α = range of limits, i.e., 1, 2, and 3.

242
At last, TIMPANY discards those local model weights 243 (as poisoned local model weights) based on their accuracies 244 which are below the computed T, i.e.,  the IIoT environment [24], and [25]. For example, in [26]-

300
For our experimentation, we divided the entire CIFAR-100 301 dataset into six datasets(subsets). Each sub-dataset consists 302 of 9,000 training images and 1,000 testing images having all 303 the 100 classes. There can be two types of cases through 304 which the malicious participant can generate and prepare 305 poisoned local model weights. For the first type of case, the 306 malicious participant trains the deployed model on a correct 307 dataset and then modifies the local model weights to make 308 those local model weights corrupted weights. In the second 309 type of case, the malicious participant trains the global model 310 on the poisoned dataset. Thus, resulting in poisoned local 311 model weights. Therefore, to cater latter possibility, we intro-312 duced Gaussian and Salt&Pepper noises to the sub dataset of 313 Participant F. Whereas, for the remaining cases, we utilized 314 the sub-datasets directly(unchanged) for Participants A, B, 315 C, D, and E. The details on the considered cases in this study, 316 characteristics of each participant, and how each participant 317 communicate with the server are mentioned in Section IV-C. 318

319
To demonstrate the applicability of our proposed framework, 320 TIMPANY, we have considered a scenario, where we have 321 conducted several experiments based on all possible model 322 poisoning attack cases as shown in Figure 3. In this sample 323 scenario, we take into account the following cases.  For the second case, we deployed the global model to 338 Participant B's device. Similar to Participant A, Participant 339 B also utilizes its dataset to train its local model and sum-340 marizes its local model weights. However, while sending the 341 summarized local model weights to the server, there was 342 a man-in-the-middle attack, i.e., the attacker was tapping 343 the communication channel between Participant B and the 344 server. Hence, during the communication, the attacker ma-345 nipulates the summarized local model weights (gradients) 346 and forwards the poisoned local model weights to the server. 347 In the third case, we also deployed the global model to 348 Participant C's device. However, in this case, the attacker 349 has physical access to Participant C's device. Therefore, 350 when Participant C trains its local model with its dataset and 351 summarizes its local model weights, the attacker modifies the 352 summarized local model weights on Participant C's device. 353 As a result, even if the communication channel between the 354 server and Participant C is secure, the summarized local 355 model weights sent to the server by Participant C are poi-356 soned local model weights.   Therefore, problems of FL, e.g., system heterogeneity and 370 statistical heterogeneity, are out of the scope of this study.

372
With the advent of sophisticated cyber attacks, it is becoming 373 essential to quantify the detection accuracy of the cyber se-374 curity systems against sophisticated cyber attacks. Therefore, 375 to quantify the TIMPANY's accuracy in detecting the model 376 poisoning attacks, we employed Percent Error (also known as 377 a Percentage Error [30]). Percent Error is a type of error used 378 to indicate the percentage of error in the analysis process. 379 Percent Error is a difference between the estimated values 380 and the actual values divided by the actual values as where, To evaluate and demonstrate the detection accuracy of 386 TIMPANY, we have also computed the values of the con-387 fusion matrix as shown in Table 2. Using the values of the confusion matrix, we then calcu-395 lated TPR (also known as Sensitivity, to measure the percent-  In contrast, to [20] and [23], which involve clients 455 in the identification and verification procedure of the 456 poison local model weights, TIMPANY, evaluates the 457 local model weights without indulging the clients in 458 the evaluation procedure at the server side. As a result, 459 TIMPANY provides complete security and privacy 460 to the clients, i.e., no information gets disclosed to 461 another client in the FL setup. The solution presented in [18] computes the largest 465 error rate from the local model weights. However, this 466 solution may incorrectly accept poisoned local model 467 weights as it only looks for the largest error loss. Thus, 468 this results in the misevaluation of the local model 469 weights.

470
Since in FL, each participant can access the global 471 model along with all the parameters, and the attackers 472 can make minor changes in the existing parameters 473 while keeping most of the parameters unchanged, the 474 approach of looking for unique characteristics between 475 poisoned and legitimate local model weights may 476 cause misevaluation of local model weights [19].

477
Next, the defence mechanism of [21] employs the 478 mean value of reconstruction errors as a T value to dis-479 tinguish between legitimate and poisoned local model 480 weights. This mechanism may also lead to severe mis-481 evaluations. Let us assume a sample scenario for [21], 482 where the number of legitimate local model weights is 483 more than the number of poisoned local model weights. 484 Then, the mean value of reconstruction errors would 485 shift towards the legitimate local model weights. Thus, 486 poisoned local model weights can be misevaluated as 487 legitimate ones and vice versa.

488
In [20] and [23], participants (specifically attackers) 489 can miscalculate the dissimilarity values and related 490 information. Thus, local model weights can be mise-491 valuated and severely corrupt the global model.

492
On the contrary to these existing defences, TIM-493 PANY evaluates the local model weights at the server 494 side using the accuracies of the local model weights 495   In [22], the authors utilized gradients to differentiate 517 both (legitimate and poisoned local model weights).

518
However, the approach results in misevaluations be-519 cause [22] lacks a technique to define a discriminating 520 and optimal decision boundary for differentiation. However, in the assumed sample scenario, the local model 544 weights received by the server from Participant B and Par-545 ticipant C are poisoned, i.e., the resultant accuracies of Par-546 ticipant B and Participant C on the initial testing dataset of 547 the global model should be low. Hence, the accuracies of 548 Participant B and Participant C are less than the computed 549 optimal T, which leads to rejection of Participant B and 550 Participant C local model weights. Figure 4 shows that Participant D and Participant F ac-552 curacies are less than the computed optimal T value and 553 discarded as the poisoned local model weights as those local 554 model weights degrade the performance of the global model. 555 Because Participant D and F are malicious participants who 556 intentionally transmit the poisoned local model weights to the 557 server. Hence, the local model weights of Participant D and 558 Participant F should result in low accuracies on the initial 559 testing dataset of the global model.  their proposed solution has achieved a system performance 596 of 100% in terms of F1-score. Therefore, we also evaluated 597 our TIMPANY with [21] in terms of F1-score.

TABLE 7. Performance Analysis of TIMPANY
Detection Solutions F1-score Li,Suyi,et al. [21] 100% TIMPANY (our proposed solution) 100% As shown in Table 6, both [21] and TIMPANY have 599 achieved an F1-score of 100%. In other words, both the 600 solutions have shown same performance. However, the so-601 lution proposed in [21] has three main limitations (e.g., 602 consumes additional computation cost, misevaluates of local 603 model weights and lacks optimized T value) which makes it 604 impracticable for various IIoT applications, particularly for 605 resource-constraint IIoT environments. On contrary, TIM-606 PANY efficiently overcomes the limitations of [21] while 607 maintaining an F1-score of 100%.

609
FL has not only revolutionized classical machine learning 610 but also provides a privacy-preserving mechanism for end-611 devices as well as allows decentralized learning in the in-612 dustrial environment. FL has proven itself to be one of the 613 potential solutions in IIoT applications, e.g., monitoring of 614 defected products, controlling of pressure and gases, au-615 tonomous quality control checks, to name a few. At the 616 same time, FL opens new security vulnerability and poses 617 a new security threat, i.e., model poisoning attack, which 618 can severely affect the decision-making and information 619 classification processes in the industrial environment. These 620 model poisoning attacks are difficult to detect because a) 621 neither server nor participants in the FL can detect poisoned 622 local models based on provided weights only, and b) there 623 are many random participants in every FL iteration. Thus, 624 verification of every participant is not viable computationally. 625 To address model poisoning attacks, we have proposed 626 TIMPANY. TIMPANY is the first detection framework for 627 model poisoning attacks which utilizes accuracy as a detec-628 tion measure. We have evaluated TIMPANY both theoreti-629 cally and experimentally. With theoretical analysis, we have 630 shown that TIMPANY addresses the existing research prob-631 lems which previous research studies fail to address, e.g., se-632 curity and privacy maintenance, optimized T value selection, 633 etc. Experimental analysis were conducted considering vari-634 ous possible cases of model poisoning attacks in the FL. Our 635 thorough experiments showed that our proposed TIMPANY 636 has secured a percentage error of 0%. This error analysis 637 further resulted in 100% accuracy and TPR while securing 638 an FPR of 0%. Thus, outperforming some of existing state-639 of-the-art detection solutions for model poisoning attacks. 640 Hence, we can conclude that TIMPANY can efficiently and 641 effectively detect the model poisoning attacks against FL.

642
In future work, we plan to investigate more sophisticated 643 and automated model poisoning attacks. Furthermore, we 644 will also plan to evaluate our TIMPANY against those so-645 phisticated model poisoning attacks. Our future directions 646 also include the enhancement of TIMPANY such that the 647 TIMPANY can cater to other issues, e.g., system and statisti-648 cal heterogeneity.