Explainable Artificial Intelligence in CyberSecurity: A Survey

Nowadays, Artificial Intelligence (AI) is widely applied in every area of human being’s daily life. Despite the AI benefits, its application suffers from the opacity of complex internal mechanisms and doesn’t satisfy by design the principles of Explainable Artificial Intelligence (XAI). The lack of transparency further exacerbates the problem in the field of CyberSecurity because entrusting crucial decisions to a system that cannot explain itself presents obvious dangers. There are several methods in the literature capable of providing explainability of AI results. Anyway, the application of XAI in CyberSecurity can be a double-edged sword. It substantially improves the CyberSecurity practices but simultaneously leaves the system vulnerable to adversary attacks. Therefore, there is a need to analyze the state-of-the-art of XAI methods in CyberSecurity to provide a clear vision for future research. This study presents an in-depth examination of the application of XAI in CyberSecurity. It considers more than 300 papers to comprehensively analyze the main CyberSecurity application fields, like Intrusion Detection Systems, Malware detection, Phishing and Spam detection, BotNets detection, Fraud detection, Zero-Day vulnerabilities, Digital Forensics and Crypto-Jacking. Specifically, this study focuses on the explainability methods adopted or proposed in these fields, pointing out promising works and new challenges.


98
DARPA, the Defense Advanced Research Projects Agency, 99 financed the ''Explainable AI (XAI) Program'' at the begin-100 ning of 2017 [2]. XAI aims to develop more understandable 101 models while maintaining a high degree of learning perfor-102 mance (prediction accuracy); and enable human users to com- 103 prehend, adequately trust, and manage the future generation 104 of artificially intelligent partners.
presents the current taxonomy and makes a crucial dis- 145 tinction between true transparency (interpretable models) 146 and post-hoc interpretations (additional techniques used to 147

FIGURE 2. XAI Principles presented by NIST in [3].
shed transparency on complex black-box models). These 148 techniques include producing local explanations for spe- 149 cific inputs or the entire model globally. Following a quick 150 overview: 151 • Model Specific or Model Agnostic: This determines 152 whether or not the interpretation method is restricted 153 to a specific model. Model-specific methods and tools 154 are those that are specific to a model. Model agnos- 155 tic methods can be applied to any ML model to gain 156 interpretability. Internal model data such as weights and 157 structural details are not accessible to these models. 158 • Intrinsic or Extrinsic (post-hoc): This indicates 159 whether the model is interpretable on its own or whether 160 interpretability requires using methods that examine 161 models after training. Simple, comprehensible models, 162 like decision trees, are intrinsic. Utilizing an interpreta-163 tion strategy after training to achieve interpretability is 164 extrinsic.

165
• Local or Global: Whether the interpretation method 166 describes a single data record or all of a model's 167 behaviour depends on whether it is local or global. 168 Global methods and tools interpret the entire model, 169 whereas Local methods and tools only explain a single 170 prediction.
GRAD-CAM. Gradient-weighted Class Activation Map-220 ping (GRAD-CAM) [8] is a technique for producing a 221 class-specific heat map from a single image. Grad-CAM 222 produces a class discriminative localization map as a result. 223 The framework makes use of the feature maps generated by 224 a CNN's final convolutional layer. This is Local-based, Post-225 hoc but Model-Specific. 226 CEM. Contrastive Explanation Method (CEM) [9] pro-227 vides explanations for classification models. More in detail, 228 it retrieves the features that should be sufficiently present to 229 predict the same class for the input instance. It also iden- 230 tifies minimal features to change for associating the input 231 instance to a different class. This is Local-based, Post-hoc but 232 Model-Agnostic. 233 235 If it were measured as a country, Cybercrime, which inflicted 236 damages around $6 trillion globally in 2021, would be the 237 world's third-largest economy after the U. S. and China. 238 CyberSecurity Ventures expects global cybercrime costs to 239 grow by 15% per year over the next five years, reaching 240 $10.5 trillion annually by 2025, up from $3 trillion in 2015. 241 In addition to being exponentially more considerable than 242 the damage caused by natural disasters in a year, this rep- 243 resents the most significant transfer of economic wealth in 244 history and poses a threat to the incentives for innovation and 245 investment [10]. 246 CyberSecurity is the process of defending ICT systems 247 against various cyber threats or attacks. A ''cyber-attack'' 248 is any criminal activity that preys on electronic information 249 systems, networks, or infrastructure. Information is primarily 250 intended to be stolen, altered, or destroyed. In the current 251 cyber-attack situation, attack vectors that take advantage of a 252 lack of readiness and (system as well as human) preparedness 253 to access sensitive data or compromise systems are frequent. 254 The main problems of CyberSecurity are the knowledge of 255 various cyber-attacks and the development of complementary 256 protection mechanisms. 257 The risks usually connected to any attack take into account 258 three security variables: threats, who is attacking; vulnerabil-259 ities, or the holes they are attacking; and impacts, or what 260 the assault does. A security incident is an act that threatens 261 the confidentiality, integrity, or availability of information 262 assets and systems. Obtaining illegal access, destruction, and 263 alteration of information to harm possibly are just a few 264 examples of potential breaches and security violations on 265 a computer system or mobile device. Threats describe all 266 of the security mentioned above infractions' potential risk 267 and hazard, and attacks describe any attempts to commit 268 a violation. 269 Measures to safeguard information and communication 270 technology, the unprocessed data and information it con-271 tains, as well as their processing and transmission, associated 272 virtual and physical elements of the systems, the degree of 273 protection attained as a result of the application of those 274 IV. RELATED WORKS 296 The following sub-sections analyze the existing surveys High-performance AI systems, particularly those based 306 on DL, behave similarly to black boxes that provide 307 good results but can hardly justify a given output in a 308 human-understandable way [12], [13]. It is essential to min-309 imize potential biases (e.g., algorithmic, racial, ideological 310 and gender biases) during the ethical AI solution development 311 stage [14], [15]. 312 Adadi and Berrada [16] conducted an exhaustive literature  Abdul et al. [17]  of recent studies on visual interpretability of neural net-329 works, covering visualization and diagnosis of CNN (Con-330 volutional Neural Network) representations, techniques for 331 disentangling CNN representations into graphs or trees, and 332 learning of CNNs with disentangled and interpretable rep-333 resentations ending with a middle-to-end learning based on 334 model interpretability.

335
The authors of [19] employed a loss for each filter in 336 high-level convolutional layers to force each filter to learn 337 VOLUME 10,2022 An external or surrogate model and the base model are used 394 in post hoc approaches. The base model remains unmodified, 395 while the external model generates an explanation for the 396 users by mimicking the behavior of the base model. In addi-397 tion, post hoc approaches are classified into two groups: 398 model-agnostic and model-specific. Model-agnostic meth-399 ods can be used with any AI/ML model, but model-specific 400 approaches only apply to certain models. in [55], [56], [57], and [58].

473
The performance of seven DL models on the CSE-CIC-474 IDS2018 and Bot-IoT datasets is examined in [59]. The 475 models are evaluated on two datasets in this benchmark, and 476 three evaluation metrics are reported. The whole execution 477 of the study is made public in order to facilitate objective 478 comparisons and transparency in [60]. For the specific field 479 of phishing interesting approach is defined in [61] and for 480 ransomware attacks in [62]. two work focus exclusively on this area, which are [63], [64]. 489 However, it must be pointed out that in [63], the authors 490 provide a quick overview and, above all, do not pay attention 491 on the different applications within CyberSecurity. In [64]   Exciting work is [65] where the authors made three con-  In [66] Vigano et al. presented Explainable Security (XSec), 504 a new security paradigm that involves several different stake-505 holders and is multifaceted by nature. In [67] the authors 506 carried out a comprehensive literature review of various DL 507 architectures applied in CyberSecurity, including state-of-508 the-art studies conducted with explainable AI. Indeed, [68] 509 focuses on Android Malware Defenses and XAI applications 510 in this field; they point out that nine out of ten primary sources 511 are proposed after 2019, indicating that Explainable Deep 512 Learning approaches for malware defenses are a current hot 513 research topic.

514
Works analysed in this section are in the last 3 years, 515 i.e., from 2020 to 2022. Although all of these publications 516 are outstanding, none demonstrate how explainability occurs 517 in key sectors of AI in CyberSecurity, which is the primary 518 focus of this survey.

520
In the following subsections, the works that seek to achieve 521 explainability in the field of CyberSecurity were reviewed. 522 In particular, the discussion focuses on the following appli-523 cation fields: The template used for describing the results of the analysis 529 of the works falling in the above application fields is this: In addition to the CyberSecurity applications aforemen-543 tioned above, other fields will be treated with lesser level of 544 detail, due to the availability of a fewer number of works, 545 focusing only on the review of works using XAI, that are: 546 Fraud Detection, Zero-Day Vulnerabilities, Digital Foren-547 sics, and Crypto-Jacking. 548 All application fields were selected according to the rele-549 vance and volume of literature to the current state of the art. 550

551
Intrusion Detection Systems enable continuous security mon-552 itoring of a cyber perimeter in order to timely identify attacks 553 on computers and computer networks.

554
IDSs can be implemented with hardware appliances or with 555 special software; sometimes, they combine both systems [69]. 556 VOLUME 10, 2022  The approach based on ML and DL automates the analyti-612 cal process to find intrusions. High performance, adaptability, 613 flexibility, and the capacity to identify zero-day assaults are 614 the significant benefits of the ML technique. However, there 615 are some drawbacks to ML-based IDS, including high bias 616 propensity, inability to manage outliers, difficulties handling 617 huge datasets, and complex data preprocessing.

618
The DL-based approach can handle dynamic data 619 that changes over time, recognize large-scale and multi-620 dimensional data and identify anomalies in the data. Never-621 theless, DL-based approaches have many drawbacks, such as 622 a lack of flow information, vulnerability to evasion attempts, 623 poor data knowledge required to design relevant features, and 624 a lack of qualified domain experts to review the implemen-625 tation. These very latter two points lead back to the need for 626 explainability, a need shared by any agent attempting to give 627 an explanation for the model result and be able to improve it 628 consequently.  Chawla et al. [71] propose a Host-based IDS that uses 631 sequences of system calls to identify the expected behaviour 632 of a system. The work describes an efficient Anomaly-based 633 intrusion detection system based on CNN layers to capture 634 local correlations of structures in the sequences and Gated 635 Recurrent Units layer to learn sequential correlations from 636 the higher level features.

637
By examining Linux kernel 5.7.0-rc1, the authors of [72]    The general idea proposed in [95] against adversarial 702 attacks is divided into two parts, initialization and detection. 703 During initialization, the model is trained with an SVM and 704 features and characteristics that make a Normal classification 705 are deduced via LIME. During detection, the Intrusion Detec-706 tion System goes to compare. If it does not find the data as 707 Normal, it classifies as an attack. On the other hand, if it is 708 classified as Normal, there is a risk of an adversarial attack 709 that is fooling the model. So a further check is done by reusing 710 LIME. After that, the final result is reached.

711
FAIXID [96] is a new proposed framework that uses data 712 cleaning techniques. They used four algorithms in the exper-713 iment to make the results explainable. They use the Boolean 714 Rule Column Generation (BRCG) algorithm [97], which pro-715 vides a directly interpretable supervised learning method for 716 binary classification. Logistic Rule Regression (LogRR) [98] 717 is a directly interpretable supervised learning method that can 718 perform logistic regression on rule-based functions. The Pro-719 toDash algorithm [99] provides example-based explanations 720 to summarize datasets and explain the predictions of an AI 721 model. Finally, the Contrastive Explanations Method (CEM) 722 is used to compute explanations that highlight both relevant 723 positives (PP) and relevant negatives (NP). Their proposal is 724 not static but involves the use of algorithms depending on the 725 specific case.

726
The work proposed in [100] defines a method to make 727 rules for accessing the network dynamically and not statically 728 as, for example, the rules set in a firewall may be. Thus, 729 Explainability is the focus of the proposal. The explanation 730 of the results consists of two main steps: i) training a model to 731 approximate the local decision boundary of the target predic-732 tive model, and ii) reasoning about the trained model and the 733 given input based on an explanation logic. The explanation is 734 Local-based. They are inspired by LEMNA [101].

735
The aim in [102] is to increase transparency in an IDS 736 based on a Deep Neural Network. Feedback is presented by 737 computing the input features most relevant to the predictions 738 made by the system. The model adopted is an MLP. Two 739 forms of feedback are generated: 1) offline feedback (after 740 training, before deployment) and 2) online feedback (during 741 deployment). In offline feedback, the user is given the most 742 relevant input features for each concept learned from the sys-743 tem. This information allows the user to evaluate whether the 744 input characteristics that guide the IDS's decision toward a 745 particular class (i.e., the type of attack) align with the domain 746 experts' knowledge. On the other hand, the user is given the 747 most relevant input characteristics for each prediction in the 748 online feedback.  In conclusion, this work mentions [109], where an explain-793 able automotive intrusion detection system is proposed, 794 and [110] where a new general method is presented and tested 795 on an IDS dataset. In [111] instead, the authors emphasize the 796 importance of trust but do not use XAI methods.  It would be good to consider frameworks with intrinsic 806 interpretability and not the application of methods for a 807 post-hoc explanation. Furthermore, the final output should 808 be aimed at precise figures and not just any user, such as 809 analysts and defenders. To be explored for future research 810 is the topic of adversarial attacks where the collaboration 811 between humans and machines is necessary and explanations 812 are fundamental to combat this type of intrusion. The term malware refers to programs potentially harmful to 815 the user, which are aimed at stealing sensitive data, control-816 ling the PC, or stealing user identity. The term malware origi-817 nates from the contraction of the words ''malicious software'' 818 and stands for a program (an executable, a dynamic library, 819 a script, an HTML page, a document with macros, etc.) 820 having unwanted and potentially dangerous effects on the 821 user such as stealing sensitive data, controlling activity at the 822 PC, identity theft, encrypting the hard disk with subsequent 823 ransom demands, and so on.

824
Malware is usually classified according to its behaviour as 825 Botnet, Backdoor, Information Stealer, Downloaders, Scare-826 ware, Rootkit, Worm, Virus, Ransomware or Trojans. When using a Signature-based approach, programmers 836 scan a file for malware, compare the information with a 837 database of virus signatures, and then verify the results. If the 838 information matches the information in the database, the file 839 is infected with viruses. This approach limits the detection 840 of unknown malware, but its main advantage is that it works 841 well for known malware.

842
Anomaly-based methods mitigate the limitations of 843 signature-based techniques, allowing detection of any known 844 or unknown malware by applying classification techniques 845 to the actions of a system for malware detection. Detec-846 tion of malware activity is improved by moving from 847 pattern-based to classification-based detection to identify 848 normal or anomalous behaviour. Applying AI to Signature-849 based and Anomaly-based detection systems improves the 850 efficiency of malware detection. Heuristic-based method use 851 data mining and ML techniques to learn the behavior of an 852 executable file. Cloud-based environments. Similar are the works in [113] 873 and [114] where extreme surveillance through malware hunt-874 ing is delivered. Keeping with Anomaly-based approaches, 875 Alaeiyan et al. introduce [115]  remains the non-Explainability and the subsequent lack of 915 trust in model outcomes, so the next section will explore 916 works that somehow attempt to solve this problem.  XMal [138] is an MLP-based approach with an attention 935 mechanism to detect when an Android App is malware. The 936 interpretation phase aims to automatically produce neural 937 language descriptions to interpret key malicious behaviours 938 within apps. Although the method is not so clear, the authors 939 say they achieve better performance in interpretation than 940 LIME and DREBIN.

941
The authors in [139] propose a backtracking method to pro-942 vide a high-fidelity explanation of the DL detection method. 943 The backtracking method selects the most important features 944 VOLUME 10,2022 contributing to the classification decision, thus resulting in a binary, they try to use this framework to demystify the work-1001 ings of the MalConv architecture. As a result, they can better 1002 explain the workings of ML algorithms and the decisions 1003 they make using the proposed framework. Additionally, the 1004 analysis will enable network inspection without starting from 1005 scratch. 1006 Hsupeng et al. [148] introduce an explainable flow-data 1007 classification model for hacker attacks and malware detec-1008 tion. The flow data used for training the model is converted 1009 from packets by CICFlowMeter. This process significantly 1010 shrank the data size, reducing the requirement for data stor-1011 age. For Explainability, they utilize SHAP further to inves-1012 tigate the relation between cyberattacks and network flow 1013 features.

1014
MalDAE [149] is a framework that explores the dif-1015 ference and relation between the dynamic and static API 1016 call sequences, which are correlated and fused by seman-1017 tics mapping. MalDAE provides a practical and explainable 1018 framework for detecting and understanding malware based on 1019 correlation and fusion of the static and dynamic characteris-1020 tics. The explainable theoretical framework divides all API 1021 calls into several types of malicious behaviours according to 1022 their impact on security and builds a hierarchical malware 1023 explanation architecture.

1024
Several works in the literature attempt to interpret malware 1025 detection by generating Adversarial attacks. The authors 1026 in [150] discovered that MalConv neural network does not 1027 learn any useful characteristics for malware detection from 1028 the data and text sections of executable files but instead has 1029 a tendency to learn to distinguish between benign and mali-1030 cious samples based on the characteristics found in the file 1031 header. Based on this discovery, they devised a novel attack 1032 method that creates adversarial malware binaries by altering 1033 a small number of file header bytes. For the explanation, 1034 they use Feature Attribution to identify the most influential 1035 input features contributing to each decision and adapt it 1036 to provide meaningful explanations for classifying malware 1037 binaries. Other such works are [151], [152] employing SHAP 1038 and [153] proposing a new explanation algorithm to identify 1039 the root cause of evasive samples. It identifies the minimum 1040 number of features that must be modified to alter the decision 1041 of a malware detector, using Action Sequence Minimizer and 1042 Feature Interpreter.

1043
To conclude the section, it is necessary to analyze the 1044 work of Fan et al. [154]. They designed principled guide-1045 lines to assess the quality of five explanation approaches by 1046 designing three critical quantitative metrics to measure their 1047 Stability, Robustness, and Effectiveness. The five explanation 1048 approaches are SHAP, LIME, Anchors, LEMNA and LORE. 1049 Based on the generated explanation results, they conducted a 1050 sanity check of such explanation approaches in terms of the 1051 three metrics mentioned. Based on their analysis, the ranking 1052 of the five explaining approaches in terms of the Stability 1053 metric is LIME ≥ SHAP > Anchors > LORE > LEMNA. The 1054 ranking of the five explaining approaches in the Robustness 1055 metric is LIME > SHAP > Anchors > LORE > LEMNA. 1056 Spam is also called junk mail. It has existed almost as 1090 long as the internet as a means of selling products or services 1091 to a larger market of buyers than have ever expressed inter-1092 est in those products or services. After obtaining the email 1093 addresses of a considerable number of individuals, spammers 1094 bulk send their offers hundreds or thousands at a time. Spam 1095 can be very dangerous if it is part of a phishing attempt. According to the IC3 report, 8 Phishing (including vishing, 1098 SMiShing, and pharming) was the most common threat in the 1099 United States in 2020, with 241, 342 victims. Following that 1100 were nonpayment/non-delivery (108, 869 victims), extortion 1101 (76, 741 victims), personal data breach (45, 330 victims), and 1102 identity theft (43, 330 victims). These data show how huge 1103 this problem directly affects the population, which, if not well 1104 educated, can easily fall into the trap. The Figure 8 proves the 1105 dizzying amount of attention that Phishing attack detection 1106 is attracting from academics in recent years. Explaining to a 1107 user why a particular email is a phishing attempt or why it 1108 has been classified as Spam is no slight advantage. XAI in 1109 this field is directly connected to the population that could 1110 benefit from it to prevent a threat that is now constant.

Phishing. State of the art on the application of AI in Phishing 1114
Detection is substantial, so only recent works with the most 1115 significant impact in terms of citations have been analyzed. 1116 Hybrid Ensemble Feature Selection (HEFS) is an 1117 interesting approach proposed in [155] with a new feature 1118 selection framework. In the first phase of HEFS, a novel 1119 Cumulative Distribution Function gradient (CDF-g) algo-1120 rithm is exploited to produce primary feature subsets, which 1121 are then fed into a data perturbation ensemble to yield sec-1122 ondary feature subsets. The second phase derives a set of 1123 baseline features from the secondary feature subsets using 1124 a function perturbation ensemble. The best performance is 1125 achieved with Random Forest. The latter is one of the seven 1126 implemented and compared models for the real-time detec-1127 tion of phishing web pages by investigating the URL of the 1128 web page explored in [156]. In [157], Yerima et al. propose 1129 an approach based on a Convolutional Neural Network tested 1130 on a dataset obtained from 6, 157 genuine and 4, 898 phishing 1131 websites; a small dataset instead is used in [158] where the 1132 authors introduce a Deep Belief Network (DBN). Jain et al. 1133 propose a ML-based novel Anti-Phishing approach that 1134 extracts the features from the client-side only. They examined 1135 the various attributes of Phishing and legitimate websites 1136 in-depth. As a result, they identified nineteen outstanding 1137 features to distinguish Phishing websites from legitimate 1138 ones. DTOF-ANN (Decision Tree and Optimal Features 1139 based Artificial Neural Network) [159] is a Neural-Network 1140     The current state of the art for Phishing and Spam detection 1183 with explainable methodologies is relatively poor. Therefore, 1184 techniques that are not created on-demand for Phishing and 1185 Spam Detection but use datasets targeted at these application 1186 domains were also considered.

1235
These two methods avoid getting into spam and are not 1236 spam detector methods. However, they still use Explainable 1237 methods of AI to avoid spam; that is why they were analyzed 1238 in this section.

1239
The following analysis will focus on techniques that were  Spam are the main threats affecting anyone using a technolog-1256 ical device, so using AI for prevention and detection is nec-1257 essary. AI that simultaneously conveys assurance about the 1258 decision made and provides awareness is required to prevent 1259 the decision-making process from becoming less effective for 1260 the business and the individual user. As seen in the analysis 1261 conducted in [175], the user accepts AI makes mistakes, 1262 as long as it is explained how and why so that it can improve in 1263 the case of a false negative above all. A consideration beyond 1264 XAI in CyberSecurity is the education that must be provided 1265 to everyone with a technological device which happens to be 1266 surfing the internet where Phishing and Spam are continually 1267 around the corner. Similar to how one trains models, one 1268 might devise strategies to teach individuals to avoid falling 1269 victim to these scams. These strategies need to be Explainable 1270 so that anyone can comprehend why certain decisions are 1271 taken.

1273
A ''Bot'' or Robot, is a software program that performs 1274 automatic, repetitive, preset operations. Bots often mimic 1275 or replace the behaviour of human users. Since they are 1276 automated, they work considerably more quickly than actual 1277 individuals [182].

1278
Malware and Internet bots can be programmed/hacked 1279 to access users' accounts, search the Internet for contact 1280 information, transmit Spam, and execute other dangerous 1281 operations. Attackers may use malicious Bots in a Botnet, 1282 or network of Bots, to launch these attacks and conceal their 1283 source. A Botnet is a collection of online-connected devices 1284 running one or more Bots, frequently without the owners' 1285 knowledge. Since each device has a unique IP address, Botnet 1286 activity comprises many IP addresses, making it more chal-1287 lenging to locate and stop the source of malicious Bot traf-1288 fic. When used to infect additional computers, Spam e-mail 1289 recipients' devices can help Botnets grow larger. They are 1290 commanded by hackers known as Botmasters or Bot herders. 1291 Botnets are hard to spot since they consume very few 1292 computer resources. This keeps them from interfering with 1293 applications' regular operation and does not make the user 1294 suspicious. However, the most sophisticated Botnets can also 1295 alter their behaviour by the CyberSecurity systems of the PCs 1296 to evade detection. Most of the time, users are unaware that 1297 their devices are part of a Botnet and are under the control of 1298 online criminals [183].   In [189], the authors point out one of their proposal's main 1340 cons, the framework's non-Explainability. They emphasize 1341 that this is a problem with DL models and that this implies 1342 a lack of confidence. The following section will analyze 1343 frameworks that try to explain why a particular classification 1344 is made. Other interesting works are [190], [191]. BotStop [192] is a Packet-based Botnet detection system 1348 that examines incoming and outgoing network traffic in an 1349 IoT device to prevent infections from Botnets. The proposed 1350 system is founded on Explainable ML algorithms thanks to 1351 SHAP use with features extracted from network packets. 1352 Once an attack is detected, the source is blocked. Always 1353 SHAP is used in [193] to determine the relevant traffic fea-1354 tures in a framework to detect traffic generated by a Bot and 1355 then determine the type of Bots using a Convolutional Neural 1356 Network. 1357 Suryotrisongko et al. [194] propose the XAI and OSINT 1358 combination for Cyber Threat Intelligence Sharing in pre-1359 venting Botnet DGA. This research applied four existing XAI 1360 techniques: Anchors, SHAP, Counterfactual Explanation and 1361 LIME. This latter is also used in [195] and [196] where the 1362 final goal is the detection in IoT Networks. [197] is a Botnet Detection Model 1364 based on Graph Neural Network. The explanation is 1365 attributable to subgraph decomposition theory [198], where 1366 it is feasible to determine whether the learned model is inter-1367 pretable by identifying the subgraph with the most significant 1368 influence on prediction and judging whether the subgraph is 1369 faithful to general knowledge.

1370
Reference [199], [200], [201], three explainable studies 1371 focused on DGA-based botnet detection, are also worth 1372 mentioning, as is [202]  As noted in the previous sections, almost all of the frame-1387 works declared Explainable use existing methods for Post-1388 hoc Explanation, SHAP and LIME above all. In BotNet 1389 Detection, the almost total focus on IoT networks and devices 1390 should be especially noted, demonstrating that these occupy 1391 a very important slice of the Net. As in the case of Spam and 1392 Phishing, it is critical to alert if you have entered a BotNet 1393 and are feeding it unknowingly, and even more important 1394 to Explain what you have inferred and how you got into 1395 it, so that you can avoid falling into it again in the future. 1396 It is moving in this direction, as evidenced by the increasing 1397 number of publications on the subject, however, one must 1398 consider that also improving is the malicious part of the 1399 fight. That is why it is increasingly important that supporting human decisions is AI, which can counter a considerable part 1401 of these attacks in an automated way. For there to be the 1402 right cooperation between human and AI, Explainability of 1403 the latter is necessary to build trust in the former.   gators, and is therefore unlikely to be justified in court.

1454
Furthermore, the admissibility of digital/network analysis 1455 performed by XAI in court is still debatable as it would 1456 necessitate a review of applicable laws (e.g., evidence law). 1457 However, XAI can be used efficiently and legally in the 1458 future to support the digital/network forensic profession if it 1459 is not viewed as a replacement for a digital/network forensic 1460 examiner but rather as a reliable tool to aid in investigations. 1461 ATLE2FC [216] is a model for IoT Forensics using Ensem-1462 ble Classification with an Explainable layer consisting of 1463 FPGrowth with GRU-based RNN classifier for rule estima-1464 tion and severity classification.  Cyber Physical Systems. When an adversary gains access 1470 to a computer system that controls equipment in a manufac-1471 turing facility, oil pipeline, refinery, electric generating plant, 1472 or other similar infrastructure, they can control the operations 1473 of that equipment to harm those assets or other property. This 1474 is known as a Cyber-Physical attack on critical infrastructure. 1475 Cyber-Physical attacks pose a risk not only to the owners and 1476 operators of those assets but also to their suppliers, clients, 1477 enterprises, and people nearby the targeted asset, as well as 1478 to any individual or entity they could negatively impact. For 1479 example, a Cyber-Physical attacker may take down cameras, 1480 switch off the lights in a building, cause a car to wander off 1481 the road, or make a drone land in the hands of adversaries. 1482 Wickramasinghe et al.
[219] propose a Desiderata on 1483 Explainability of unsupervised approaches in Cyber-Physical 1484 Systems since they generate a large amount of unlabeled 1485 data. These are potential solutions for meaningfully mining 1486 these data, maintaining and improving desired functions, and 1487 improving the safety of these systems.  Crypto-Jacking. Crypto-jacking, a new Malware that 1493 resides on a computer or mobile device and uses its resources 1494 to ''mine'' Cryptocurrencies, is a severe online threat. In addi-1495 tion to compromising various devices, including PCs, laptops, 1496 cellphones, and even network servers, Crypto-Jacking can 1497 take control of web browsers. Using Crypto-Jacking, crim-1498 inals compete with sophisticated Crypto mining operations 1499 without the high overhead costs by stealing computational 1500 power from victims' devices.

1501
It is a threat comparable to BotNets, where unknowingly 1502 the user feeds activities with malicious purposes through their 1503 device.

1504
There are no works that make Explainable Artificial 1505 Intelligence methods in the detection of Cryptojacking, 1506 one that goes in this direction in the detection of Cryp-1507 tomining is that of Karn et al. [222]. They designed and 1508 implementated an automated cryptomining pod (manage-1509 ment of applications inside containers) detection in a 1510 VOLUME 10, 2022 Kubernetes cluster. Explainability is provided using SHAP, 1511 LIME, and a novel auto-encoding-based scheme for LSTM 1512 models.

1514
Due to the broad spectrum of XAI approaches, analyzing 1515 the different surveys involving these works were preferred to 1516 better orient the reader. It is also unthinkable to include all 1517 studied papers; hence only a selection of works was discussed 1518 in this survey for synthesis and relevancy considerations, 1519 prioritizing all work that proposed XAI methods with appli-1520 cation in CyberSecurity.  ML/DL model, the type of explanation and a summary concerning the taxonomy presented in section II-A.     users are likely to ask. Thus, explainability can only occur 1561 through human-machine interaction. In [242], the authors 1562 present an example and approach for creating a concept for 1563 an XAI-driven junior cyber analyst based on understanding 1564 the information needs of both humans and AI components in 1565 terms of the work context and workflow. This method may 1566 be required to design future systems that people can use, par-1567 ticularly for critical systems where human stakeholders can-1568 not interact with black-box outputs from intelligent agents, 1569 as is the case in many CyberSecurity applications. Therefore, 1570 the idea and proposal are to think about and build frame-1571 works that have human-machine interaction at their core for 1572 CyberSecurity applications, which is vital in many cases. The 1573 only way to get there is to build models understandable to 1574 humans.

1575
How to achieve Explainability. In the current state of 1576 the art, as shown in the Table 2, the proposed methods use 1577 post-hoc explanation in most cases. Developing models that 1578 provide an intrinsic explanation is a priority; an explana-1579 tion method developed ad-hoc for that particular type of 1580 application is necessary for a field such as CyberSecurity, 1581 where one risks providing an assist to the attacker. Moreover, 1582 the problem may be precisely in terms of explanation, and 1583 the risk is to provide an untruthful output. As pointed out 1584 several times in [101], LIME, one of the most widely used 1585 methods, assumes that the decision boundary is locally linear. 1586 However, when the local decision boundary is non-linear, 1587 as it is in the majority of complex networks, those expla-1588 nation approaches cause significant inaccuracies. In some 1589 cases, the linear portion is severely constrained to a relatively 1590 tiny region. The artificial data points beyond the linear zone 1591 are easily struck by standard sampling methods, making it 1592 hard for a linear model to estimate the decision boundary 1593 near x. The challenge then is not easy, the inverse correlation 1594 between model opacity and performance is well known, but 1595 an effort is needed to develop increasingly high-performing 1596 but transparent models. tions, methods, and applications in interpretable machine learning,'' Proc.