Comparison and Investigation of AI-Based Approaches for Cyberattack Detection in Cyber-Physical Systems

The demand for cyber-physical systems (CPSs) has recently increased in various domains, such as smart grids, intelligent transportation, and critical infrastructure. The massive data networks and communication layers generated make CPSs vulnerable to threats and cyberattacks. To mitigate these threats, artificial intelligence (AI) approaches are employed. However, AI models struggle to keep up with the constantly changing attack landscape. This study investigates the application of extreme gradient boosting (XGBoost) and long-short-term memory (LSTM) AI models for cyberattack detection in a CPS. Accuracy, precision, recall, and the F1-score validate the approach as evaluation metrics. The methods were tested on a gas pipeline industrial control system dataset and other benchmark datasets, such as NetML-2020 and IoT-23, which contain various cyberattacks. The performance of the two methods was found to be better than other models such as support vector machine (SVM) and artificial neural networks (ANN) on several evaluation metrics. Finally, we present recommendations for future research.


I. INTRODUCTION
Cyber-physical systems (CPSs) were introduced in 2006 by Helen Gill at the National Science Foundation (NSF) workshop in the United States (US) [1].CPSs combine the integration of computational physical systems, including storage, sensors, and actuators for mission-critical tasks, to increase the efficiency of communication technologies.As an emerging defect in CPSs, data protection and data authentication are vulnerable to cyberattack threats.These attacks typically occur because CPSs are connected through wireless connections and the internet to transmit their data, making it The associate editor coordinating the review of this manuscript and approving it for publication was Genoveffa Tortora .easy for them to attack during regular network communication [2].For example, in recent decades, there have been numerous threats to significant cyberattack issues within the CPS environment [3].Data privacy concerns in network management and as sources for facility analysis of CPS security monitoring [4].
Figure 1 illustrates the holistic cyber-physical framework, where CPS applications are executed, including sensors and actuator networks.Additionally, CPS's framework encompasses three fundamental components: physical processes, interfaces, and cyber systems.
The term ''physical processes'' pertains to the observable and measurable natural phenomena that are subject to monitoring or regulation, while ''cyber systems'' pertains to a class FIGURE 1. Holistic framework for cyber-physical systems [6].
of embedded devices that can process information.The physical world is connected to cyber systems through intermediate components, including sensors, actuators, and communication networks.Sensors and actuators convert energy into electricity and vice versa [5].
Due to the rapid growth of CPSs in many areas, such as smart buildings, smart grids, intelligent transportation, and critical infrastructure, huge amounts of data are being generated.This makes the system vulnerable to cyberattacks.With the increased demand for technology leading to the Fourth Industrial Revolution (IR 4.0), CPS sensors are often used for performing real-time analysis, monitoring, and forecasting system malfunctions.These data have an impact on the entire manufacturing system if they are contaminated or compromised because of cyberattacks, giving false predictions and insights and ultimately leading to catastrophic failures.The physical layer of CPSs is vulnerable to attacks involving the injection of false data into sensors and actuators, which can compromise the integrity of complex network systems [7].Several cyberattacks have occurred in CPSs.For example, the Ukrainian power plant in 2016 and the Stuxnet worm, which targeted nuclear power plants, have been attributed to such methods [8].
Machine learning (ML) and deep learning (DL) techniques are subsections of artificial intelligence (AI) that are currently used for the detection of cyberattacks, such as threat detection, malware clarification, and intrusion detection.Extensive research has been conducted on the use of ML learning algorithms to enhance cyberattack issues in the CPS environment [9], [10].The ML method can also be applied to detect and identify anomalies [11].For example, XGBoost classifiers are used for intrusion detection in input datasets that contain normal and anomalous instances [12].Gad et al. [13] proposed an XGBoost technique to detect and reduce malicious activity in IoT.Furthermore, a DL model based on Long short-term memory (LSTM) memories (LSTM) was used to detect cyberattacks in CPSs [14].
Moreover, AI and ML have significantly transformed the field of cybersecurity, especially CPS.They offer exceptional capabilities in detecting and minimizing cyberattacks that disrupt the interconnected infrastructure of vital systems, such as power grids, transportation networks, and industrial control systems [15].However, traditional signature-based detection techniques frequently find it difficult to keep up with cybercriminals' increasingly complex techniques using advanced technology.In contrast, AI and ML algorithms can gain experience and continue to improve at identifying new threats.The ability to adapt is crucial for protecting CPS because it remains constantly changing and vulnerable to emerging threats.AI and ML algorithms can process and interpret these data with incredible speed and accuracy, allowing them to detect anomalies and suspicious patterns that may indicate a cyberattack.Furthermore, these intelligent models provide excellent performance in terms of their ability to analyze, detect, and adapt to new threats and evolving attack methods in real-time.
This study presents an approach that demonstrates greater efficiency in cyberattack detection for CPSs, and addresses cybersecurity concerns.The integration of LSTM and XGBoost has improved the detection of cyberattacks in gas pipeline systems.The temporal feature extraction capabilities of LSTM, combined with the robustness of XGBoost, improve the detection and classification of various cyberattacks.The model's effectiveness has been extended beyond the gas pipeline by including other domains, such as IoT datasets containing various cyberattacks.In addition, this study investigates the risks and threads associated with CPSs, as well as how to overcome them using potential AI approaches.This study's specific contributions are as follows: a) We employed the XGBoost and LSTM models to detect sophisticated cyberattacks in CPS by examining temporal and context relations in the data.b) The two models were tested on a gas pipeline system based on industrial control system (ICS) datasets and other available benchmark datasets, such as NetML-2020 and IoT-23, which contain various cyberattacks.

A. RISKS AND THREATS
The emergence of CPSs presents new challenges against cyberattack risks and threats.Ensuring data protection against security risks and cyberattacks is one of the most complex issues within the CPS environment [16], [17].Such cyberattacks include denial of service (DoS), Trojans, worms, and buffer overflow.When these attacks succeed, they affect the CPS through breaches of confidentiality, privacy, integrity, availability, and safety, which can lead to failure.However, if the attacker had evaluated the encryption key, he could have illegally obtained access to the monitoring center and destroyed normal system operations.Moreover, CPSs comprise both physical and cyber components through a range of integrated components.The ML and quantitative base risk assessment approaches play vital roles in the analysis and identification of threats to the CPS environment.These security-risk cybersecurity threads can compromise security and privacy.An attacker's malicious activities can spread and could lead to failure, power failure, and security threats when using these devices [18].As the number of devices increases, major problems continue to develop in real-world scenarios [19].These problems include connectivity, security, trust, interoperability, scale, and the environment.
The remaining section of this paper is organized as follows.Section II provides a literature review based on related work on cyberattack detection techniques in CPSs using ML approaches.Section III presents the study methodology, which includes the data collection procedure, the proposed comparison method, and evaluation criteria.Section IV presents the implementation and result analysis, including the importance of their characteristics and comparison performance analysis.Section V provides an AI-based detection roadmap.Section VI provides a potential countermeasure.Finally, we present our conclusions and future directions in Section VII.

II. LITERATURE REVIEW
In this section, related studies are discussed.Various approaches have been proposed to solve cyberattacks using the ML method.For example, Almiani et al. [20] presented a fog system security and a fully automated intrusion detection system for cyberattacks by proposing a model using multilayer neural network designs that are very close to end users.To better understand the problem, the model was evaluated using typical varieties, Mathew's correlation, and Cohen's kappa coefficient.Mall et al. [21] demonstrated various ML models that can be used to identify distributed denial-of-service (DDoS) attacks in a software-defined CPS framework.This was achieved through the implementation of a flexible and scalable software-defined network (SDN) design.Bitirgen and Filik [22] proposed a new approach to improve the functionality of convolutional neural networks for long and short-term memory (CNN-LSTM) to detect fault detection, isolation, and accommodation (FDIA) in smart grid (SG) systems.Thapa et al. [23] conducted a comparative analysis of various ML and DL models using Coburg intrusion detection datasets (CIDDSs).
In a major advance in 2022, [24] conducted a comprehensive survey on the use of DL for detecting cyber-physical system attacks, which represents a significant advancement in cybersecurity.The authors employed a modified methodology that encompasses CPS scenario analysis, identification of cyber-attacks, formulation of ML problems, customization of DL models, acquisition of training data, and performance evaluation.The reviewed studies demonstrate significant promise in identifying cyber-attacks on CPSs using DL modules.In [25], the authors developed an innovative method called PRO-DLBIDCPS, which is a poor and rich optimization with DL for blockchain-enabled intrusion detection in a CPS environment.The PRO-DLBIDCPS technique introduces an adaptive harmony search algorithm (AHSA) for selecting feature subsets.The CPS-GUARD system was developed using an innovative intrusion detection method that relies on a single semi-supervised autoencoder.In addition, a technique has been implemented to establish a threshold that distinguishes normal operations from attacks.The technique is designed to be aware of outliers, i.e., it uses outlier detection to address the inherent imperfections present in the training data [26].Several authors have investigated the impact of the DL model on cyberattack detection in CPS.For instance, [27] conducted a comparative analysis of various state-of-the-art deep learning techniques for the classification and categorization of malicious applications.The proposed method involves using an ensemble dynamic weighted voting model to accurately detect and categorize a diverse range of malicious applications using the CCCS-CIC and Mal-2020 datasets.A DL approach for identifying and analyzing time delay attacks (TDA) has been introduced.This approach involves the development of a hierarchical long short-term memory model.The model is designed to handle real-time data streams from relevant CPS sensors with an understanding of any embedded signals that may indicate an attack [28].
Moreover, the authors of [29], [30], and [31] explained the potential of ML techniques to detect various attacks on CPS, including smart grids, power grids, and cyber-physical power systems.Lin et al. [29] used deep reinforcement learning (DRL), propose a model for false data injection attacks and counter-detection techniques.Jahangir et al. [30] proposed a novel approach for the identification and localization of high-resolution.This method uses a multi-output network that includes a two-dimensional neural network classifier and a reconstruction decoder.Presekal et al. [31] introduced a novel technique for identifying anomalies in time-series data using classification.This approach uses a hybrid DL model that integrates graph convolutional long and short-term memory (GC-LSTM) with a deep convolutional network.Almuqren et al. [32] developed a technique known as the Explainable Artificial Intelligence Enabled Intrusion Detection Technique for Secure Cyber-Physical Systems (XAIID-SCPS).Furthermore, the XAIID-SCPS technique incorporates the XAI methodology known as local interpretable model-agnostic explanation (LIME) to enhance the comprehension and interpretability of the black-box algorithm, thereby facilitating accurate intrusion classification.
More recent evidence by Tertytchny et al. [33] demonstrates that CPS classifies network abnormalities as faults and attaches them to the IoT using ML.The authors established a formal definition of the issues arising from component failures and network attacks, considering the impact of communication behavior.They demonstrated the correlation between these two abnormal sources and presented a framework based on ML.A concept paper on the adaptation of ML and blockchain techniques in CPS to address security issues related to cyberattacks was presented [34].Sowmya and Mary Anita [35] presented a comprehensive taxonomy of the extant literature on ML, DL, and ensemble learning.The analysis includes 72 research papers and considers detection-related factors such as the algorithm and performance metrics.Finally, a comprehensive review was conducted, which involved categorization, classification, and examination of the existing literature on artificial intelligence (AI) techniques used to identify cyberattacks in the Internet of Things (IoT) settings [36].
This study compares and investigates existing DL and ML algorithms for cyberattack detection in CPSs.Based on our knowledge, this study is different from other studies because we focused on critical industrial control systems, i.e., gas pipeline cyberattack detection using LSTM and XGBoost models.However, our work added value by understanding the various AI models from empirical studies to overcome current trends in cybersecurity attacks in CPSs and IoT environments.We also analyzed and validated the models using the available benchmark datasets that containing cyberattacks.Table 1 summarizes the findings based on the related studies.
In addition, our comparable contribution attempts to address key limitations of existing approaches.For example, [20] evaluated there DRNN model for intrusion detection systems using a single dataset.The authors in [23] use both the KDD99 and NSL-KDD datasets, which appear to have network biases.In addition, [21] limits the capabilities of their proposed DL model to a single DDoS attack in the SDNbased domain.To overcome these limitations, our study uses a variety of datasets from various domains, including gas pipelines, NetML-2020, and IoT-23, which contain a variety of cyberattack scenarios.We also investigate the capability of combining LSTM and XGBoost to detect cyberattack scenarios in industrial control systems.

III. MATERIALS AND METHODS
This section provides details of the study methodology, implementation, and design of the proposed methods for intrusion detection systems in CPS.The proposed framework combines several independent processes and comprises data collection and observation.During this process, datasets were collected and observed in detail based on the type of data.The entire dataset was processed, consisting of cleaning the data, visualizing the data using vectorization steps, and feature engineering.The training of the dataset used ML.An optimization method was used to create the final model.The study will use the XGBoost classification, which is based on the decision tree algorithm (DT), and the LSTM, which is based on the recurrent neural network (RNN) and uses the conventional gradient descent technique.
Figure 2 presents the research methodology flowchart.First, data collection from a real-world gas pipeline system contains various cyberattacks.Followed by data preprocessing, which involves data cleaning and normalization.Model creation consists of the LSTM and XGBoost algorithms.The sampling data were split into training and testing, followed by the learning algorithms.The model evaluation would be based on the ACC.After evaluation, the model would predict if there were cyberattacks or if it was in normal status.Cyberattacks are predicted based on anomalous activities in the input data.Finally, anomalous activity can be classified as active or passive.

A. DATASETS EXPLANATION
The datasets were obtained from a gas pipeline system based on an industrial control system (ICS) at Mississippi State University.The dataset comprised various components, including sensors equipped with actuators from a gas pipeline.The dataset contained seven different categories of cyberattacks [37].There are two actuator components for gas pipelines in conjunction with a pressure sensor, which are components of the SCADA system.Actuators, comprising solenoids and pumps, are used to regulate the physical processes of the system, thereby ensuring that the pressure set by the SCADA is maintained.The modes of the gas pipeline system were classified into three distinct groups: manual, automatic, and off.The components of a communication network refer to the protocols used in a serial Modbus remote terminal unit (RTU).Each packet transmitted through this  system comprises a header and a payload.The components responsible for supervisory control encompass the master terminal unit (MTU) and the human-machine interface (HMI).The MTU is configured in various setups where each subordinate device functions as a RTU that receives directives from the MTU, and subsequently, the RTUs react to the MTU's commands.
In addition, the MTU was linked to the HMI to provide human operators with a means to oversee the system and the supervisory controls.However, the fault has been simulated because of the huge network traffic and imbalanced data in the SCADA system, where system commands and responses are being manipulated.Figure 3 shows the gas pipeline system and HMI.
Figure 4 shows the comprised input datasets, which consist of the five most important features that describe the possibility of cyberattacks.Hence, the XGBoost model classifies all numeric input features as simple binary classification problems.The LSTM model learns from a function consisting of a sequence of past observations as input (x) to an output observation (y).Furthermore, one feature output indicates whether a specific attack has occurred after training and testing.
Table 2 shows that datasets consist of seven separate types of attacks, comprising both normal and attack samples, which have been identified as follows: The attack values range from 0 to 7, which is accomplished by establishing a parameterization.This range was created to provide updates on all attack possibilities that can be executed using a specific parameter.The dataset is a comma-separated value (CSV) text file consisting of 19 features of network field states provided by one packet delivered by the MTU or RTU as shown in Table 3.Each dataset for MTUs or RTUs includes information on network traffic and payload.The payload contains crucial data related to the state, parameters, and settings of the gas pipeline.This information is essential for comprehending the system's behavior and identifying any deviations from normal operation.

B. ALGORITHM THEORETICAL CONSIDERATIONS 1) LONG SHORT-TERM MEMORY (LSTM)
LSTM is based on a recurrent neural network (RNN) and aims to capture sequence-dependent behavior or model time in a range of applications, such as IDS for detecting intrusions in network traffic.The process involved in this study involved providing the output of the neural network layer at a specific time point T as input to the subsequent layer at time T + 1.The LSTM model is an extension of the RNN architecture.It incorporates memory components that facilitate the transmission of acquired knowledge from a given time step T to subsequent time steps, including T + 1, T + 2, and T + 3.Moreover, an important attribute of the LSTM model is its ability to selectively discard irrelevant components of the prior state while simultaneously selecting the updated state and producing pertinent components of the state that are pertinent to future predictions.The LSTM cell shown in Figure 5 indicates the use of input features xt, which correspond to input data x at a given time t.The input gate is responsible for regulating the flow of input data into the cell.
In addition, the LSTM cell consists of three primary components: the input gate, forget gate, and output gate.They are responsible for regulating the flow of information within the cell.
a) Input gate: The input gate determines which parts of the current input (X) will be incorporated into the cell state (Ct).It also serves as a filter, that identifies valuable elements of the new memory vector.b) Forget gate: The forget gate regulates the extent to which the previous cell state (Ct-1) is forgotten.Also, determine the relevant components of the cell state by considering the previous hidden state and the new input data.c) Output gate: The output gate determines the amount of the LSTM cell's state (Ct) that is output.It also determines the LSTM network's final hidden state.The i t define the input/output of gate activation, where f t determine forget gate activation, while o t finds control flow to output gate activation.The c t determines cell state and h t−1 define the hidden state, while σ sigmoid acts as an activation function.The components of the LSTM equation cell functions are indicated below.
where i t is the function that determines which information from the current input should be stored in the cell state, h t−1 represent the previous hidden state, x t denotes the current input, and Wi, and bi represent the weight and bias for the input gate, respectively.
where f t decides what information in the cell state should be forgotten or retained, h t−1 represent the previous hidden state, x t denotes the current input, and Wf, bf represent weight and bias for the forget gate, respectively.
Here, after gt the Ct updates the cell state by combining the candidate cell state update and the previous cell state Ct-1, using the input it and forget gates f t, respectively.
ht function produces the new hidden state based on the updated cell state and the output gate, ht represents gate output, and ctdenotes the cell state.
The LSTM architecture has the capability to detect cyber-attacks by learning to identify patterns in network traffic that are indicative of attacks.To control how the LSTM network learns these patterns, use the input gate, forget gate, and output gate.For example, the input gate can be used to direct the network's attention to specific aspects of network traffic, such as the IP addresses of the source and destination hosts, packet size and frequency, and protocol type.The forget gate can be used to prevent the network from forgetting previously learned important patterns.The output gate can be used to regulate how much of the network's output is used to forecast the possibility of a cyber-attack.

2) EXTREME GRADIENT BOOSTING (XGBOOST)
XGBoost is rooted in the concept of gradient boosting as introduced in Friedman's ''A Gradient Boosting Machine'' for function approximation [26].XGBoost is a supervised learning algorithm used to solve problems by treating data with multiple features xi to predict the value of the target variable yi.The model's objective functions are training loss and regulation with yi by a various of tasks such as ranking, classification, and regression.The training task is to find the parameter θ that best fits the training of data xi and labels yi.The silent characteristics of the objective function consist of two parts: training loss and a regularization term.
where L is the training loss function and is the regularization term.Training loss measures how the predictive model respects the training data, where a common choice of L is the squared error.
The chosen XGBoost model is based on DT ensembles and consists of a classification and regression tree (CART).Treeboosting training is based on supervised learning models that satisfy the objectives.
The complexity of the model is important in the regularization term, where the complexity of the tree, (f ), is also defined as f (x).
where W is the vector of scores for the leaves, q is the function assigned to each data point on the corresponding leaf, and T is the level number.The XGBoost complexity is defined as Figure 6 shows the structure of the XGBoost model, which involves the iterative process of fitting decision trees to the data and updating the model parameters using the loss function gradient.This process is repeated until convergence, providing a model that is both highly accurate and scalable.Moreover, XGBoost has demonstrated its efficacy as a machine learning algorithm in diverse cyber security domains, encompassing intrusion detection, malware detection, and phishing detection.

C. EVALUATION CRITERIA
The metric used to evaluate the model's performance is ACC.The evaluation of classification models involves the use of the Model ACC metric, which assesses a limited portion

1) CONFUSION MATRIX
A confusion matrix is a table that is used to summarizes the performance of a classification model.It shows the number of true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN) that the model produced by the model.
a) The TP are the instances that were correctly classified as normal.b) The FP are the instances that were incorrectly classified as attacks.c) The TN are the instances that were correctly classified as attacks.d) The FN are the instances that were incorrectly classified as normal.

2) ACCURACY
The accuracy of a model represents only a portion of its overall performance.The accuracy metric is commonly employed in the evaluation of classification models.It is calculated as follows: 3) PRECISION Precision is the positive predictive value.The metric quantifies the ratio of correctly identified positive instances by the model to the total number of positive instances identified by the model.In addition, precision is the fraction of instances that were classified as attacks.It is calculated as follows:

4) RECALL
The recall metric, also referred to as the actual positive rate, quantifies the proportion of positive instances correctly identified by the model concerning the total number of positive instances present in the dataset.Additionally, recall is the fraction of instances that are attacked that were classified as The F1 score can also evaluate the performance of a model as well.The metric in question is a calculated value that combines the precision and recall of a given model, considering their respective weights.In addition, the F1 score is a measure of the accuracy of the classification model.It is calculated as the harmonic mean of precision and recall.

6) RECEIVER OPERATING CHARACTERISTIC CURVE
The receiver operating characteristic (ROC) curve is a widely used graph that summarizes a classifier's performance across all possible thresholds.In addition, the ROC curve is a plot of the true positive rate (TPR) against the false positive rate (FPR).

IV. IMPLEMENTATION AND RESULT ANALYSIS A. EXPERIMENTAL SETUP
The experiment was conducted using Desktop-G7BDT90, with the operating system edition of Windows 10 Home 64bit (22H2, Build 19045).The processor was Intel(R) Core (TM) i5-6400 CPU @ 2.70GHz, 2.70GHz.The memory for the desktop was 16.0 GB RM.In addition, for data analogy, we used Python (version 3.8.11)for the artificial neural network and the machine learning Keras library, along with its functionality on the back end, TensorFlow, to perform low-level operations using Keras.For data analysis, the Scikit-learn library was used; for data visualization the Matplotlib library and Seaborn library; and for data cleaning and feature engineering, the Pandas, and Numpy libraries were used.

B. RESULT ANALYSIS
This subsection provides details of the implementation and validation of the proposed methods mentioned in Section III.A feature selection method was used to improve the accuracy score.In addition, a comparative performance analysis was conducted, in which we trained and tested our employed models with other available benchmark datasets that contain various types of cyberattacks.Detailed statistics of datasets that only consist of binary classification tasks.Additionally, 274,628 are samples, while 60,048 are attack-related (See Table 6).Figure 7 presents a comparative analysis of the LSTM and XGBoost models in terms of their accuracy in classifying cyberattacks.Following the completion of training and testing, the XGBoost model achieved a higher level of efficiency of 98%.This result represents a 1% improvement over the performance of the LSTM model.
The employed LSTM model for detecting cyber security attacks in cyber-physical systems (CPSs) achieved a training accuracy of 98.80% and a testing accuracy of 97.80%, as shown in Figure 8.This indicates that the model learned the patterns in the training data and generalized well to new data.The training loss was 0.4911, whereas the testing loss was 0.4796.This indicates that the model could effectively fit the training data and minimize the testing data prediction error.However, the LSTM model's high accuracy and low loss indicate that it is a promising approach for detecting cybersecurity attacks in CPSs.This is because the model was able to learn the patterns in the training data and generalize effectively to new data.Similar performance of the model on testing and training data that the model does not overfit the training data.In addition, the model accuracy and loss results are comparable to or better than those reported by other studies on the detection of cyber security attacks in CPSs using LSTM models.The performance of the model on testing data is comparable to its performance on training data, indicating that the model does not overfit the training data.Overall, the model accuracy and loss results indicate that the employed LSTM model is a promising technique for detecting cyber security attacks in CPSs.
Table 7 presents a comprehensive analysis of the outcomes obtained from the LSTM and XGBoost models for detecting CPS cyber security attacks.The LSTM model attained a classification accuracy of 97%, precision of 86%, recall of 97%, and F1-score of 91% when evaluated on the ICS Gas Pipeline dataset.This finding indicates that the LSTM model demonstrates efficacy in detecting cyber security attacks in CPSs; however, there is a possibility of generating false positive results.The XGBoost model demonstrated notable performance on the ICS gas pipeline dataset, achieving an accuracy of 98%, precision of 99%, recall of 98%, and F1-score of 98%.This finding proved that the XGBoost model demonstrated high efficacy in identifying cyber security attacks in CPSs while exhibiting minimal occurrence of false positives.
In terms of comparative analysis, it is evident that both the LSTM and XGBoost models have demonstrated a commendable level of accuracy when employed for detecting cybersecurity attacks in CPSs.In comparison, the XGBoost model exhibited marginally superior accuracy and precision compared with the LSTM model.This implies that the XGBoost model could offers advantages in the context of cyber security attack detection in CPSs, where the consequences of false positives are financially burdensome.
The findings obtained from the LSTM and XGBoost models demonstrate their capability to acquire knowledge regarding the characteristics of cyber security attacks in the ICS gas pipeline dataset.This phenomenon can be attributed to the capacity of both models to acquire intricate associations among sensor data.Furthermore, the findings derived from the LSTM and XGBoost models demonstrate the efficacy of both models in identifying cyber security attacks within CPSs.In comparison, the XGBoost model exhibited marginally superior accuracy and precision compared with the LSTM model.This implies that the XGBoost model could be a more favorable option for detecting cyber security attacks in CPSs when the consequences of false positives are significant.In general, the outcomes derived from the LSTM and XGBoost models exhibit promise and indicate the potential use of these models in the creation of efficient cyber security attack detection systems for CPSs.
The confusion matrix for the XGBoost model, as shown in Figure 9, provides valuable insights into its performance in classifying various types of cyber-attacks.The matrix shows that the model is highly accurate correctly identifying ''Normal'' and ''Recon'' attacks, with few misclassifications.However, some difficulties in distinguishing between similar attack types, such as ''NMRI,'' ''CMRI,'' and ''MPCI,'' have been observed, resulting in a few misclassifications.
31996 VOLUME 12, 2024 Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
Overall, the XGBoost model performs well in cyber security attack detection, especially in detecting common and severe attack types.However, the confusion matrix of the LSTM model highlights its exceptional performance across all attack types.The model detects normal and recon attacks with near-perfect accuracy, with almost no misclassifications in these categories.Notably, the LSTM model excels at distinguishing between attack types that are closely related, such as ''NMRI,'' ''CMRI,'' and ''MPCI,'' resulting in few misclassifications.These findings highlight the robustness and effectiveness of the LSTM model in detecting cybersecurity attacks, making it a promising choice for protecting CPSs against various threats.
Furthermore, the employed model's results were directly compared with the other models, which are SVM and ANN.In terms of overall performance as measured by the F1-Score, the employed models, XGBoost and LSTM, outperformed the other models, SVM and ANN.The employed models achieved F1-Score of 0.94 on average, whereas the other models received F1-Score of 0.86 on average.However, the models also performed well in terms of accuracy, precision, and recall.The highest accuracy (0.98) was achieved by XGBoost, while the highest precision (0.99) and recall (0.97) were achieved by LSTM.The comparison indicates that the employed models, XGBoost and LSTM, outperform SVM and ANN for cyber-attack detection in cyber-physical systems.

C. FEATURE IMPORTANCE
The important features consisted of a bar graph visualization of the top ten important features sorted according to the highest score among all features to improve the model accuracy score.This achievement was achieved by calculating the frequency of the time division of the features in the boosting trees integrated within the model.A feature with a high-value score only contributes when predicting an attack.In addition, a technique for determining the importance of the characteristics was applied to assess the significance of each characteristic in the datasets.The feature importance technique was employed after every training session to modify the attributes of the datasets.The top ten features of the study are presented in Figure 10.

D. COMPARISON PERFORMANCE ANALYSIS
This section provides a comparative performance analysis of our proposed approach.ROC curves are used to evaluate our models' discriminatory capability and efficacy in differentiating between instances of attack and non-attack.We also further investigate the efficiency of our models using other available real-world benchmark datasets that contain different types of cyberattacks.
As shown in Figure 11, the XGBoost and LSTM model's exhibit a robust performance in identifying between attack and non-attack instances, as evidenced by the AUC-ROC value of 0.86.The findings indicate that the XGBoost model is effective in detecting cyber security attacks.After comparing the AUC-ROC values the LSTM and XGBoost models, it becomes evident that the LSTM model exhibits superior performance in terms of its overall discriminative capability when compared with XGBoost.The LSTM model exhibits a higher AUC-ROC, which indicates its superior capability in accurately classifying attacks while minimizing the occurrence of false positives.
In addition, the receiver operating characteristic (ROC) curves provide a visual representation of the trade-offs in performance.The LSTM model consistently exhibits a superior true positive rate compared with XGBoost across different thresholds of false positive rates.
Figure 12 presents our model's comparative performance across various datasets containing different types of intrusions and cyberattacks.In addition, Table 8 provides details about the comparative performance and includes additional evaluation metrics.We also investigated the capabilities of AI techniques for cyberattack detection across real-world benchmark datasets.However, our proposed method has been evaluated using benchmark datasets adopted from [38], [39], [40], and [41].Moreover, these targeted datasets enable a focused evaluation of our method's efficacy against diverse cyberattacks in these critical systems.
Table 9 provides a comparison of our study with other state-of-the-art studies.We also analyzed the compared studies based on the ML classifiers used, ACC score, predictive features, strengths, and limitations.The analysis findings indicate that ACC is competitive with other techniques for attack detection.However, they can be enhanced.Finally, these studies provide better performance toward various types of cyberattack detection using LSTM and XGBoost classifiers.

V. AI-BASED DETECTION ROADMAP AND THREAT MODEL ANALYSIS
In this section, we investigate the AI capabilities for detecting cybersecurity attacks in CPSs and understand their threat to analysis.The attacks were adapted from the several cyber-attacks included in our datasets, which are naïve malicious response injection (NMRI), complex malicious response injection (CMRI), malicious function code injection (MFCI), denial of service (DoS), and reconnaissance (Recon).Figure 13 shows the AI approaches for detecting attacks using a roadmap.Naïve malicious response injection (NMRI) attacks are measured by a lack of knowledge regarding the physical system and its control logic.The effects of NMRI attacks are effective because of the attacker's ability to inject or modify response packets in the network.AI-based methods can be used to identify this attack.For example, Wang et al. [42] presented an approach for using a DNN with explanatory attributes for the purpose of intrusion detection in industrial control networks.In addition, support vector machines (SVM) and random forest (RF) have been employed as effective methods for ensuring the reliable detection of network attacks in SCADA systems [43].
Furthermore, NMRI attacks are a type of network-based threat that targets CPSs.These attacks exploit communication protocol vulnerabilities by injecting crafted responses into the network and manipulating the system's perception of physical process control.NMRI attacks can pose significant threats to 31998 VOLUME 12, 2024 Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.ICS and SCADA systems.These attacks can disrupt control loops by injecting false or misleading information, resulting in equipment malfunctions, production outages, or even safety hazards.A comprehensive cybersecurity strategy is required to effectively mitigate NMRI attacks.This strategy should include network segmentation, intrusion detection and prevention systems (IDS/IPS), and vulnerability assessments.
Complex malicious response injection (CMRI) attacks encompass a category of response injection attacks that exploit vulnerabilities in industrial control systems.The effect of CMRI attacks hides the actual state of the physical process.The attack can be detected using AI-based techniques.Shitharth et al. [44] developed sophisticated machine-learning models to enhance the security of SCADA systems.These models are based on the Block Correlated Neural Network (BCNN) used to detect and classify attacks in SCADA systems.In addition, an architectural framework has been proposed to enhance malware detection using two ensembles: one employing a deep belief network (DBN) and the other using a standard classifier, specifically SVM [45].
Furthermore, CMRI attacks target CPSs by injecting malicious responses that mimic normal process functionality.CMRIs are especially difficult to detect using this advanced technique because they effectively mask the true state of the system and negatively impact feedback control loops.CMRI attackers typically have extensive knowledge of the targeted system, allowing them to craft responses that blend in with legitimate data.These attacks can cause catastrophic consequences in critical infrastructure environments by manipulating sensor readings, controlling signals, or even deactivating safety mechanisms.A multi-layered approach to CMRI defense is required that combines intrusion detection systems, anomaly detection algorithms, and continuous monitoring of system behavior.ML techniques are capable of detecting patterns and anomalies that may indicate CMRI activity, enabling timely intervention and mitigation strategies.
Malicious function code injection (MFCI) attacks involve use inherent protocol functions that deviate from their intended purpose.For instance, a force listen-only mode attack is a type of cyber-attack that interrupts network transmission by a MODBUS server.Attacks on the MFCI can cause abnormalities in the system's time and control parameters, which affect its normal operation.Using AI capabilities, these attacks can be detected.For example, Wu [46] employed the C4.5 decision tree (DT), naive Bayes (NB), and CNN model to conduct an analysis and compare their respective impacts on intrusion detection.A more appropriate machine learning model for intrusion detection in industrial IoT is used through experimental analysis.
MFCI attacks pose a significant threat to the CPS cybersecurity.These attacks use communication protocol vulnerabilities to inject malicious commands into programmable logic controllers (PLCs).The attacker manipulates built-in protocol functions to achieve unintended consequences that could result in production process disruptions, safety hazards, and even financial losses.MFCI attacks can be classified into several types based on the specific functions targeted.For example, the ''Force Listen Only Mode'' attack disables a Modbus secondary device from transmitting data, effectively silencing it on the network.Combating MFCI attacks requires a multi-pronged approach involving multiple layers of defense.Network segmentation can be used to prevent unauthorized access to ICS devices, while firewalls and intrusion detection systems can be used to filter malicious traffic.
A denial-of-service (DoS) attack disrupts the services of a host on a network, rendering the connected resource unavailable to the intended users.DoS attacks have significant effects, including substantial response delays, excessive losses, and service interruptions.These effects directly impact the availability of a system or service.An AI detection model based on logistic regression (LR) and NB has been proposed as a method for detecting attacks as well as normal scenarios [47], [48].The authors in [49] presented an intelligent agent system that incorporates the K-nearest neighbors (KNN) algorithm to detect distributed denial-ofservice (DDoS) attacks.The system uses automatic feature extraction and selection techniques.
In addition, DoS attacks exploit vulnerabilities present in network protocols or system configurations to deplete substantial resources, including bandwidth, memory, or processing capacity.Consequently, the targeted system experiences a state of unresponsiveness or is overwhelmed, thereby impeding legitimate users from accessing crucial services or resources.DoS attacks can manifest in several forms, including volume-based attacks, protocol-based attacks, application-based attacks, and reflected DoS attacks.These attacks employ third-party servers to enhance the impact of the attack by redirecting the traffic back to the target.The detection of DoS attacks is necessitated by the implementation of a comprehensive strategy that encompasses various aspects such as network security, application security, traffic analysis, and incident response planning.
Reconnaissance (Recon) attacks are security attacks employed by an attacker to acquire comprehensive information about the target before initiating an actual attack.The effect of Recon attacks includes using the gathered information to determine the precise location of the intended target.Furthermore, based on these data, a hacker can determine the type of infrastructure the target uses.AI-based techniques can detect this attack.Kwon et al. [50] presented a proposed intrusion detection system that incorporates reconnaissance to detect anomalous attacks in a CPS using RNN.In addition, an AI technique based on XGBoost and KNN has been proposed for detecting reconnaissance attacks [51].
Furthermore, recon-attack activities encompass the systematic exploration of the target's network infrastructure, where vulnerabilities are identified, network topology is mapped, and sensitive data are uncovered.Threat of enables proactive cybersecurity practices that aid organizations in anticipating and mitigating potential Recon threats.This is achieved by identifying potential attack scenarios, analyzing of vulnerabilities, and implementing of suitable countermeasures.The procedure entails the careful examination of multiple factors, including the capabilities of the attacker, the assets possessed by the target, and the potential consequences that would arise from a successful attack.Common reconnaissance techniques in the field of cybersecurity encompass a range of methods such as open-source intelligence (OSINT), footprinting, vulnerability scanning, and social engineering.The mitigation of Recon threats can be achieved through the implementation of robust cybersecurity measures, such as network segmentation, access control, vulnerability management, and security awareness training.

VI. POTENTIAL COUNTERMEASURES
This section presents several potential countermeasures for addressing cyber security attacks in the CPS.It is imperative to acknowledge that safeguarding a CPS against all forms of cyber security attacks cannot be achieved through the implementation of a single countermeasure.Nevertheless, it is imperative to adopt a multi-layered security strategy to effectively minimize the potential vulnerabilities and threats posed by malicious attacks.CPSs frequently exhibit intricate and decentralized characteristics that pose challenges in ensuring their security.Furthermore, CPSs are frequently employed in critical infrastructure contexts, making them attractive targets for malicious actors.

A. COUNTERMEASURES FOR NMRI AND CMRI ATTACKS
Remove potentially harmful characters and code from all user inputs through input validation.The use of prepared statements is recommended to execute database queries, as it aids in mitigating the risk of SQL injection attacks.A web application firewall (WAF) can be employed as a protective measure against prevalent web application attacks, including cross-site scripting (XSS) and NMRI attacks.

B. COUNTERMEASURES FOR THE MFCI ATTACK
Input validation entails checking the user input for malicious code and characters.The implementation of a allowlist can be employed to impose limitations on the range of functions that can be defined within the application.The use of a sandbox facilitates the segregation of functions, thereby preventing the potential impact of a compromised function on other functions.

C. COUNTERMEASURES FOR A DOS ATTACK
Using a firewall prevents DoS attacks from overwhelming the system and filters out malicious traffic.The use of a load balancer is recommended to evenly distribute network traffic among multiple servers, thereby mitigating the impact of a Denial of Service (DoS) attack on any individual server and ensuring the continued functionality of the remaining servers.A content delivery network (CDN) can be employed to cache static content and distribute it to users from servers near their locations.This approach can effectively mitigate the adverse effects of denial-of-service (DoS) attacks.

D. COUNTERMEASURES FOR THE RECON ATTACK
A firewall should be employed to impose access restrictions on the system and network, permitting only essential traffic to traverse.Intrusion detection and prevention systems (IDS/IPS) are security mechanisms designed to detect and prevent unauthorized access or malicious activities within a computer network.Using an intrusion detection system (IDS) or intrusion prevention system (IPS) to actively monitor both the system and network for potentially malicious activities, including but not limited to port scanning and reconnaissance attacks.It is imperative to ensure the regular updating of system and network software with the most recent security patches to maintain optimal security measures.

VII. CONCLUSION AND FUTURE DIRECTIONS
In conclusion, we have presented a comparison and investigation of AI approaches for cyberattack detection in a CPS environment.The LSTM and XGBoost classifiers were used to analyze the performance toward advanced cyberattack detection in the CPS network communication layer.
The model was trained and tested using real-world benchmark datasets from gas pipelines.Due to the large number of datasets, we had to monitor ACC and validation trends for 100 training epochs.The prediction classification rate was ACC of 97.80% from LSTM and XGBoost 98.69%.The experiment confirmed that XGboost performed better by achieving higher accuracy scores and cyberattack classification rates in CPSs.
We hope that further research can focus on real-time ICS system datasets to detect threats, such as DoS and DDoS attacks, using unsupervised learning.However, our analysis indicates that classification outcomes may be enhanced by including or excluding attributes from gas pipeline datasets or larger sample datasets.Finally, the findings from the ROC curve analysis highlight the effectiveness of both the LSTM and XGBoost models in cyber security attack detection.The implications of these findings are of great importance in the context of improving the security of cyber-physical systems.Additional potential future research could entail investigating ensemble methods or improving hyperparameters to enhance the model's performance.Finally, the nomenclature, which includes symbols, notations, and their descriptions, has been provided (see Table 4).

FIGURE 3 .
FIGURE 3. The gas pipeline system is shown on the left (a), and the right HMI is shown on the right (b) [37].

FIGURE 4 .
FIGURE 4. Machine learning data flow for the input and output of selected features.

FIGURE 5 .
FIGURE 5. Illustration of the LSTM cell architecture.

FIGURE 6 .
FIGURE 6. Architecture of the of XGBoost model. of the model's overall performance.Additionally, performance metrics included confusion matrix, precision, recall, and F1-score, which summarizes the classification model's effectiveness.Furthermore, it comprises true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN) for the developed model.

FIGURE 8 .
FIGURE 8. Model accuracy (left) and loss performance metric (right) for 100 epochs based on training and validation of an initial network (RNN-LSTM).

FIGURE 10 .
FIGURE 10.Selection of the top 10 features of importance (XGBoost).

FIGURE 11 .
FIGURE 11.ROC curves for the XGBoost and LSTM models.

FIGURE 12 .
FIGURE 12.Comparison of AI models across various datasets.

TABLE 9 .
Comparison with other related studies based on the LSTM and XGBoost models.

FIGURE 13 .
FIGURE 13.Illustration of artificial intelligence methods based on attack categories.

TABLE 1 .
Summary of the findings of the related studies.

TABLE 2 .
Seven types of categories from the datasets.

TABLE 4 .
Summary of Important Symbols/Notations.
Meanwhile, for the output gate, Ot determines the output based on the current input and the updated cell state, h t−1 denotes the previous hidden state, x t denotes the current input, Wo, bo represent the weight and bias for the output gate, respectively.

Table 4
summarizes the symbols and notations used in this study.Table5provides the XGBoost classifier parameters.The silent characteristics of the objective function consist of XGBoost classifier parameters.

TABLE 6 .
Statistics of the datasets used in the experiment.

TABLE 7 .
Model performance results.

TABLE 8 .
Comparison analysis of AI models across datasets.