R-EDoS: Robust Economic Denial of Sustainability Detection in an SDN-Based Cloud Through Stochastic Recurrent Neural Network

Cloud computing is now known as the most cost-effective platform for delivering big data and artificial intelligence services over the Internet to enterprises and cloud consumers. However, despite many recent security developments, many cloud consumers continue to express great concern about using these platforms because they still have significant vulnerabilities. Typically, Economic Denial of Sustainability (EDoS) attacks exploit the pay-as-you-go billing mechanisms used by cloud service providers, so that a cloud customer is forced to pay an extra fee for the additional resources triggered by the attack activities. In our previous work, we already proposed an system to mitigate such EDoS attacks. Overall, this previous work presented an effective system for detecting abnormal events; however, the false-alarm rates still remain relatively high and detection rates are low, because abnormal events could be caused by the cloud customer. Furthermore, our previous work still consumes a large number of computing resources. Therefore, in this paper, we propose an enhanced scheme to detect and mitigate EDoS attacks efficiently and reliably. Our proposed scheme is composed of online and offline phases, implementing a gated recurrent unit, which not only can capture complex temporal dependence relations in the data, but also can reduce the vanishing gradient problems in time series. First, to reflect the normal patterns, our proposed scheme learns accurate representations of multivariate time series. Next, these representations are used to reconstruct input data. Finally, the reconstruction probabilities not only can be used to find anomalies, but also can provide interpretations. The proposed scheme also introduces a self-adjusting threshold to reduce error rates, whereas existing solutions normally use a hard threshold to analyze the anomalies, which causes increasing error rates. Our comprehensive analysis of the results shows outstanding performance compared to other solutions and our previous work.

conventional computing services. However, there remain grave concerns about cloud computing security [5]. For instances, exploiting a pay-as-you-go pricing engine on the cloud can force users to pay more for their resource usage. This is commonly known as an EDoS attack, and is one of the most difficult cloud security challenges.
The definition of EDos attack is as follows. A special feature on the cloud called auto-scaling monitors various metrics such as network bandwidth, memory, and CPU usage. This is triggered when the parameters cross predefined thresholds. Once activated, a cloud service provider automatically allocates more resources to meet the increased demands and charges the cloud user according to the service-level agreement. An EDoS attacker exploits the auto-scaling engine to force a target cloud user to pay more for extra resources. EDoS problems become extremely difficult to tackle because: • In the use of SFC, virtual network functions (VNFs) are normally public to internet users. Therefore, EDoS attackers exploit this to launch more VNFs or other reasons [7].
• Existing solutions addressing EDoS attacks are mainly hard-threshold-based solutions with high false-alarm rates [6]. Also, they work well only for a certain distribution of attack traffic, i.e. a Poisson distribution. Moreover, they only focus on some specific types of attack, leaving significant remaining vulnerabilities [39].
• There is still a trade-off between resource consumption and detection performance for an EDoS defender. Therefore, it is a difficult challenge to find a solution that satisfies both resource usage and detection performance requirements. The harmful effects of EDoS attacks are thus a serious security limitation of essentially all cloud-based network systems. Moreover, not many solutions can tackle EDoS attack effectively [6]. A more intelligent, robust approach is needed to ensure that the EDoS attacks can be mitigated effectively.

B. OUR PROPOSAL
We propose a robust, intelligent, and efficient scheme to tackle EDoS attacks, employing a technique named multivariate time series anomaly detection. In the proposed scheme, an anomaly score, used as a basis of attack detection, is compared with a dynamic threshold set automatically according to a trained machine learning model processing traffic statistics collected from a network system. Thus, our scheme can detect EDoS attacks efficiently and is well-adapted to diverse network systems. Furthermore, as a recommendation inspired by N. Agrawal et al., an SDN-based cloud security service provides various options on the cloud (i.e., software-based traffic analysis [47]). Therefore, we propose a more complete EDoS defense system to detect and mitigate the EDoS attacks, which we name robust EDoS defender (R-EDOS).

C. CONTRIBUTION
In summary, our main contributions are listed as follows: • First, existing solutions addressing EDoS attacks are mainly hard-threshold-based solutions with high falsealarm rates [6]. Also, their solutions work well only for a certain distribution of attack traffic, i.e. a Poisson distribution. Moreover, they only focus on some specific types of attack, leaving significant remaining vulnerabilities [39]. Therefore, we propose a new approach that can tackle the above issues, known as a multivariate timeseries, to deal with stealthy EDoS attacks, effectively.
• Secondly, we propose an efficient scheme called R-EDoS which applies the proposed approach to detect anomalous network data generated by EDoS attacks.
• Next, we investigate the characteristics of EDoS attacks, and present common types of EDoS attacks in an SDNbased cloud environment.
• We conducted an experiment using simulated different EDoS attacks on a popular cloud computing platform called OpenStack and an ONOS SDN controller [49]. This integration could be replaced by other SDN controllers, such as OpenDaylight or Floodlight.
• Finally, we collected and analyzed the experimental data, and made a detailed comparison with other algorithms and other existing proposals, including our previous work, in different EDoS attack scenarios. An extensive comparison of R-EDoS against other approaches proves the detection capability of our proposed scheme.
The remainder of this research study is presented as follows. The next section presents related work and Section III discusses background information. Section IV introduces our research rationale, system design analysis, and internal modules. Section V describes our testbed. In Section VI, a variety of evaluation metrics are used to evaluate our proposed scheme. Finally, Section VII draws some conclusions and presents future improvements.

II. RELATED WORK
Recently, EDoS has constantly attracted the attention of researchers for preventing the EDoS attacks from manipulating the auto-scaling engine of the cloud providers. Table 1 shows different approaches to detect and mitigate EDoS attacks. Even though many proposals have been proposed to tackle EDoS attacks (as presented in Table 1); however, no existing proposals have the right approach and propose an effective scheme to tackle such stealthy EDoS attacks effectively and dynamically. From the above analyses and recommendations on defense inspired by two in-depth studies on EDoS characteristics proposed recently ( [4] and [47]), we propose a novel effective solution to deal with EDoS attacks.

A. EDoS ATTACKS ON SDN-BASED CLOUD
In EDoS attacks, by definition, the main purpose is to force a cloud user to rent extra computing resources by manipulating the auto-scaling engine on the cloud so that the user has to pay more money. As shown in Figure 2, before launching an EDoS attack, only three virtual machines are launched. A virtual machine (VM) is a virtual environment that functions as a virtual computer system. However, an EDoS attack exploits the auto-scaling engine in order to launch extra six VMs to meet the resource consumption of the cloud applications so that a cloud user is requested to pay for the attack activities.
Compared to a DDoS attack, an EDoS attack is stealthier and can be highly concealed [6]. A DDoS attacker aims to shut down the network system entirely by irrationally  by simply launching an attack with the utmost resources. Conversely, EDoS attackers are usually more sophisticated and highly logical. They increase the amount of unauthorized traffic gradually, and the main purpose is to force their target to pay more. EDoS attacks always try to stay below a traditional intrusion detection system's threshold value. In an SDN context, EDoS attacks not only degrades the SDN/NFV paradigm on the service provider end, but also forces a cloud user to pay for the unexpected attack event caused by an EDoS attacker. Figure 3 shows different traffic regions among normal traffic, EDoS, and DDoS. The intensity of EDoS attacks normally stays below the intensity of DDoS attacks. Therefore, an EDoS attack can easily bypass DDoS defense mechanisms. Thus, detecting an EDoS attack becomes a very complex challenge. An intelligent and robust approach is needed for cloud users and service providers to detect to detect EDoS attacks emerging in real time.
Compared to an EDoS attack in the cloud, an EDoS attack in an SDN-based cloud environment is basically similar. However, by integrating an SDN paradigm into a cloud environment, an EDoS attack can be easily detected because the SDN paradigm provides software-based traffic analysis [47], which means it provides various important traffic statistics provided by an SDN controller where it controls, manages, and collects all the network traffic. Therefore, we first propose a novel EDoS defense approach, leveraging the benefits of the SDN paradigm, to not only detect and mitigate different EDoS attacks effectively but also overcome the shortcomings of the existing approaches (as stated in sections I.A and I.C).  Figure 4) [2] includes a control module including SDN controller and an NFV orchestrator, and network devices. The SDN controller takes responsibility for managing the traffic path to exchange information with forwarding devices (OpenFlow switch) using the OpenFlow protocol primarily. And the NFV orchestrator is responsible for managing NFV services. VMs, which operate on top of hypervisors, are normally supported by hypervisors in launching and running network functions. Readers can refer to [2] for specific details of the SDN/NFV paradigm. Note that a VM in Figure 4 implies a virtual machine.

C. GRU, VAE, VARIATIONAL METHOD, AND NORMALIZING FLOWS
Recurrent neural networks (RNNs) [18] are a set of feedforward neural networks commonly used for time series data. Unlike conventional neural networks, RNN's have an integrated internal memory to process sequences of inputs. However, RNNs are vulnerable to exploding and vanishing gradient problems. Moreover, they cannot process long sequences. Fortunately, a variant form of RNN called gated recurrent unit (GRU) can overcome RNNs' major disadvantages. It is explicitly designed to avoid the long-term dependency problem which normally arises in time series data [19]. Thus, in this work, we employ the GRU mechanism.
In detecting anomalies of higher-dimensional data like multivariate time series [21], a famous method known as a variational autoencoder (a.k.a. VAE) [20] is often used. This provides a probabilistic model for describing an observations in latent space, and can also be transformed into a lowerdimension space preserving particular properties of the original dimension space.
Another variational algorithm known as stochastic gradient variational Bayes (SGVB) [20] commonly is used along with VAE to tune parameters. As proposed in [26], SGVB is not only used to tune parameters of both φ and θ but also maximize the evidence lower bound (a.k.a. ELBO).
To train a machine learning model, its parameters must be tuned for increasing the probability of predicting data in the training dataset under the model. However, this can be problematic if the output of the model is assumed to show a Gaussian distribution, as the true probability density function (PDF) of real data is often far from Gaussian. Therefore, Danilo Rezende et al. proposed a transformation technique called Planar normalizing flows (Planar NF) [22] to capture (q φ (z t |x t )) which is the non-Gaussian posterior density by transforming it using invertible mappings.
The basic idea of the combination of the four key techniques above is described as follows. GRU first captures complex spatiotemporal dependencies in state-space. Next, a variational algorithm (VAE) maps observations in statespace to stochastic variables. Next, a connection technique introduced in [27] known as linear Gaussian state-space modeling connects stochastic variables to GRU hidden variables [26]. Finally, a normalizing flow technique known as ''planar flow'' is used to support stochastic variables in qnet (defined in section IV.D.6) in capturing highly complex distributions of input data.

IV. R-EDoS: ROBUST EDoS DETECTION AND MITIGATION
In this section, our research rationale and an analysis of our system design are first given. Then, the components, workflow scheme, and the system process logic are introduced, respectively. Finally, the internal components of our proposed scheme are thoroughly explained. Also, all variables, constants, and parameters are summarized in Table 2.
A. PROPOSED SCHEME RATIONALE AND DESIGN ANALYSIS Cloud users are normally monitored with various metrics over time when auto-scaling of resources is critical for service level agreements (SLA). For instance, as illustrated in Figure 5, cloud instances' parameters, such as CPU utilization (cpu_util), memory consumption (memory_consumption), number of incoming bytes (network.incoming.bytes) and so on, are collected and tracked by the auto-scaling engine. If an EDoS attack is initiated, some of the chosen metrics will abruptly change their values. Therefore, to detect anomalous periods in our multivariate time series during the attack, we need an intelligent model capable of learning from the network system. Thus, in this paper, we employ a method known as a stochastic recurrent neural network [26] that not only is able to learn from historical data but also to simultaneously keep track of the multivariate time series to detect an EDoS attack. The key idea of this technique is to output an anomaly score by learning from the patterns of the multivariate time series; then to compare this anomaly score with a dynamic threshold by using the reconstruction probabilities, thus detecting anomalies accurately.
A single time series involves consecutive observations that are appended at equally spaced timestamps [29]. Multivariate time series can be expressed as  from time t − T to t written as {x t−T , x t−T +1 , . . . , x t }. The objective of anomaly detection in multivariate time series is to decide if a new observation x t is normal. Moreover, the historical data is crucial for predicting the current value of a sequence in a time series. Therefore, we take all the sequences x t−T :y to calculate an anomaly score. An anomaly score for x t is produced from the anomaly detection model, then this anomaly score is compared against a dynamic threshold to calculate the anomaly prediction result. Figure 6 is sketched to show a complete overview of our EDoS defense framework extending from both a control layer and a application layer with different modules. Specifically, a raw data preprocessing scheme, a training database, and an attack handler are placed in the SDN control layer; whereas a model training, a dynamic threshold, and an online detection modules are located in the SDN application layer.

B. R-EDoS MAIN MODULES AND SYSTEM WORKFLOW
Our EDoS defense scheme is placed in the SDN controller, including five main modules, as presented in Figure 7, which are a raw data processing scheme consisting of five submodules, an offline training model, an online detection, a dynamic threshold module, and an attack handler. This raw data processing module includes a traffic collector, an feature extractor, a transformation & standardization module, a Savitzky-Golay filter module, and an augmented Dickey-Fuller test, in that order.
The data preprocessing scheme is constructed from five different sub-modules and is also used by both model training and online detection. The process inside the module proceeds as follows. First, the traffic collector sub-module reliably gathers data on both data plane and instances at a certain periods of time. Subsequently, a feature extractor sub-module receives this data. Next some of the extracted features are selected as more relevant, and are further transformed into scaled data and smoothed using a Savitzky-Golay filter technique in order to raise the precision of the data without distorting the signal tendency and is checked stationarity using an augmented Dickey-Fuller sub-module. Once the data is identified as stationary, then it is divided into a series of sequences through sliding windows [30] (T + 1 in length). After the raw data is preprocessed through this scheme, and given amount of time elapses (i.e. 5 minutes), the training module starts a training session using the preprocessed data. During training with the preprocessed data, the learning model tries to discern typical patterns in the data, and then scores a new observation. This scoring value is called an anomaly score, later compared with a dynamic threshold following the peaks over threshold (POT) approach [32].
Note that, to effectively adapt our proposed scheme to different network systems, we continually update the training database with new attributes that are collected from the raw data preprocessing in a preset time.

C. SYSTEM PROCESS LOGIC
As seen in Algorithm 1, N is represented a sequence of observations where each observation is an M -dimensional vector where each dimension indicates a network metric.
It is later compared with the length of x ι t−T :t at the next current time t. Before sending the sequence x ι t−T :t with T consecutive observations to both the offline model training (OfflineMT ) and online detection (OnDetect) models, x ι t−T :t is fed into a transformation & standardization sub-module (TSS) so that the input can be transformed, and standardized x ι t−T :t . Once the input is already transformed and standardized in the TSS sub-module, a new sequence is formed from x ι t−T :t . This sequence is then filtered using the Savitzky-Golay technique (SAGF) and is checked for stationarity or using an augmented Dickey-Fuller test (AUDF). x t−T :t represents the sequence of successfully passing the stationarity test. Once an anomaly threshold th F and an anomaly score anomaly_score are obtained from the dynamic threshold module D − thres, and OnDetect; a new observation will determine whether the anomalous data indicates an attack as follows: • If S 2t >= th F is true, then it indicates the new observation coming from a normal source. Thus, the algorithm is looped over again.
• Otherwise, the new observation is predicted to be derived from an attack source. The observation is sent to the attack handler to perform either drop or remove actions. Then, the algorithm is also looped over again.

D. INTERNAL MODULES
Here, we present all the components of R-EDoS in order, as shown in Figure 7.
There are two sources (data plane and instances) that the traffic collector needs to gather to guarantee the adequacy of all key features as described in Table 3. First, a statistics request function is run periodically to request statistics of an OpenFlow switch in order to collect data from the data plane ( Figure 7). Once the OpenFlow responds to the request function, the OpenFlow channel forwards a StatsResponse message which includes requested statistics to the data processing scheme. Meanwhile, a Python script fetches other necessary statistics of cloud instances monitored by Open-Stack Ceilometer.

2) FEATURE EXTRACTOR
At this stage, we run a feature selection process to automatically select features that contribute the most to the classification decision, applying the Boruta algorithm [34] to select both unbiased and stable attributes. There are some factors that make Boruta a notable success in finding top important features from particular data like time series; for instance, it takes both the interactions and the relationships of multiple variables into account. The Boruta algorithm workflow can be summarized as follows. First, it duplicates the original dataset; then the z_score values, which are acquired from the two dataset using a random forest algorithm, are used to compare with each other. Next, if z_score value in the Note that: ''Und'': Undetermined, ''Im'': Important, ''Un'': Unimportant duplicated dataset is consecutively smaller than in the original dataset, then the selected feature is chosen.
The following Algorithm 2 and 3 explain the processes of feature selection with Boruta in detail. First, an addition system is created by using shadow samples (var shadow f ) for all variables f t ∈ F S in which all the features are mixed to eliminate biases and high correlations among them. Next, a random forest algorithm (RandF_Clf ) is trained using this addition system, and then it is used to evaluate the importance level of each feature (var ip ). Classification is then conducted using decision trees from different bagging samples. The z_score score is then calculated. It is noted whether the z_score score is statistically significant for the feature; if not the procedure is run again several times. During every iteration, by comparing a random set of attributes with actual attributes, a maximal importance (max z_score ) is calculated. For all the attributes, a statistical test is conducted. After the test, trivial attributes are removed. If the importance value of an attribute is greater than max z_score , then the attribute is considered ''important''. On the contrary, If the importance value of an attribute is considerably smaller than max z_score , then the attribute is considered as ''unimportant'' and will be removed. Lastly, attributes that are marked as ''undetermined'' receive no further actions.

3) FEATURE TRANSFORMATION & STANDARDIZATION
This module is responsible for transforming all the selected features from the previous stage. First, the categorical features such as src_ip and dst_ip are transformed into numeric variables using one-hot encoding [35]. Second, as stated in section I.A, unlike existing solutions which are limited to some specific distributions (typically a Poisson distribution), in this paper, we employ a method called Yeo-Johnson transformation [16] which is a power transform. Yeo-Johnson not only stabilizes variance, which can solve issues related to heteroscedasticity, but also normalizes the data distribution, transforming it to more closely resemble a Gaussian distribution, to increase the prediction accuracy of our detection model without having to take into account the mixed distributions of the attack traffic. Therefore, all the selected features are further transformed based on Yeo-Johnson's formula as follows. In equation 1, d i indicates the preprocessed data, and β indicates the transformation parameter. Note that we also set standardize = True to standardize the preprocessed data.

4) SAVITZKY-GOLAY FILTER
After being transformed and standardized in the previous step, the data is further smoothed to remove noise and outliers caused by non-stationary time series. Specifically, some data may be lost or unusual while collecting or transmitting data, and this may cause interference data. To smooth the original sequence, remove noise, and retain time series' peak and width, the least square polynomial smoothing method (also known as a Savitzky-Golay (SG) filter) [36] is used. The method works best with a window size of 5.
A time series is represented as: where x t means a set of tasks at time slot t, N + ={1, 2, . . .}, X is the workload, and Y n (n[m + 1,t-m]) indicates a subsequence of X . Y n is defined as: To find the best mean square, a set of 2m + 1 consecutive values is used. In which coefficients of a polynomial are as follow: It is noted that the value of n ranges from −m to m, and n = 0; therefore, the least-square criterion requires that the squares of the differences in total between the observed values x m+n and the computed values p(n) be the minimum over the time slot being observed [37], defined as: Then, the central point of the fitted polynomial implies the smoothed data point.

5) AUGMENTED DICKEY-FULLER TEST
For the model to learn from time series data reliably, it is crucial to check whether a sequence is stationary or not, therefore, a test called augmented Dickey-Fuller (ADF) [37], also known as a unit root test, is employed to check the stationarity of each data sequence. The role of ADF is to determine how strongly a time series is characterized by a trend. The test checks if a unit root is included in a sequence; if not then the sequence is non-stationary; otherwise, it is stationary.
For the ADF experiment, the null hypothesis (H0) occurs if the time series is considered as non-stationary. Therefore, if the processed sequence fails in the stationarity test, a differencing technique is run, after which the sequence will be tested again. On the contrary, if H0 is rejected, it suggests that the examining sequence does not have a unit root, meaning that it is stationary; therefore, it is then sent to the training model.
In ADF, there are three degrees of rejection 1%, 5%, and 10%, in which a rejection degree of 1% indicates strict rejection of the original hypothesis. That means if the ADF value is less than the rejection degree of 1%, then the examining sequence is not non-stationary. Moreover, we interpret the test result using the p-value. If the p-value lies under the rejection degree of 5% or 1%, then it indicates stationarity (rejection of H0), otherwise a p-value lies above the rejection degree, indicating nonstationarity. For example, the results of the ADF test of a flow_duration feature are shown in Table 4. The calculated ADF value is much smaller than the rejection degree of 1%, and the p-value is extremely low (< 0.0001). That means the selected feature is stationary and is qualified to be used in training the model.

6) MODEL TRAINING
The model training is composed of two main networks: qnet and pnet (Figure 8(a) and 8(b), respectively). First, in the pnet network, z t−T :t , indicating a latent representation, is used to reconstruct x t−T :t . The more precise the representation of x t−T :t , the lower the reconstruction loss. Second, the qnet network approximates the pnet network by optimizing itself in order to obtain a higher quality latent representation.
As proposed in [26], by tuning three network parameters (u*-s, w*-s, and b*-s), both the qnet network and pnet network are trained at the same time. The size of each input sequence data (e.g., x t−T :t ) is T + 1. According to [26], for the l-th sample z (l) t−T :t , where q ≤ l ≤ L and L is the sample length, the loss function is formulated as: In Equation 6 given above, for each sample: • The first term is the negative reconstruction error and it is equal to t i=t−T log(p θ (x i |z t−T :i)) . The posterior probability of x i is formulated as: p θ (x i |z t−T :i ) ∼ N (µ x i , σ 2 x i I ) [26].
• The second term is equal to t i=t−T log(p θ (z i |z i−1 )), in which z i is acquired by linear Gaussian state space modeling proposed in [26].
• The third term approximates the authentic posterior distribution in the z−space of the qnet of z i and it is equal VOLUME 9, 2021 to − t i=t−T log(q θ (z i |z z i −1 , xt − T : t) where z i (i.e., z k i ) is converted from a normalizing flow technique called planar NF (section III.C).
and f k is invertible function mapping [26].

7) ONLINE DETECTION
At this stage, the trained model is used to determine whether or not an observation at some given time step (i.e., x t ) is anomalous. It is noted that the input of our learning model is a set of sequence data with length T + 1. Hence, x t−T :t , including a sequence of observations, is taken as an input to reconstruct x t [25]. This reconstruction then acts as an anomaly score by applying the conditional probability, as suggested in [23]. S 2 t represents the anomaly score of x t , so S 2 t = log(p θ (x t |z t−T :t )) [26]. A high value of S 2 t implies x t is reconstructed well, meaning that the observation closely follows the usual patterns of the time series. In conclusion, if the calculated threshold value is greater than S 2t , then the observation x t is abnormal and vice versa.

8) DYNAMIC THRESHOLD MODULE
First, an anomaly score is calculated while running the model training module, in which each observation is represented as a multivariate time series of N . The module then produces a univariate time sequence in the form of S 1 , S 2 , . . . , S N , as illustrated in Figure 7. Second, an anomaly threshold (th F ) is calculated by applying extreme value analysis (EVA) [31]. EVA is a statistical method that is commonly applied to discover extreme deviations of a probability distribution without making assumptions about the distribution. According to [31], the extreme values are often located in the tail of the probability distributions. A common approach in EVA is generally referred to as peaks-over-threshold method (POT) [32], which learns the tail area of the probability distribution using a generalized Pareto distribution (GPD) with two parameters, which must be tuned (low_quantile < 7% and q = 10 −4 ). Readers can refer to [26] for further details.
Formally, anomaly threshold is defined as follows.
In Equation 7, N is the number of observations, N th the number of S i s.t. S i < th, and q the intended probability of finding that S < th.

9) COMPARISON FUNCTION
This function simply compares the calculated threshold to the calculated anomaly score. If S 2t < th F is true, the model determines that the input data indicates attack traffic, and the attack traffic is sent to an attack handler. Otherwise, it was determined to be a normal traffic.

10) ATTACK HANDLER
Once the training model of R-EDoS detects attack traffic, then an attack handler forwards a flow_mod message, including a delete action, to the OpenFlow switch. The attack handler requests the SDN controller to perform dropping packet_in messages of the attack traffic (also presented in Figure 7).

V. EXPERIMENTAL SETUP
In this section, we show common EDoS attack scenarios that are also simulated by us in this work. Next, the description of our dataset is given. We also show readers our sensitivity analysis and a comprehensive description of model training. Finally, we describe our implementation in an SDN-based cloud.

A. ATTACK SCENARIOS
There are no existing works introducing comprehensively what are some common EDoS attack scenarios in an SDNbased cloud that are normally used by an attacker. In this work, we divide into five common EDoS attacks that are mentioned in these studies ( [12], [17], [39] and [42]). Note that we also followed their instructions to launch these five different EDoS attacks.

1) TCP-HTTP FLOODING ATTACK (HTTPAttack)
An HTTP flood attack [39] is one of the DDoS attacks where a webserver is exploited by an attacker through HTTP GET or POST requests. The attacker forces the targeted webserver to allocate the utmost resources for each request. The request is normally a resource-intensive request.

2) DATABASE API REQUESTS (APIRequests)
In the case of Database API requests attack, a virtual machine works as a database server where it is frequently queried through HTTP requests. A cloud provider usually charges a user for the number of API requests. For example, a cloud user wants to create a custom dashboard that makes calls to a relational database RDS provided by AWS [48]. The user needs two APIs such as DescribeDimensionKeys, and GetResourceMetrics and the dashboard calls these two APIs every 5 seconds. The number of calls in 30 days would be: 2 API calls * (30d * 24h * 60m * 60s) / (5s) = 1,036,800 API calls. If AWS provider monitored only two instances with the custom dashboard refreshing every 5 seconds, the total cost would be: ((2 * 1,036,800 -1,000,000) / 1,000) * 0.01 = $10.74 per month. By manipulating this pricing calculation, an EDoS attacker can create more fake API calls more frequently and the user needs to pay extra fees.

3) TCP-SYN FLOODING ATTACK (SYNFlooding)
An SYN flooding attack [39] also known as ''half-open attack'' aims at consuming all available resources of a server so that it is unable to respond to upcoming traffic. This attack behavior is similar to DDoS attacks. An attacker continuously sends an initial connection request to a targeted server making all ports unavailable to respond to upcoming legitimate traffic.

4) YO-YO ATTACK (YoYo)
Attacking the auto-scaling engine literally means EDoS attack, in which Yo-Yo attack [42] with two modes (on-attack and off-attack) is revealed attack against the auto-scaling engine. In the first mode, an EDoS attack forwards a burst of network traffic to trigger the scaling engine so that more instances are launched. Conversely, in the second mode, after the attacker believes that all instances are up and the service is functional; then it will halt forwarding network traffic until it verifies that scale down has occurred.

5) SLOWLORIS ATTACK (Slowloris)
Slowloris attack [46] is commonly known as ''low and slow'' attacks because it utilizes low bandwidth and aims at mimicking normal traffic. Its goal is to open and to keep many connections as long as possible by using partial HTTP requests and targeting a webserver on the cloud. Once the attack is launched, the web server will be overloaded because there are so many opening threads that need to handle. If the connections exceed the web server's maximum connections, then a denial-of-service phenomenon will emerge, which means upcoming requests will not be responded to. Note that EDoS attacks are similar to low-rate DDoS in characteristics [4]. For example, as specified in [3], a highrate DDoS attack occurs when the packets per second > 10, 000; therefore, 10,000 packets/s are considered the standard threshold to differentiate between low rate DDoS and high rate DDoS. Moreover, low rate DoS attacks normally account for 10-40% of normal traffic [3]. Hence, based on both the studies, to create realistic EDoS attacks with the five attack scenarios introduced above, we stimulated different levels of EDoS attacks, in which EDoS attacks always account for 20% of normal traffic, and the number of requests ranges from 1000 to 7000 following the EDoS evaluation scheme proposed by Al-Haidari et al. [43] as shown in Table 5.

B. DATASET DESCRIPTION
One of the most realistic data of network traffic with various metrics such as CPU usage, network, and memory consumption, and so on of different servers known as SMD was chosen. This new dataset was publicly published in KDD 2019 [26] and was collected from real network traffic statistics of a large Internet company. It is also used in many anomaly detection studies mentioned in [25]. In our study, the offline model module was trained and tuned its hyper-parameters using the SMD dataset, in which we sampled this dataset with 20% attack traffic. The shapes of the training set and testing set are (28479, 38) and (20300, 38), respectively. Furthermore, to evaluate our proposed scheme with a reallife dataset, we also collected more data samples from our simulated testbed environment with the shape of (16250, 38) including attack and normal traffic.

C. SENSITIVITY ANALYSIS OF USER-DEFINED PARAMETERS WITH TAGUCHI AND MODEL SETUP
To achieve the best results of R-EDoS where the number of experiments is as little as possible, it is essential to investigate its user-defined parameters. In our proposal, four highly sensitive parameters are needed to find their best values, which are data sequence length, z− space), planar NF length, and GRU layers and dense layers. In order to pick a fair combination between factors rather than the entire combination, we use the popular Taguchi method proposed in [28], in which fractional factorial designs (a.k.a. OAs), VOLUME 9, 2021    To produce the best outcomes and get valuable insights significantly from the learning model of R-EDOS, we obtained the best settings of user-defined parameters based on the Taguchi results, and we also closely followed the descriptions in [26]. Based on the sensitivity analysis Taguchi given above, the z-space dimension equaled 3 and the planar NF length was assigned as 20. Besides, the GRU and dense layers are equal to 500 units and the was set to 10 −4 in the standard deviation layer. The input data sequence is 100 in length. Also, here are other hyper-parameters settings. The model was trained with 20 epochs and a batch size of 50 including the early stopping regularization criterion. Adam optimizer was selected as an optimization algorithm for stochastic gradient descent. Besides, the value of the learning rate was 10 −3 . While running back-propagation, gradients may increase their values rapidly with a very large number resulting to the overflow problem (i.becomes float.nan) of model parameters [26]. To avoid this problem, the gradient clipping was used and was set to 10. In addition, all layers in the learning model were penalized by applying the L2 regularization technique. Finally, the hyper-parameters settings used in this work are summarized in Table 7.
In Boruta, an irregular z_score value problem may cause an infinite loop issue. Fortunately, the issue can be addressed by setting the depth to 7 [33]. Hence, in our work, we also set the depth of tree = 7.

D. DETAILED IMPLEMENTATION AND TEST PREPARATION
In this work, as suggested in [44], we set up our OpenStack-SDN based testbed according to it. Figure 9 displays the testbed in a production environment for virtual networks to simulate different EDoS attacks. Our testbed consists of an OpenStack platform (one controller of SDN known as (ONOS Quail version 2.0.0) [49], a controller of OpenStack, two computing nodes attached with OvS version 2.3.2), and one networking node). In which, Apache Spark is also installed into our cluster so that we can not only leverage an optimal preprocessing process but also the learning process for machine learning algorithms. We used OpenFlow version 1.4 in this work. Bonesi simulator was employed to perform flooding attacks with low attack rates, besides sFlow and sFlow-RT tools were employed to perform collecting data from the SDN paradigm. As we introduced in section IV-A, the modules of R-EDoS are sat in the ONOS node ( Figure 6). Our system configurations are presented in Table 6 running on Ubuntu operating system 64bit version 18.04. It is noted that we created an auto-scaling group on OpenStack with nova computing instance configurations as follows, instance type: m1.small, VCPUS: 1, memory: 2048MB, storage: 20GB. For two particular attacks (HTTPAttack and Slowloris), we created an auto-scaling group in which instances work as a web server. And for an APIRequests attack, OpenStack service named Trove is a database as a service of each instance.
To run both Savitzky-Golay filter and augmented Dickey-Fuller test, SciPy package and statsmodels were used respectively. Also, the Yeo-Johnson was run by using sklearn package and chose its λ parameter automatically.

VI. RESULT ANALYSIS
In this section, we first compare R-EDoS with other solutions and then present detection performance. Next, resource usage and the overall performance of our R-EDoS scheme are analyzed. Then, we interpret the learning model of R-EDoS and then briefly discuss our deployment. We conclude the section with a general discussion.

A. COMPARISON WITH OTHER SOLUTIONS
To prove the effectiveness of R-EDoS, we conducted practical experiments to make comparisons with other existing works using the same environmental set-up. First, to the best of our knowledge based on a review of the relevant literature. Abbasi et al. [39] is the only work applying machine learning technique to an EDoS detection problem, excluding our previous work [17]. Hence, we first compared our proposed scheme with their two approaches (support vector machine (SVM) and neural network (NN)) independently). Second, to prove our approach, which is multivariate time series detection, to be practical, we also compared it with another machine-learning-based method called one-class SVM [40]. According to Schölkopf et al. [40] one-class SVMs separate all the data points into two spaces where the distance between the two spaces are maximized as much as possible. It is a well-known algorithm and has been used in many anomaly detection studies [41]. Last, our previously published work [17], was also selected for comparison to demonstrate the expected enhancement of R-EDoS.
It is noted that we implemented these studies, [17], [39], and [40], in our testbed to keep not only their novel features but also their original functionalities.

B. ANOMALY DETECTION PERFORMANCE
To rigorously assess the anomaly-detecting capacity of R-EDoS, four criteria, including F1-score, detection rate, accuracy, and false alarm rate mentioned in [45] are employed. To calculate the four criteria, we need four other different measurements including false positive (FP), false negative (FN), true positive (TP), and true negative (TN). TP indicates the ratio of illegitimate traffic that is recognized as such TN means the percentage of normal traffic that is detected as anomalous traffic, FP shows the percentage of legitimate traffic that is detected as anomalous, and FN reflects the percent ratio of abnormal traffic that is classified as normal traffic. The formula of the four criteria are expressed as follows: Hence, this score takes both FPs and FNs into account.: • F1-score (F1) is the weighted average of P and R.
where P = TP TP+FP In summary, the detection rate reveals the percentage of correctly detected attacks over the total number of attacks. FAR represents the ratio of falsely detected attacks to the total of falsely identified entities, whereas accuracy tells how accurate R-EDoS is. Lastly, F1-score takes both FPs and FNs into account implying the weighted average of P and R. Figure 10 shows the detection performance of five mentioned solutions: R-EDoS, NN-Abbasi et al. [39], SVM-Abbasi et al. [39], our previous work [17], and one-class SVM [40]. The results were acquired through testing on 10 sub test sets. It is obvious that R-EDoS both outperforms the other solutions in regards to accuracy, F1-score, and detection rate and produces the lowest false-alarm rates under different EDoS attack scenarios. Conversely, the NN-Abbasi et al. solution performs poorly regarding to the accuracy, detection rate, and F1-score. Furthermore, it produces false-alarm rates with the highest rate. Among the three other solutions, it is shown that our previous work still performs better than both SVM-Abbasi et al. and one-class SVM solutions, while the former performs slightly better than the latter.
Based on formulas [9]− [12], we calculated anomaly detection performance metrics on average as presented in Table 8. Regarding accuracy and detection rate, R-EDoS and our previous work clearly outperform the other three solutions. R-EDoS demonstrates stable prediction performance and it always achieves the highest scores among the models compared, with a detection rate of 96.27%, and an accuracy of 94.33%. Moreover, our two works also produce fewer wrong warnings than the others, with R-EDoS only showing 4.72% of the total test traffic sequences as false-positives, whereas [17] had a higher false-positive rate, 9.75%. In comparison among NN-Abbasi et al. [39], SVM-Abbasi et al. [39], and one-class SVM [40], a pure neural network struggles to learn from higher dimensional time series data; therefore, it performs poorly in detection and produces a very high false-alarm rate, 20.86%. One-class SVM is specifically designed for detecting anomalies and its main advantages are less time and storage space to run compared to two-class SVM [41]; therefore, as shown in Table 8, its prediction performance is slightly poorer than SVM-Abbasi. Lastly, the F1-score is an appropriate measurement criterion to properly assess the efficiency of R-EDoS since it is taken into account by both FPs and FNs. As introduced in Table 8, our proposed scheme demonstrates its outstanding prediction ability with a 95.51% F1-score, which is higher than the second highest F1-score by 4.63%. The approaches Following NN-Abbasi et al., SVM-Abbasi et al., and one-class SVM achieve 76.58%, 85.4%, and 82.2%, respectively.
Regarding comparison among five attack scenarios (as shown in Figure 10 and Table 8), R-EDoS outperforms the others, and it can perform well on all five attack scenarios overall, even though it has some difficulties in detecting both YoYo and Slowloris attacks, due to their stealthy characteristic of mimicking normal traffic. Furthermore, the box plots of R-EDoS (in Figure 10) demonstrate the high stability of our proposed scheme's detection capability.
Through the comprehensive analyses above, it is clear that our multivariate time-series approaches (R-EDoS and our previous work [17]) outperform both deep-learningbased and machine-learning-based approaches (neural network, SVM [39] and one-class SVM [40]) in detecting EDoS attacks. Our proposal (R-EDoS) shows impressive results in detecting various types of attacks with low false-alarm rates.

C. ATTACK MITIGATION PERFORMANCE
Agrawal and Tapaswi [47] recently proposed some important measurements to assess the effectiveness of a defense scheme, such as accuracy, attack detection time, service response time, etc. Hence, the metrics given above are adopted in this paper. Note that all the metrics presented in this subsection and the following subsection were measured by us during a SYNFlooding attack with a fixed attack rate of 7000 req/s excluding the response time metric.
First, to measure how quickly a new attack is exposed, we created a function to calculate the average time for a new attack to be exposed during the simulated SYNFlooding attack we launched (as shown in Figure 11). As mentioned before, one-class SVM's main advantages are less time and storage space to run; it only requires 0.8 seconds on average to raise an alarm if it detects an EDoS attack, and definitely  outperforms others in terms of detection time. However, the detection time difference between one-class SVM and our two approaches is not very significant because the data sequence length of R-EDoS and our previous work was only set at 100 and 250, respectively. That means the detection time of R-EDoS is only 0.91 seconds (and 0.97 seconds for our previous work).
Second, service response time is one of the most important metrics in evaluating any defense system [47]. This is defined as follows. Once a cloud user makes a request, then one corresponding response is sent back to the user. Figure 12 shows that the higher the attack rate, the longer the response time. Overall, the response time of all five solutions is short, varying between 11.1 ms and 32.6 ms. Specifically, R-EDOS, our previous work, and SVM-Abbasi et al. are similar in response time, gradually increasing in the 20 -30 ms response time range.

D. RESOURCE CONSUMPTION AND COST CALCULATION
To demonstrate the lightweight mechanism of R-EDoS, resource consumption, including CPU and memory, is also provided along with the total estimated cost.  As mentioned in section V.D, we placed all modules in an SDN controller node. CPU and memory usage were measured over time. For CPU usage, owing to setting the data sequence length = 100, R-EDoS consumes the lowest amount of CPU, varying between 28% and 33% over time, compared to other solutions (as presented in Figure 13), whereas, other solutions are highly resource-intensive and they tend to perform unreliably in that their CPU utilization widely fluctuates (i.e., SVM-Abbasi et al. varies between 55% and 69%). This wide fluctuation behavior may degrade the whole cloud network. As for memory usage, all solutions consume a low amount of the memory resource (under 9.4%), with R-EDoS only fluctuating between 2.86% and 3.7% (as presented in Figure 14). Evidently, R-EDoS outperforms other solutions, including our previous work, in terms of CPU and memory usage. This proves a great improvement of R-EDoS in resource usage compared to our existing work, as stated in the abstract. Figure 15 shows the number of cloud instances allocated during the simulation of the SYNFlooding attack, with a fixed attack rate of 7000 req/s and the CPU upper threshold = 80%. The results show that when under attack, R-EDoS can save from 33% to 60% on the number of running instances. For example, 6 instances were launched after 120 minutes under the attack. By applying R-EDoS to the simulation, only 3 instances were spawned in the same amount of time. Or in the 105th minute, the number of instances under the attack with R-EDoS was only 2, compared to 5 instances without integrating R-EDoS during the attack.
Recently, Al-Haidari et al. [43] proposed a formula that can easily be used to estimate the future cost that a cloud user would have to pay for their resource usage based on this formula: where Price bw indicates the price of bandwidth for every 1 GB, λ GB/s (= λ legitimate + λ malicious ) is the arrival rate in which λ legitimate and λ malicious are the legitimate and attack traffic rates respectively, in requests per second, Price com implies the basic fee charged per hour for each instance, and T means the total number of hours that the cloud user intends to use cloud resources for their application, and S indicates the number of running instances. Assume a cloud user needs to create a Web server for its business website. Therefore, the user chooses AWS [48] to deploy its application with system configurations as follows. Ubuntu Server version 20.04 runs on CPU t3.xlarge size (4 vCPUs, 2.5 GHz, 16 GiB memory, 8 GiB EBS with IOPS = 100 / 3000), the data transfer rate supports up to 5 Gigabit, and the on-demand price per hour in the Asia Pacific (Seoul) region is $0.1344 [48]. Based on Equation 12, if the cloud user intends to run the business website for one month under EDoS attacks, doing so will cost an estimated $654.372, where Price bw = 0.09 [48], Price com = 0.1344, S = 6, λ GB/s = 1, and T = 730. Using our proposed scheme, the cloud user need only pay AWS with the amount of $327.186 under EDoS attacks with R-EDoS integrated into the server, where S = 3 and λ GB/s = 0.5. We estimate that in this scenario, the cloud user can avoid paying an extra $327.186 each month for the EDoS attacks.

E. INTERPRETABILITY AND DEPLOYMENT DISCUSSION
First, during the simulation of different EDoS attacks, we interpret the experimental results using a feature importance technique [38] for each type of attack. The interpretability results are as follows.
• For the SYNFlooding attack, three key features that have the most contribution to the prediction are no_syn_pkt, no_pkt_per_flow, and no_ack_pkt.
• For the HTTPAttack attack, src_ip, network_protocol, no_established_connections, and dst_ip play the main role in detecting this type of attack.
• For the YoYo attack, the four main features that have the most contribution to the prediction are no_requests_to_webserver, no_running_instances, src_ip, network_protocol.
• For the Slowloris attack, no_established_connections, no_ack_pkt, and network.incoming.packets are two factors that greatly contribute to detect the Slowloris attack.
Second, during tuning of R-EDoS hyperparameters, we observed the sensitivity of the results to variations in the value of the hyperparameters. We found that the model becomes highly sensitive if we set z-space dimension at a high value. In this case, we found that the reconstruction probability was unable to find a good posterior, whereas the underfitting problem remains unchanged.
Third, in the case of a legitimate cloud user suddenly sends a large number of requests, which increase the values of some key features such as network.incoming.packets, network.incoming.bytes, no_established_connections, etc. R-EDoS may produce false-positive rates in this case. However, the dynamic threshold will dynamically adjust its value to reduce the false-positive rates because the constructed usual pattern of R-EDoS will change its pattern according to the values of the key features.

F. GENERAL DISCUSSION
Based on our comprehensive result analyses given above, we summarize some outstanding points that demonstrate the effectiveness of R-EDoS in detecting real EDoS attacks conducted in our practical testbed: • R-EDoS not only produces low error rates but also yields high detection rates, accuracies, and F1-scores, which clearly show that it outperforms other solutions.
• Our proposed scheme can defense against some common EDoS attacks such as HTTPAttack, APIRequests, and SYNFlooding effectively, though it faces some slight difficulties in detecting both YoYo and Slowloris attacks.
• The proposed scheme needs only a very short time to detect a new attack.
• R-EDoS consumes a very low amount of CPU and memory.
• Using this defense scheme can save up 30% to 50% of the total cost that a cloud user is forced to pay for the unexpected cost coming from EDoS attacks.

VII. CONCLUSION AND FUTURE WORK
In this work, we propose a novel approach to deal with stealthy EDoS attacks known as a multivariate time-series anomaly detection system. This approach can apply to different network systems and adapts well. Second, we propose an intelligent, robust scheme called R-EDoS, which can process and extract valuable information from high-dimensional time-series data. This lays the foundations for an intelligent model to detect anomalous network traffic behaviors occurring in EDoS attacks easily. Our work not only can help cloud users to avoid paying for various EDoS attacks such as slow HTTP/SYN flooding, database API requests, Yo-Yo, and Slowloris, but also can allow cloud service provider to improve their services. In future research, we not only plan to enhance the proposed scheme but also plan to draw a detailed comparison with other EDoS defense systems, using more evaluation criteria for particular EDoS attacks.