A Service Function Chain Deployment Method Based on Network Flow Theory for Load Balance in Operator Networks

,


I. INTRODUCTION
The original intention of network function virtualization technology (NFV) is to realize network functions by means of ''general hardware+software'', reduce the capital expenditure and operating expense of networks, and improve the flexibility of service deployment. The NFV has gradually become a key technology to promote the development of 5G and future network [1]- [3]. The use of a low-cost high-performance generic computing platform in network infrastructure can greatly reduce the cost of operators. Network functions can be virtualized into instances of plain software referred to virtual network function (VNF) in the NFV, which can greatly shorten the deployment cycle and The associate editor coordinating the review of this manuscript and approving it for publication was Yue Zhang .
improve the deployment flexibility and elasticity of network service [4], [5].
As a kind of new developing network technology, softwaredefined network (SDN) has been widely applied in recent years. Its characteristics, such as separating the control plane from the data plane and centralized control based on the global perspective, significantly improve the flexibility of network management and utilization of substrate resources [6]. The SDN and NFV are highly complementary: SDN can further simplify the NFV deployment and reduce the burden of operation and maintenance. On the other hand, NFV can provide SDN with infrastructure support. The combination of SDN and NFV technology can significantly improve the efficiency, programmability and flexibility of networks, which is widely used in 5G networks [7], [8]. VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ While NFV technology has many advantages, it also faces some challenges in the deployment process. In operator networks, different types of service flows are required to be served by different types of VNFs, and service function chain (SFC) is formed by these VNFs connected in a predefined order. In SFC deployment, the bandwidth resources, forwarding resources and computing resources of the substrate network should be occupied simultaneously. The SFC deployment is also an NP-hard problem like the virtual network mapping [9], [10]. Therefore, deciding how to design an efficient and reasonable SFC deployment strategy is particularly important. It is related not only to the revenue of operators, but also the quality of service (QoS) provided by operators. This demand makes the SFC deployment a hot spot in the field of industry and academia.
With respect to the issue of the SFC deployment, many researchers have proposed solutions. To deal with the load imbalance phenomenon in SFC deployment, the authors in [11] design two deployment methods to balance the load of the substrate network, and achieve good results. However, these methods only aim at minimizing the forwarding cost, and the consumption of substrate resource has not been fully considered. The work in [12] aims to improve the overall performance of the SFC deployment and uses a discrete particle swarm optimization algorithm to conduct the SFC deployment. In [13], a linear programming formulation is established with the goal of minimizing the resource scheduling delay, and a heuristic algorithm is designed based on a greedy algorithm to determine the resource scheduling method. In [14], the SFC deployment in 5G networks with the goal of resource optimization and QoS demand is studied. Sun et al. [15] propose a reliability-aware approach for SFC deployment and improve the resource utilization of the substrate network. The problem of resource orchestration in 5G networks with the goal of QoS and efficient utilization of network resources is studied in [16]. The authors in [17] propose an SFC deployment method to ensure efficient resource utilization and low delay. Wang et al. [18] study the SFC deployment in internet of things, and propose a new deployment method that can perceive resources and service in real time. In [26], the SFC deployment is studied and a novel algorithm is proposed to optimize the operating expense. The simulation results show that this method can significantly reduce service cost. Kim et al. [27] propose an SFC deployment strategy to improve QoS. However, unilaterally emphasizing QoS would affect the revenue of operators.
Most of the studies only optimize the single performance of SFC deployment, but they have not fully considered the load balance, QoS and reasonable utilization of the substrate resource. This work proposes an SFC deployment method based on the network flow theory (SFCD-NFT) that not only guarantees the load balance of the substrate network, but also considers the QoS and the revenue of operators. First, the global perspective of SDN controller is used to perceive substrate resources in real time, and a candidate node set is determined according to the resource demand of SFC and the locations of ingress/egress switch nodes. Second, the candidate node set is used to form a directed network, and the SFC deployment problem is transformed into a task of finding the optimal path in the directed network. Then a node disassembling method is used to construct a capacityflow-cost network based on the obtained directed network. Finally, taking the link transmission delay as the cost and the SFC bandwidth demand as the constraint, a minimumcost maximum-flow algorithm of network flow theory is used to find the optimal deployment path and complete the SFC deployment.
This work gives full consideration to VNF combination and eliminates bottleneck nodes in the candidate node selection stage. It chooses the substrate network path that has the least bottleneck links and bottleneck forwarding nodes as the optimal deployment path, and therefore guarantees the load balance of the substrate network, greatly reduces the link transmission delay and improves the performance of SFC deployment. The experiments show that the method can guarantee the load balance of the substrate network, and improve the overall QoS and resource utilization.
The main contributions of this paper can be summarized as follows.
(i) The SFC deployment problem is transformed into an optimal path selection problem. According to the locations of ingress/egress switch nodes and the resource demand of VNF, a candidate server node set of VNF is determined. Then the candidate server node set, the ingress/egress switch nodes and the connection between them are used to form a directed network, and thus the SFC deployment problem is transformed into a problem of finding the optimal path in the directed network. VNF combination is fully considered and bottleneck nodes are eliminated in the candidate node selection stage. It is beneficial to the load balance of the substrate network.
(ii) The obtained directed network is converted into a capacity-flow-cost network by a node disassembling method, and a minimum-cost maximum-flow algorithm is used to find the optimal deployment path of SFC in the capacityflow-cost network. The path that has the least bottleneck nodes/links and required transmission delay is selected as the optimal deployment path. Taking the link transmission delay as the cost and using a minimum-cost maximum-flow algorithm to solve the optimal path selection problem can make the result closer to the optimal solution, reduce the average transmission delay of service flow and improve the resource utilization of the substrate network.
The rest of this paper is organized as follows. In Section II, some related works are reviewed. Section III gives the problem statement. In Section IV, a network model and evaluation indicators are provided. Section V illustrates the integer linear programming formulation of the SFC deployment. The SFCD-NFT is elaborated in Section VI. Section VII validates and evaluates the proposed method with extensive simulations and experiments. We conclude this paper in Section VIII.

II. RELATED WORK
The tight coupling of infrastructure hardware and network service in the traditional network leads to high capital expenditure and operating expense. To solve this problem, some researchers have tried to use NFV to decouple infrastructure hardware and network service, which can reduce costs of the service deployment and network operation and improve the flexibility of service deployment. In the NFV technology, an SFC is formed by multiple VNFs connected in a predefined order to provide users with various end-to-end network services. The SFC deployment problem has become a hot spot in recent years. As the stuides in [2], [7], [8] have made a relatively detailed overview of SFC deployment, this section simply summarizes the SFC deployment problem.
The work in [19] proposes an SFC deployment strategy with the goal of improving resource utilization, but the scenario setting is relatively simple. The work in [20] proposes a heuristic algorithm to solve the deployment problem of SFC with the goal of minimizing the overall delay, and achieves good effect, but the objective function of this method is incomplete. A heuristic algorithm is proposed in [21] for dynamic SFC deployment, which balances the minimum resource consumption and the minimum resource scheduling operation. In [22], a linear programming formulation with reliability perception and delay constraint is established to maximize the service reliability and minimize the delay of SFC. An optimal approximation algorithm is proposed in [23] to solve the problem of SFC deployment, which improves the reliability of SFC and reduces the deployment cost. However, this method cannot guarantee the deploying success ratio.
The emergence of many lightweight virtualization technologies, such as docker and linux container technologies, makes it possible for deploying multiple VNFs simultaneously on the same high-performance server. The work in [24] studies the SFC deployment in 5G networks, and proposes a method of VNF combination and mapping temporary link to reduce network resource consumption. Wang et al. [25] propose a method to conduct joint-optimization of SFC and resource allocation, and discuss the influence of virtual machine reusing and VNF combination on the performance of SFC deployment. Mechtri et al. [28] propose an SFC deployment method that aims at maximizing the revenue of network operators and satisfying the service demands of users. An optimal deployment method of SFC is proposed in [29], which is scalable to network topology. However, it does not take into account the reasonable using of substrate resources, and the overall deployment performance still has room for improvement. Sun et al. [30] propose a backtracking mechanism to solve the initial failure of SFC deployment, thus significantly improving SFC deploying success ratio.
The work in [31] proposes a heuristic algorithm that first builds a multicast routing tree between the source node and the destination nodes, and then places the VNFs along the built routing tree. He et al. [32] studies the optimal placement of VNFs with multiple instances to minimize cost, as well as guarantee the load balance. The work in [33] proposes an efficient deployment algorithm for online SFC requests, and the algorithm is expected to guarantee the load balance and improve the deploying success ratio. These studies allow for one-to-many and many-to-one VNF embeddings to improve the load balance. However, the splitting of service flow will cause performance degradation.
The above studies mostly pursue single performance, failing to fully consider the overall performance of SFC deployment, such as load balance, QoS and the reasonable use of substrate resources. This is also the motivation of this paper.

III. PROBLEM STATEMENT
Existing SFC deployment strategies mostly pursue single performance, leading to the unreasonable use of the substrate network resources, mainly including: (i) The use of the substrate network resources is unbalanced and there are some bottleneck nodes and links in the substrate network.
(ii) The unreasonable use of the substrate network resources may also lead to the problems of increased transmission delay and reduced revenue. This paper studies the reasonable using of the substrate network resources and performance optimization of the SFC deployment from the perspective of operators.

A. VNF COMBINATION STRATEGY
Lightweight virtualization technologies make it possible to deploy multiple VNFs simultaneously on the same high-performance server. Therefore, multiple adjacent VNFs of an SFC can be deployed on the same server node under the condition of meeting resource constraints and VNF types constraints, called VNF combination. The VNF combination mechanism can effectively improve the utilization of substrate network resources, reduce link transmission delay and improve SFC deployment performance, as shown in Figure 1. Assume that the number of VNF types that the server node can host is m. Then the adjacent VNFs whose number is m can be deployed in the same server node on the basis of meeting the VNF type constraint and resource constraint. Otherwise, the VNF combination is not allowed to avoid a Ping-pong VOLUME 8, 2020 routing problem. For example, if m = 2, the i-th VNF and the (i − 1)-th VNF can be deployed on the same server node, and the i-th VNF cannot be combined with other VNFs except the (i − 1)-th and (i + 1)-th VNF. Otherwise, it is easy to generate the Ping-pong routing problem, as shown in Figure 2.

B. SFC DEPLOYMENT STRATEGY
From the perspective of operators, SFC deployment should ensure the low delay demands of 5G services, enhance the utilization of substrate resources as much as possible and reduce resource fragmentation, so as to improve the revenue of operators.
In this paper, the SFC deployment problem is transformed into an optimal path selection problem that is solved by the network flow theory. First, the candidate server node set of VNFs is determined according to the locations of the ingress/egress switch nodes and the resource demand of SFC. Then the ingress/egress switch nodes, the candidate server node set and the connection between them are used to form a directed network, and the SFC deployment problem is transformed into an optimal path selection problem. Next, the directed network is transformed into a capacity-trafficcost network by a node disassembling method. Finally, link transmission delay is taken as the cost of optimal path selection, and the bandwidth demand of service flow is taken as the constraint, the path that has the lower delay and the least bottleneck nodes/links is selected as the optimal deployment path by a minimum-cost maximum-flow algorithm.
Due to the full consideration of the VNF combination and bottleneck nodes/links in the process of optimal path selection, the load balance of the substrate network is improved and the overall performance of SFC deployment can be guaranteed.

IV. NETWORK MODEL AND EVLUATION INDICATORS
A. NETWORK MODEL 1) SFC REQUEST An SFC request can be represented by a weighted undirected For simplicity, we assume service traffic comes from one single ingress switch node and ends at the single egress switch node, S and T represent the ingress switch node and egress switch node, respectively.
. . , v P } represents the set of VNFs that are arranged in a predefined order. When the VNFs are deployed on server nodes, the computing resources will be consumed. E v represents the virtual link set of SFC, e ij v ∈ E v represents the virtual link between v i and v j , and bw(e ij v ) represents the bandwidth demand of the virtual link e ij v . When a service flow traverses through switch nodes, it needs to consume the forwarding resources of switch nodes. When an SFC is deployed, it needs to occupy the computing and bandwidth resources of a substrate network. d v represents the delay constraint of the SFC. D d is defined as the actual transmission delay of the SFC that is successfully deployed, and it should be less than or equal to d v , as shown in Figure 3(a).

2) SUBSTRATE NETWORK
This paper selects operator networks as the application scenario, which is composed of a core network and a number of servers (or data centers) connected to it, as shown in Figure 3 . , n n f } represents the switch nodes with fixed forwarding capacity, and N c = {n 1 c , n 2 c , . . . , n n c } represents the server nodes with fixed computing resources and types of VNFs that can be hosted. Due to the high-speed fiber connection between a pair of switch node and server node, the link transmission delay between them is negligible. In this paper, E s is used to represent the set of links between the switch nodes, and e lm s ∈ E s represents the substrate inter-switch link between the n l f and n m f . D s represents the link transmission delay set of E s , and d lm s ∈ D s represents the link transmission delay of the e lm s .

3) SFC DEPLOYMENT
When an SFC request arrives, the operator instantiates the VNFs according to the predefined order that the service flow traverses through the VNFs. SFC deployment needs to consume the computing resources, forwarding resources and bandwidth resources of the substrate network, as shown in Figure 3(c). In order to prevent performance degradation caused by the splitting of service flow, this paper stipulates that a flow cannot be split into more than two paths.

B. EVALUATION INDICATORS 1) DEPLOYING SUCCESS RATIO
Deploying success ratio is one of the main indicators to describe the deployment performance and overall resource utilization of SFC. It can be defined as follows: where SFC(t) is the number of SFC requests at time t, SFC suc (t) is the number of SFC requests successfully deployed at time t, T d is the total operating time of operators, and δ is a constant that is infinitely close to 0.

2) LONG-TERM AVERAGE REVENUE TO COST RATIO
as follows: where cpu(v i ) represents the required CPU resources of Usually, the long-term average revenue to cost ratio is used to represent the performance of SFC deployment method under steady state, and it can be defined as follows:

3) LINK LENGTH EXPANSION COEFFICIENT
In this paper, the link length expansion coefficient is defined as the expanded proportion of the total substrate path hops that virtual links deployed compared to the sum hops of SFC that are successfully deployed. Denoted by K 1 , the link length expansion coefficient reflects the effective utilization of substrate link resources in SFC deployment, as defined as follows: ) is the sum of substrate path hops deployed by virtual link e ij v , and L v is the sum of SFC hops deployed successfully.

4) AVERAGE TRANSMISSION DELAY
The transmission delay of SFC is the time that it takes for packets to travel from an ingress switch node through the VNFs to an egress switch node. It includes queueing delay, processing delay, and link transmission delay, and it shall be less than or equal to the end to end delay demand [34]. In the scenario where flow splitting is not allowed, the processing delay and queueing delay are mainly related to the hardware performance and packet size. However, the link transmission delay is directly affected by the performance of SFC deployment algorithm. This paper defines the average transmission delay as the average value of link transmission delay of successfully deployed SFCs. It denoted as K 2 can reflect the performance of SFC deployment, as defined as follows: where D p d represents the actual transmission delay of the p-th successfully deployed SFC.

5) PROPORTION OF BOTTLENECK NODES/LINKS
The substrate switch nodes and substrate inter-switch links with load over 95% are defined as bottleneck nodes and links, respectively. The proportion of bottleneck nodes and links can reflect the utilization of substrate resources, and they can be described as follows.
where K 3 is the proportion of bottleneck links and K 4 is the proportion of bottleneck nodes. L linkload−0.95 is the total number of bottleneck links, and L s is the number of all substrate inter-switch links. N nodeload−0.95 is the number of bottleneck nodes, and F 1 is the number of all substrate switch nodes.

6) PROPORTION OF IDLE NODES/LINKS
The substrate switch node and links with resource utilization less than 5% are defined as idle nodes and links, respectively. The proportion of idle nodes and links can reflect the utilization of substrate resources, and they can be described as VOLUME 8, 2020 follows.
where K 5 is the proportion of idle links, K 6 is the proportion of idle nodes, L linkload−0.05 is the number of idle links, and N nodeload−0.05 is the number of idle nodes.

V. INTEGER LINEAR PROGRAMMING FORMULATIONS
The integer linear programming formulation of SFC deployment is formulated and the objective function and constraints are given in this section. This paper aims at minimizing the average transmission delay of SFC in operator networks on the premise of ensuring a balanced utilization of substrate network resources. The objective function is as follows.
where N represents the number of all successfully deployed SFCs at time T d .
Constraint (12) indicates that if the VNF v i is deployed to the server node n l c , x l i = 1. Otherwise, x l i = 0. Constraint (13) denotes that if the virtual link e ij v is deployed to the substrate link e lm s , y lm ij = 1. Otherwise, y lm Constraint (14) indicates that the number of VNF types hosted by the server node are m. Constraint (15) means that each VNF of an SFC can only be deployed to one server node. Constraint (16) denotes that the residual CPU computing resources of a server node cannot be less than the CPU resource demand of VNF. Constraint (17) represents that the residual bandwidth resources of a substrate network link should be larger than the link resource demand of an SFC. Constraint (18) shows that the residual forwarding capacity of a substrate switch node should be greater than the forwarding resource demand of a service flow to be deployed, and forward(n l f ) represents the residual forwarding capacity of the substrate switch node n l f . The constraint (19) indicates that the VNF type demand must be satisfied by the server node, and f i represents the type demand of v i .
Constraint (20) means that the virtual link should be deployed to the loopfree path of the substrate network to prevent the Ping-pong routing problem. Constraint (21) indicates that SFCs deployed successfully must meet the delay demands of service flows, and d p v represents the delay constraint of the p-th successfully deployed SFC.
This integer linear programming formulation is known to be an NP-hard problem, and therefore the SFCD-NFT is proposed to solve it.

VI. SFCD-NFT
In order to improve the SFC deployment performance and guarantee the load balance of the substrate network, this paper proposes the SFCD-NFT that first transforms an SFC deployment problem into an optimal path selection problem and then uses a minimum-cost maximum-flow algorithm to solve it.

A. THE PROCESS OF SFCD-NFT
As shown in Figure 4, first, a candidate node set is obtained according to the locations of the ingress/egress switch nodes and resource demands of VNFs. The candidate node set, the ingress/egress switch nodes and the connections between them are used to form a directed network, and thus the SFC deployment problem is transformed into an optimal path selection problem. Then, the obtained directed network is transformed into a capacity-traffic-cost network by a node disassembling method. Finally, a minimum-cost maximumflow algorithm is used to find the maximum flow path with the minimum cost in the capacity-traffic-cost network, and the result is employed to complete the SFC deployment.
Due to the full consideration of resource utilization and VNF combination in the SFC deployment, the load of the substrate network is more balanced and the SFC deployment path length is significantly reduced.

B. DETERMINE A CANDIDATE NODE SET
The candidate node set of VNFs can be determined by the following factors: • the locations of the ingress/egress switch nodes. • the given order of the VNFs in an SFC. • the length L of an SFC request that is equal to the hops between the first VNF and the last VNF.
• the length R of the shortest path between an ingress switch node and an egress switch node. In order to prevent the QoS problem caused by an excessive long deployment path, the deployment locations of VNFs should be restricted. The load parameter N load of a substrate node is introduced to monitor the load of server nodes and the switch nodes in real time through an SDN controller. Define the hops between the ingress switch node and the l-th server node n l c as I (n l c ), and the hops between the egress switch node and the l-th server node n l c as O(n l c ). For the i-th VNF v i , the server nodes that meet the following constraints should be selected as the candidate nodes: 1 cpu(v i ) represents the computing resource demand of v i , and cpu(n l c ) represents the residual computing resource of the l-th server node n l c ; 3. f i ∈ C(n l c ); C(n l c ) represents the service types that the n l c can host. This means that the type of v i is contained in the types of VNF that can be hosted by the server node n l c ; 4. N load (n l c ) < 0.95 & N load (n l f ) < 0.95; N load (n l c ) is the real-time load of the server node n l c , and N load (n l f ) is the real-time load of the switch node n l f . This means that the candidate server node n l c and the switch node n l f connected to n l c are not bottleneck nodes. The specific process of determining a candidate node set is shown in Algorithm 1.

Algorithm 1 Determine a Candidate Node Set
Input: the real-time resource and topology of the substrate network, L, R Output: the candidate node set N LIST (1) an SDN controller perceive the resource and topology of the substrate network in real time (2) n l c is an eligible server node, and put it in the candidate set of v i ; (6) return a candidate set n(i) LIST of v i (7) end if (8) end for (9) end for (10) N LIST = {n(1) LIST ; n(2) LIST ; ... ; n(q) LIST } According to the result of Algorithm 1, a candidate node set of VNFs can be obtained. Use the candidate server node set N LIST , the ingress/switch nodes and the inter-switch links between them to form a directed network G d , as shown in Figure 5. The SFC deployment problem is converted to an optimal path selection problem of finding a path from the ingress switch node through the candidate node to the egress FIGURE 5. The example of directed network Gd. VOLUME 8, 2020 switch node. This path must pass all the candidate node set of VNFs, and each candidate node set can be passed only once. The overlapping nodes in the candidate node set can be used as VNF combination deployment nodes to reduce the link delay.

C. CONSTRUCT A CAPACITY-FLOW-COST NETWORK
In order to find the optimal deployment path in the directed network G d , it is necessary to convert the directed network G d into a capacity-flow-cost network by using a node disassembling method, as shown in Figure 6. First, nodes of G d are numbered, each node Al is disassembled into a node pair Al and Al , and an edge connects Al to Al . Then, the capacity and the flow of the edge are defined to be both equal to 1, and the cost between a node pair is set 0 (equivalent to the cost from oneself to oneself). If node Al is directly connected to Am, Al is connected to Am by an edge. Finally, the link delay d lm s between switch nodes of the substrate network is used as the cost, and the directed network G d is thus converted to a capacity-flow-cost network G c , as shown in Figure 6. The specific process of node disassembling method is shown in Algorithm 2.

Algorithm 2 Node Disassembling Method
Input: A directed network G d Output: A capacity-flow-cost network G c (1) number the nodes of G d (2) disassemble the node Al into a node pair Al and Al (3) use an edge to connect the node Al and node Al (4) define the capacity and flow of an edge to be both equal to 1, the cost between a node pair Al and Al is 0 (5) if node Al is directly connected to Am (6) Al is connected to Am by an edge (8) use d lm s as the cost (9) end if (10) return the capacity-flow-cost network G c

D. MINIMUM-COST MAXIMUM FLOW ALGORITHM
Using the link transmission delay of the substrate network as the cost, a minimum-cost maximum-flow algorithm is adopted to solve the problem of the SFC deployment, which can ensure that the deployment path has the minimum transmission delay and maximum flow, and guarantee the performance of the SFC deployment. In order to state the problem clearly, the concept of residual network is introduced. Each capacity-flow-cost network G c and a flow on it correspond to a residual network R(f) whose nodes are the same as the network G c . The residual network reflects the residual link capacity, as shown in Figure 7. The specific process of minimum-cost maximum-flow algorithm is shown in Algorithm 3.
Finally, the SFC deployment is completed after occupying the computing resource, forwarding resource and link bandwidth resource on the optimal deployment path.

E. COMPLEXITY ANALYSIS
For each SFC flow, the time complexity of SFCD-NFT includes the time complexity of determining a candidate node set, the node disassembling method and the minimum-cost maximum-flow algorithm. F 2 is the number of all VNFs in an SFC request, and the complexity of determining a candidate node set is at most O(F 1 * F 2 . The time complexity of the node disassembling method is O(F 1 ). The complexity of the minimum-cost maximum-flow algorithm is O(L s * logF 1 ). Thus, the total time complexity of SFCD-NFT at the worst situation is O(L s * logF 1 + F 1 * F 2 + F 1 ). Comparing with other SFC deployment algorithms, the deployment method proposed in this paper has lower time complexity.

VII. SIMULATION
In this paper, MATLAB is used for simulation and the SFCD-NFT proposed in this paper is evaluated in a large-scale network scenario and compared with three other algorithms.

A. SIMULATION ENVIRONMENT
The substrate network topology and SFC topology used in the simulation experiments are generated by the improved Salam network topology random generation algorithm. Set the substrate switch node and the server node to be deployed in the Algorithm 3 Minimum-Cost Maximum-Flow Algorithm Input: An SFC request G v , a capacity-flow-cost network G c Output: the optimal deployment path ODpath (1) k = 0, the minimum-cost maximum-flow path f k = 0(minimum cost flow with zero traffic), and the set of minimum-cost maximum-flow path MCMFpathlist = 0 (2) for i = 1: k (3) construct the residual network R(f k ) (4) the link bandwidth demand is used as the constraint, the link transmission delay as the cost, and the depth-first search algorithm is applied to find the minimum cost path from the ingress switch node to the switch egress node in R(f k ) (5) if the minimum cost path P is nonexistent (6) f k is path of the maximum flow with the minimum cost, end the calculation (7) put the f k into the MCMFpathlist (8) else (9) update f k to f k + P, update the R(f k ) (10) end if (11) end for (12) return MCMFpathlist (13) if MCMFpathlist is empty or MCMFpathlist cannot meet the delay constraint (14) return deploying failed (15) else (16) choose the path with the least bottleneck nodes/links as the optimal deployment path (17) end if (18) return optimal deployment path ODpath (19) if ODpath meet SFC computing resource constraint (20) return deploying success (21) else (22) return deploying failed (23) end if same location of operator networks, their numbers are both 100, and the link connection probability between the switch nodes is 0.5. The computing resources of server nodes and the forwarding resources of switch nodes obey the uniform distribution of [50]-[60], and the link bandwidth between the substrate network switch nodes obeys the uniform distribution of [20]- [25]. The link transmission delay of the substrate network obeys the uniform distribution of [1], [2]. It is assumed that each server node can host any two types of The ingress switch node and egress switch node of service flow are randomly determined according to an SFC request. The number of VNFs contained in an SFC request obeys the uniform distribution of [2], [4], and the types of VNFs are different. The computing resource demands of VNFs obey the uniform distribution of [7], [10], and the bandwidth demands of SFCs obey the uniform distribution of [5], [8]. The maximum allowable transmission delay of SFC obeys the uniform distribution of [6], [8]. The arrival ratio of SFC requests obeys the Poisson distribution with the parameter 0.1 and the life time obeys the exponential distribution with parameter 1500.
The duration of simulation experiments is 10,000 time units. In order to eliminate the effect of random factors on the experimental results, the simulation experiment is conducted for 10 times, and its average value is taken as the final simulation result.

B. EXPERIMENTAL ANALYSIS
Two sets of experiments are set up to verify the SFCD-NFT proposed in this paper. In the first set of experiments, the SFCD-NFT is compared with three latest research methods under the same experimental conditions, as shown in Table 1. The second set of experiments analyzes the performance of the SFCD-NFT in the utilization of the substrate network resources.

1) EXPERIMENT 1: COMPARISION OF SFC DEPLOYMENT METHODS
This set of experiments mainly compares and verifies the performance of the SFCD-NFT from four evaluation indicators: deploying success ratio, long-term average revenue to cost ratio, average transmission delay and average link expansion coefficient.
As shown in Figures 8 and 9, the SFCM-FOCL utilizes the virtual machine reusing strategy, VNF combination strategy and temporary link mapping strategy to complete the SFC deployment. The deploying success ratio remains at 0.5 and the long-term average revenue to cost ratio at 0.55. The JoraNFV makes the joint-optimization for the three stages  of NFV resource allocation with the goal of optimizing resource utilization. The deploying success ratio remains above 0.5 and the long-term average revenue to cost ratio above 0.6, which are slightly higher compared with the SFCM-FOCL. The NFV-CG uses the decomposition method to establish an exact mathematical model of the SFC deployment, and solves it by the column generation algorithm. Due to the excellent performance of column generation algorithm in solving large-scale linear programming problem, the NFV-CG obtains relatively good results, the deploying success ratio and the long-term average revenue to cost ratio remain at 0.67 and 0.7, respectively.
The above three methods all aim at improving resource utilization and achieve good results. However, there is still room for improvement. The three methods are prone to cause the excessive resource consumption and resource fragmentation, because the reasonable utilization of the substrate network resources is not fully considered. In contrast, the SFCD-NFT proposed in this paper takes into account the balanced utilization of the substrate network resources. The deploying success ratio is maintained above 0.7 and the long-term average revenue to cost ratio is maintained around 0.72. Overall, the SFCD-NFT has the best performance in the four SFC deployment methods.
As shown in Figures 10 and 11, the average transmission delay and average link expansion coefficient of the four  methods rise with the increased SFC request arrival ratio steadily. This is because the average length of deployment path and the transmission delay of SFC increase with the consumption of the substrate network resources. However, when the arrival ratio λ is equal to 0.1, 0.15 and 0.2, the average transmission delays of the SFCD-NFT are equal to 5.41, 5.57, 5.78 and the average link expansion coefficients are equal to 0.45, 0.49, 0.59, respectively. The average transmission delay and average link expansion coefficient of the SFCD-NFT are lower than the three other methods. With the increase of the arrival ratio, the variations of the average transmission delay and average link expansion coefficient are not large. These verify the performance of the SFCD-NFT in terms of the average transmission delay and the average link expansion coefficient.

2) EXPERIMENT 2: ANALYSIS OF THE SUBSTRATE NETWORK RESOURCE UTILIZATION
This set of experiments mainly analyzes the substrate network resource utilization of the SFCD-NFT, and compares it with the three other methods from four evaluation indicators: proportions of bottleneck nodes and idle nodes, proportions of bottleneck links and idle links.
As seen from Figures 12 and 13, because the JoraNFV and SFCM-FOCL fail to fully consider the node real-time load of the substrate network in the SFC deployment, the proportions of bottleneck nodes and idle nodes are both high,  and grow rapidly over time, leading to the unreasonable utilization of the substrate network resources. The NFV-CG algorithm uses a decomposition method to establish an exact mathematical model of the SFC deployment and solves it by the column generation algorithm. Due to the superior performance of the column generation algorithm in solving large-scale linear programming problems, the proportions of bottleneck nodes and idle nodes are significantly lower than that of the JoraNFV and the SFCM-FOCL. The SFCD-NFT considers the resource utilization in the node selection stage and chooses the nodes with lower load as the candidate nodes. Therefore, the proportions of bottleneck nodes and idle nodes are minimal, and the node load of the substrate network is relatively balanced.
As shown in Figures 14 and 15, because the JoraNFV and the SFCM-FOCL fail to give full consideration to the link real-time load in the SFC deployment, the proportions of bottleneck links and idle links are higher, and the proportions of bottleneck links grow quickly with the consuming of substrate link resources. The utilization of substrate link resources is not reasonable. Due to the advantages of an exact mathematical model and the performance of the column generation algorithm in solving large-scale linear programming problems, the bottleneck links proportion and idle links proportion of the NFV-CG are both lower than that of the JoraNFV and the SFCM-FOCL. The SFCD-NFT takes full account of the resource utilization and eliminates  the bottleneck nodes/links in the link selection. Therefore, the proportions of bottleneck links and idle links are minimal and the link load of the substrate network is more balanced. This experiment verifies the performance of the SFCD-NFT.
According to the above two sets of experiments, the SFCD-NFT proposed in this paper maintains better performance compared with the three other methods. This verifies the superiority of the SFCD-NFT.

VIII. CONCLUSION
In this paper, the SFCD-NFT is proposed to meet the demand of load balance, low delay and efficient utilization of substrate resources in operator networks. First, a candidate node set and a directed network are determined, and the SFC deployment problem is transformed into an optimal path selection problem. Then, a node disassembling method is used to transform the obtained directed network into a capacity-flowcost network. Finally, a minimum-cost maximum-flow algorithm is used to solve the optimal path selection problem and complete the SFC deployment. Experiments show that the SFCD-NFT can guarantee the load balance of the substrate network and the reasonable using of the substrate network resources, reduce the average transmission delay of service flow, and thus improve the revenue of operators. We will study the SFC deployment in a more realistic scenario and try to use more practical simulation platforms to improve the credibility of experimental results in the future work. ZHENHUA YU received the B.S. and M.S. degrees from Xidian University, Xi'an, China, in 1999 and 2003, respectively, and the Ph.D. degree from Xi'an Jiao Tong University, Xi'an, in 2006. He is currently a Professor with the College of Computer Science and Technology, Institute of Systems Security and Control, Xi'an University of Science and Technology, Xi'an. He has authored more than 20 technical papers in conferences and journals, and holds two invention patents. His research interests mainly focus on cyber-physical systems, system security, and social networks. YU ZHAO received the B.S., M.S., and Ph.D. degrees from Air Force Engineering University, Xi'an, China, in 2009, 2011, and 2015, respectively. He is currently a Visiting Scholar with Air Force Engineering University. His research interests include cyber-physical systems and system security. VOLUME 8, 2020