Energy-Aware Virtual Network Migration for Internet of Things Over Fiber Wireless Broadband Access Network

The virtualized fiber wireless broadband access networks (V-FiWi) paradigm, effectively embedding heterogeneous virtual networks (VNs) originated from the service provider (SP) into shared substrate network (SN) provided by infrastructure provider (InP), plays a tremendous role in meeting the differentiated requirements between wireless frontend subnetwork and fiber backhaul subnetwork to achieve the interoperability of heterogeneous resource allocation. However, most of the existing V-FiWi integration systems mainly focused on the virtual network request (VNR) acceptance ratio, InP revenue, and substrate resource utilization, and they ignored the crucial issue called higher energy consumption cost, which was resulted from the imbalanced consumption of the substrate resource between the arrival and departure of VNR. In this article, we devote to exploring the energy-aware virtual network migration (EA-VNM) problem over the FiWi access technology, aiming to reoptimize the energy consumption while maintaining the high InP revenue and the large substrate resource utilization. In response to this issue, we first represent the service-oriented V-FiWi broadband access network architecture from the perspective of computing, storage, and network resource constraints, in which a migration model consisting of migration node and migration time is explained in detail. Then, we propose an enhanced KM-based energy-aware node migration (EKM-ENM) algorithm to economize on more bandwidth resource. More specially, the EA-VNM technology consists of network topology attributes and global network resources-based node-ranking measurement (NRM) phase, maximum weight matching-based node migration phase, and energy-aware link migration phase via Dijkstra shortest path algorithm. Finally, a rather large number of simulations are analyzed and evaluated numerically. Simulation results suggest that the proposed EKM-ENM algorithm outperforms the traditional embedding algorithms in terms of saving energy cost, decreasing time complexity, and improving VNR acceptance ratio.


I. INTRODUCTION
R ECENTLY, the prominent prevalence of blockchain, cloud computing, artificial intelligence (AI), big data, Internet of Things (IoT), and the rapid development of 5G communication technology jointly facilitated the backbone transmission networks and the converged wireless optical networks toward the green era of higher capacity, larger connection, and lower latency [1], [2]. A promising fiber wireless access network (FiWi) paradigm, such as the integration of Ethernet passive optical network (EPON) and IEEE 802. 16 broadband access network [3], has been proposed to moderate the access bottleneck of last mile in 5G and beyond communication networks because of the complementary advantages of wireless frontend subnetwork characterized by mobility, ubiquity, flexibility, and easy installation and optical backhaul subnetwork with stability, reliability, high bandwidth, and longer arrival, and has further fetched extensive attentions in the third industrial revolution (TIR) economy, e.g., smart grid [4], smart cities [5], tactile Internet [6], Cloud of Things [7], and Internet of Vehicle (IoV) [8]. Meanwhile, both the technological breakthroughs in regular traffic for the human-type communication (HTC) and the service provisioning of emerging IoT traffic in the current Internet have further promoted the architecture evolution of the FiWi access network, such as FiWi-enhanced LTE-A HetNets, IoT over FiWi access network, sensors-enhanced FiWi access networks, and machine-to-machine communications over FiWi access networks, which not only can manage the resource-efficient IoT service provisioning but also can enhance the resource utilization.
In the collaborative allocation of computing, network, and storage resources, the network virtualization (NV), designed to satisfy the important requirements, has been one of the most promising paradigms to resolve the network solidification problems and the inefficient utilization of resource, This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ which are caused by the significant differences of network topology architecture, communication protocol formats, and heterogeneous resource allocation in the converged FiWi access network, especially for the real-time applications or live events [9]. In this paradigm, the virtual network embedding (VNE) technology provides the multiheterogeneous virtual networks (VNs) with the single-domain, underlying, and shared substrate network (SN) to achieve the resource allocation for the NV. Leveraging on this precondition, Internet service provider (ISP) can be further categorized into infrastructure provider (InP) with global network resource and service provider (SP) with virtual resource demand. Among them, InP leases the global network resource, e.g., node CPU, node storage, and link bandwidth, to the SP who satisfies the demand of VNE while charging a certain fee as revenue via exact, heuristic, and metaheuristic algorithms.
To fulfill the delay-sensitive requirements and applications in the 5G networks, the promising solution that deploys MEC server at network edge brings about the fundamental changes to the access network architecture, reducing the end-to-end service delay and providing high reliability. During this transformation of tree topology into a mesh architecture, a mobile frontend transmission architecture based on the virtual passive optical network (VPON) allows east-west communication to coexist with traditional north-south communication [10]. In such a case, an effective VNE algorithm, which can provide application requests with computing, network, and storage resource, is formulated to improve the VN request (VNR) acceptance ratio, resource utilization, and InP revenue of cloud data centers. However, there must exist more challenges in the ring optical data centers considering wavelength selection, continuous wavelength constraint, and physical path selection. Therefore, a promising integration of the optimized programming formulation, VNE algorithm, and efficient physical path and wavelength selection is envisioned to increase the acceptance ratio and revenue [11]. The development of industrial wireless network to the intelligent factory is facing the solidification problem of single application and single network architecture. However, it is worth noting that the existing VNE algorithms have no regard for QoS-compliant capacity. In terms of the suboptimal solution, an intelligent latency-aware VNE covering static embedding and dynamic forwarding is proposed to evaluate the load balancing and learning efficiency [12]. Given the important role of VNE technology in the wired and wireless access networks, the virtual resource embedding methods over the FiWi system have gained widespread attention from the communication and other economic sectors.
The prior arts on the virtual FiWi (V-FiWi) network embedding framework have been described to increase the InP revenue [13], balance the resource consumption [14], maximize the resource utilization [15], and manage the resource-efficient IoT service provisioning [16] from the viewpoints of QoS satisfaction and network reconfiguration, load balancing and priority, load balancing and QoS requirement, and Q-learningbased traffic prediction, respectively. However, most of the existing VNE algorithms ignored the energy consumption cost for accommodating more VNRs while maintaining substrate resource utilization and average InP revenue. In addition, the global network resources of the SN change enormously over the forthcoming VNRs arrival and departure, and it cannot provide enough CPU resource for the VNs, which leads to the low energy efficiency of VNE. Therefore, given the paramount effect of the energy-aware VN migration (EA-VNM) problem on the FiWi access network, an enhanced KM-based energyaware node migration (EKM-ENM) algorithm has been put forward to decrease the energy consumption of the shared substrate FiWi network. Although, during the migration process from optimal solution to suboptimal solution, there are still some untouched issues in the V-FiWi SN, which takes full account of the minimization strategy of energy consumption. Overall, the main research highlights of this article can be listed as follows.
1) With the development of the NV over the FiWi networks, many VNE-based technologies have been proposed to improve InP revenue, resource utilization, and service provisioning. However, the active VN migration (VNM) scheme over the FiWi access technology, to the best of our knowledge, is first put forward to diminish migration link bandwidth difference and optimize energy consumption cost. 2) To distinguish the existing VNE technology and the emerging VNM paradigm, we develop the VNMenabled FiWi access network architecture, covering the network model represented by weighted undirected graph, migration model in terms of migration node and migration time, key performance metrics, and problem formulation. 3) To resolve the EA-VNM problem over the FiWi access network, we develop an EKM-ENM algorithm to guarantee a lower energy consumption, which consists of network topology attributes and global network resource-based node-ranking measurement (NRM) stage to quantify the relative importance degree of each node in the V-FiWi, node migration stage based on the maximum weight matching, and link migration stage based on the Dijkstra shortest path selection. 4) To demonstrate the superiority of the proposed EKM-ENM algorithm to the existing arts in terms of substrate resource utilization, VNR acceptance ratio, and architecture energy consumption, extensive simulation experiments have been conducted. The comparison of the proposed EKM-ENM algorithm with the existing embedding algorithms, such as congestion-aware and energy-aware VN embedding (CEVNE) and energyaware VN embedding (EA-VNE) algorithms, shows that the energy consumption can be saved by 5.5% and 8.2%, respectively. The remainder of this article is organized as follows. Section II first reviews the related works about the FiWienhanced 5G networks, VNE technology, and VNM paradigm. After that, the EA-VNM problem formulation is introduced in Section III, from the viewpoints of network model, migration model, key performance metrics, and problem formulation. Then, Section IV puts forward the high-efficiency EKM-ENM algorithm to achieve the maximum weighted matching, and decrease energy consumption while maintaining the InP revenue and resource utilization. Finally, performance evaluation and results discussion in Section V are expounded before validating the conclusions in Section VI.

II. RELATED WORKS
In this section, we will review the related works from the point of view of the converged 5G-enhanced FiWi network, VNE technology in the fundamental category, and energyaware VNM paradigm.

A. Fiber Wireless Broadband Access Network
The seamless convergence and cooperation of fiber and wireless system has been designed to the various key usage scenarios for 5G and beyond networks, e.g., enhanced mobile broadband (eMBB), massive machine-type communication (mMTC), and ultrareliable and low latency communication (uRLLC). From a technological point of view, to investigate the flexible implementation and the key performance of emerging vertical services and eMBB scenario, 5G multiapplication over the enhanced FiWi systems, which consists of DSPbased flexible-waveform signal, 5G new radio (NR) signal, and an additional vector signal, is provided to evaluate the total of accessible throughput 4.41 Gb/s and the low-latency high-bandwidth M2M application [17]. Recently, the joint user association and power allocation raise more and more concern on the mmWave 5G NR transmission between the user end and remote radio unit (RRH). Besides, the mobile fronthaul of the 5G FiWi access network is heading toward a bidirectional multiband, multibeam mm-Wave beamformer [18], and a full field-of-view self-steering beamformer [19]. With the introduction of mmWave radio, 5G mmWave fronthaul for the next-generation FiWi access network has been designed to guarantee large data transmission and face low-latency challenges in order to make it applicable to ultradense hot-spot area and metropolitan area.
From an economic point of view, the architecture evolution of converged optical wireless networks compliances with the tendency varying from the coarse-grained functional split, e.g., RRH split baseband unit (RRU-SBBU) (4G) and RRU distributed unit centralized unit (RRU-DU-CU) (5G), to finegrained functional split, e.g., RRU fine-grained unit (RRU-FU) (5G and beyond network), achieving the prospect of high resource efficiency and cost effectiveness in the existing CRAN deployment, and mitigating the contradiction between transmission resource saving and centralization gain [20]. On the other hand, the crucial requirements caused by the extensive growth of wireless devices and traffic volume urge the next-generation optical access network (OAN) to pose the computing capacity onto network edge or fog node, which can bear the weight of latency-sensitive application and prolong the life span of resource-constrained devices [21]. The integration of green fog with next generation of long-reach OANs is evaluated via upstream decentralized or centralized bandwidth resource allocation. For the sake of the lower cost of development for radio access networks, the hybrid FiWi 5G fronthaul seamlessly coexists with the end-to-end 4G network on the same fronthaul infrastructure [22].
From the viewpoint of application, under the background of delay-sensitive and compute-intensive services, the convergence architecture of Tactile Internet with MEC services and computation offloading, envisioned in enhanced FiWi networks-based Heterogeneous cloud radio access network (H-CRAN), is proposed to analyze user association and resource allocation [1], in which the decentralized coordination based on a full dual decomposition is superior to the traditional association and resource algorithms, e.g., double-sided auction-based distributed resource allocation (DS-ADRA) [23], semidefinite relaxation with gaussian randomization [24], and interference-aware greedy heuristic algorithm (IA-GH) [25], in terms of decreasing the delay and minimizing the energy consumption, especially for the highload scenarios. However, to the best of our knowledge, the smart home data, the smart grid big data, and the big data in IoVs scenario heavily rely on the integrated FiWi networks to make decisions at the optical line terminal (OLT), especially for the VNE technology of the collaborative computing task offloading.

B. Virtual Network Embedding
The existing VNE, which is primarily responsible for managing the substrate resource allocation, is a significant technical element of the NV, overcoming the resource ossification of broadband access, especially for the converged FiWi access networks in 5G and beyond. The shared SN can be allowed to accommodate multiple customized VNs requested by the different end users simultaneously. Cao et al. [9] have studied the embedding algorithm for the VNE paradigm from the perspective of VNE problem formulation, NV business model, main performance metrics, existing VNE algorithms in terms of fundamental category and complex category, and key future research directions. Given that the energy cost for accommodating the forthcoming VNRs is more than half of the operation and maintenance cost for the InP, which causes the emergence of energy-aware VNE for cloud networks. On the other hand, the VNE is an NP-hard problem. However, the traditional metaheuristic algorithms are confronted with key technical challenges of ineffective node and link mapping. Therefore, researchers have successively proposed the particle swarm optimization algorithm (PSO), e.g., constructive PSO for VNE (CPSO-VNE) [26] and historical archives and setbased PSO for VNE (HA-PSO-VNE) [27]. The development and popularization of information and communication technology (ICT) expedite energy consumption and resource consumption; therefore, the green communication system focusing on component optimization is becoming a promising solution [28]. Besides, the prior VNE algorithms mainly focus on maximizing the revenue of InP while ignoring the energy consumption minimization of the underlying SNs. Therefore, one of the key issues in designing VNE lies in how to host more VNRs and then embed them into the SN with minimum energy consumption and high embedding validity. Xu et al. [29] described the enhanced bacterial foraging optimization algorithm for energy-driven VNE (BFO-EVNE), with the goal of minimizing energy consumption, and getting high VNRs acceptance ratio and InP revenue. However, the existing VNE solutions only consider the single objective optimization, e.g., resource utilization, embedding reliability, QoS satisfaction, congestion control, and energy consumption, and there exists multiobjective optimization-based VNE problems, e.g., CEVNE [30], network load and embedding reliability [31], and QoS satisfaction and network reconfiguration [13]. In conclusion, the VNE problems over the substrate FiWi access network are created under the condition of considering efficient virtual resource embedding.

C. Virtual Network Migration
The dynamic embedding and reembedding of VNRs over the SN can conform to the VNE solutions on the optimality and resource efficiency. The novel paradigm called VNM, stemming from the VNE technology, is designed to mitigate the underutilization of resource for the SN, and further minimize the energy consumption. Generally speaking, energy-aware mapping and migration strategy consists of node migration and link migration, in which node migration takes precedence over link migration. To achieve the minimization of the energy costs, the software-defined cloud data centers (SDDCs) for the VN allocation take VN computation cost, VNM cost, and data transmission cost into consideration to develop an energy consumption model [32]. Furthermore, the suitable conditions for the migration play a crucial role in achieving the efficient VNE. It is necessary to evaluate the effect of the weight parameter on the number of virtual element migration. As for the inefficient resource allocation of the shared SN, Labriji et al. [33] took full advantages of EA-VNM and enhanced group-based EA-VNM (EA-VNM-G) to further cut the energy consumption cost while maintaining the similar revenue. To facilitate the protocol establishment and application deployment, the energy-aware mapping and live migration strategy for the VNE is designed to mitigate the negative effects of dynamic allocation and deallocation on the substrate resources [34].

A. Network Model
In the VNE paradigm, a sequence of heterogeneous VNs, with the demand of node capacity and link capacity, should be allowed to map onto the shared yet underlying SN, which has a certain amount of resources for node and link capacity. In general, the typical node attributes contain storage capacity, CPU capacity, and node location, and the link attributes include delay provisioning and bandwidth capacity. For ease of understanding, both the SN abstracted from the general FiWi broadband access network that is managed by the InP and the VNs requested by the SP are modeled as weighted undirected graphs, characterized by , respectively, in which N s and N v represent the set of substrate nodes and virtual nodes, L s and L v denote the set of substrate links and virtual links, A s N , A v N and A s L , A v L specify the attribute set of nodes and links, and t a and t d represent the arrival time and departure time of the VNRs, respectively. Therefore, each VN can achieve the service duration of t d − t a . To the best of our knowledge, the VNRs attached to the time attributes can be transformed into the customized VNs in the primary stage of the NV. As for the physical SN, both the general substrate node n s i ∈ N s and substrate link l s ij ∈ L s have the global network resource attributes concluding the residual node CPU capacity resource C r (n s i ), the residual node storage capacity resource M r (n s i ), the node type t(n s i ) (i.e., t(n s i ) ∈ {olt, onu, wr}, among which the elements indicate the OLT, optical network unit (ONU), and wireless gateway/wireless router (WR), respectively), and the node geographic location l(n s i ), as well as the residual link bandwidth capacity resource B r (l s ij ), the link type t(l s ij ), i.e., t(l s ij ) ∈ {fiber, cable, wireless}, are also included, among which the elements represent the fiber link connecting OLT with remote ONUs, the cable link between ONU and wireless gateway, and the wireless link located in the wireless mesh network (WMN) frontend, respectively. Here, the different link types are equipped with the differentiated link bandwidth capacity to provide invaluable insights for the application scenarios. Similarly, the VN resource constraints, accommodated by the general virtual node n v x ∈ N v and the virtual link l v xy ∈ L v , can be characterized by the node CPU capacity demand , where A and F denote the access node and the forwarding node, respectively), and the link bandwidth demand B(l v xy ). Different from the traditional VNE scheme, the access node in the forthcoming VNs must be embedded into the OLT of the substrate FiWi access network, and the forwarding nodes are the WR or the wireless gateway or the ONU. For a simple and intuitive analysis, the service-oriented V-FiWi broadband access network is depicted in Fig. 1, in which more details are available to the interested readers.
To improve the success probability of migration in the VNE progress, the residual yet available node CPU capacity resource, node storage capacity resource, and link bandwidth capacity resource in the substrate FiWi access network must be larger than the node CPU capacity demand, node storage capacity demand, and link bandwidth capacity demand in the forthcoming VNs, i.e., C r (n s , in which the residual global network resources can be further represented by, respectively, where C(n s i ) represents the node CPU capacity resource of initialization, M(n s i ) describes the node storage capacity resource of initialization, B(l s ij ) denotes the link bandwidth capacity resource of initialization, and ∀n v x → n s i and ∀l v xy → l s ij specify the node and link embedding functions, respectively.

B. Migration Model
It is generally considered that the arrival of the VNs follows the Poisson distribution, and each of them departs after the duration time t d − t a , which follows exponential distribution. The remaining resources of the SN, meanwhile, are caught in a certain situation of the attractive dynamics. Therefore, the optimal mapping solution will be transformed into a suboptimal embedding solution with larger energy consumption and inefficient resource allocation. The reason is that some physical nodes with unbalanced resource consumption become active. To remap the VNs and save energy, the energy aware-based active migration technique is proposed to reoptimize the VNE technology. More particularly, the essential idea of remapping and migration is to construct the candidate virtual nodeset and the candidate target substrate nodeset. In other words, which virtual nodes should be migrated from whose source substrate nodes to whose target substrate nodes when the corresponding source substrate nodes switch from the active state to the sleep state. The larger the total amount of the source substrate nodes powering off is, the more energy consumption cost the VNM scheme can be achieved. In the heuristic approach-based NRM, the greedy node embedding approaches, i.e., the large-to-large (L2L) and the small-tosmall (S2S) embedding schemes, have been intensively studied and employed in the prior arts [35]. To turn off more and more source substrate nodes, the small-to-large (S2L) migration scheme is put forward to embed the virtual nodes with the smaller NRM value into the substrate nodes with the larger NRM value after the L2L embedding strategy instead of the substrate nodes with smaller NRM value, which can be regarded as source substrate nodes and represented by S s (S s ⊆ N s ). One key issue in the active migration scheme is how to first execute the L2L embedding scheme based on the greedy approach. For the sake of analysis, the virtual nodes with the smaller NRM values are regarded as the candidate virtual nodeset, denoted by U v (U v ⊆ N v ), while the substrate nodes with the larger NRM value are modeled as the candidate target substrate nodeset, represented by T s (T s ⊆ N s ), because the candidate target substrate nodeset can provide the candidate virtual node with enough global network resources after the L2L embedding scheme. To the best of our knowledge, both the threshold of resource consumption ratio and the migration sequence of the candidate virtual nodes have a dominant effect on the minimization of energy consumption. However, the multiple virtual nodes originating from the different VNs can be embedded into the same substrate nodes. The redefinition of the node embedding is based on manyto-one strategy, instead of the one-to-one strategy. In other words, with the help of replacing the traditional S2S scheme with the S2L scheme, the greater number of powering off the source substrate nodes S s is, the more energy consumption of the substrate FiWi network can be further saved. Therefore, it is necessary to quantize the total number of powering off the source substrate nodes that can primitively accommodate the virtual nodes, which can be further derived as follows: (4) where α u st defines a binary variable, it is equal to 1 when the valid migration occurs between the source substrate node s and the target substrate node t, otherwise, it equals 0. If and only if all the virtual nodes are reembedded from the source node s into the target node t via the migration path P st , the whole migration process stops with the embedding interruption and it should be transparent to the customer premises, because both the SP and the InP have mastered the global network resources and the VNM tasks in advance. Besides, the active migration scheme of the virtual nodes needs to consume the path bandwidth resources for forwarding the storage capacity resources of the virtual node. However, the migration process incurs not only the transient service interruption but also the path bandwidth consumption, during which the disadvantage is to incur the additional migration time, and the advantage is to decrease the energy consumption. In a nutshell, the migration time of forwarding the storage capacity demand of the virtual node u from the source substrate node s to the target substrate node t can be expressed as follows: where M(u) defines the storage capacity demand of the virtual node u, and B(P st ) denotes the substrate path bandwidth capacity between the source substrate node s and the target substrate node t. More specifically, the larger the substrate path bandwidth capacity is, the shorter the migration time is. Therefore, the migration path with the larger bandwidth capacity should be given the first priority in the link migration stage.

C. Key Performance Metrics
For the realization of green communication and networking over the promising FiWi architecture, one key issue for the active migration scheme proposed in the V-FiWi access network is how to quantize the saved energy consumption while maintaining the InP revenue and the substrate resource utilization simultaneously. For the kth VN, we next elaborate on the key performance metrics of InP revenue, substrate resource utilization, and energy consumption in the service period [t a (k), t d (k)].
1) Long-Term Average InP Revenue: For hosting more VNs, the InP provides the underlying network resources for the SPs, which are abstracted from substrate FiWi access network, and charges them some fees as the revenue in return. Here, for the kth VN, the profit of the InP P(G v k ), depending entirely on the service duration of VN, the node CPU capacity demand, the node storage capacity demand, as well as the link bandwidth capacity demand, is represented by subtracting the cost clarifies the joint virtual node CPU and storage capacity demand, and β C and β B denote the revenue per unit of node capacity demand and link capacity demand, respectively. γ C and γ B describe the cost per unit of node capacity resource and link capacity resource, respectively. L v k and N v k explain the link set and node set of the kth VN, respectively. B(l v xy ) introduces the link bandwidth capacity demand of the kth VN, and hop(l v xy → l s ij ) defines the total hops of the substrate migration path. Therefore, the profit of kth VN can be represented by In addition, the long-term average InP profit for a sequence of the instantiated VNs in the time period of [0, d] can be defined as where |G v (d)| denotes the total amount of VNs to be migrated in the given time period [0, d]. The larger profit InP can achieve, the larger revenue InP can first acquire, i.e., the more VNs can be successfully embedded into SN.
2) Long-Term Average Substrate Resource Utilization: In the substrate FiWi broadband access network utility, the SN resource utilization R u (G v ) refers to the ratio of the total amount of node and link capacity demands for the forthcoming VN entities to the total amount of underlying SN node and link capacity resources. This converged InP and SP performance metric evaluates the superiority of the existing VNE algorithms. The larger SN resource utilization is, the more VNs can be embedded into a substrate FiWi network. The longterm average substrate resource utilization in the time period [0, d] can be expressed as follows: (9) where s = n s i ∈N s C(n s i ) + M(n s i ) represents the substrate resource provisioning in terms of node CPU and storage capacity, and B(l s ij ) introduces the link bandwidth capacity resource.
3) Long-Term Average Energy Consumption: To the best of our knowledge, the CPU computation, the active source substrate node, and the switching target substrate node will consume a plethora of energy. More specially, the available CPU capacity of the substrate node can be applied to complete the migration computation of the VNs, in which the total energy consumption is proportional to the total amount of the node CPU capacity resource, and the energy consumption is further represented by E c . In addition, the practical servers, e.g., OLT, ONU, wireless gateway, and WR, will consume some energy for the VNE scheme and VNM paradigm, which can be represented by E b and E s , respectively. Therefore, the energy consumption category of the substrate FiWi access network mainly consists of CPU capacity resource consumption of the substrate node, baseline power of the active source substrate node, and switching cost of the candidate target substrate node. Based on such observation, the long-term average energy consumption in a certain period [0, d] can be calculated as follows: where E c , E b , and E s denote the energy consumption per unit of the CPU capacity resource usage, the active substrate node, and the switching target node, respectively. More specially, the first term indicates that the energy consumption of the substrate CPU resource has a linear relationship with the total amount of the CPU capacity demands. Then, the second term E b · (|N s | − M node ) plays a very large part of the total energy consumption, where |N s | denotes the total amount of the substrate nodes. In other words, if the maximum number of the migration node M node can be achieved, the energy consumption will be significantly minimized. Furthermore, the third term denotes the switching cost energy consumption, which depends on the interaction times of powering the substrate node up and the switching unit cost, where |T s | represents the total amount of the candidate target substrate nodes.

D. Problem Formulation
The active migration technique of the energy-aware V-FiWi broadband access network has been described in detail, with the goal of minimizing migration time and maximizing migration nodes. Even if an optimal embedding solution can be found when a VNR arrives, the global optimal VNE solution may become suboptimal solution after a period of time because a sequence of VNRs arrive and depart dynamically and the remaining resource of the SN will also change correspondingly. Besides, solving the VNE problem with multiobjective constraints is an NP-hard. Similarly, the CEVNE model achieves the suboptimal solutions via heuristic solution, which involves in node-mapping algorithm and link mapping process. Meanwhile, the EA-VNE is formulated as an integer linear programming problem searching for the suboptimal solution via heuristic algorithm and particle swarm optimization. To solve the EA-VNM problem, the active VNM technique can be leveraged in the reoptimization. In other words, the EA-VNM problem formulation can be further transformed into two subproblems. More specifically, one main subproblem is related to the maximization of the amount of the substrate migration nodes in (4), which is directly related to long-term average energy consumption. The other subproblem, alleviating the disadvantage of the active migration paradigm from the source substrate node s to the target substrate node t, realizes the minimum migration time in (5) when the main subproblem can be solved. Therefore, the energy consumption problem of the VNM paradigm over the substrate FiWi network can be formulated as follows: Subject to: Constraints of node migration Constraints of link migration B(l uv ) ≤ B r P tp ∀u ∈ U v ∀s ∈ S s ∀v ∈ N v ∀t ∈ T s ∀p ∈ N s , v → p, l uv → l tp . (13) Constraints of connectivity where α uv tp defines a binary variable, which has the value of 1 if migrating the virtual link l uv from the source substrate link l sp to the target substrate link l tp is achieved and 0 otherwise.

IV. ALGORITHM REPRESENTATION FOR
THE EA-VNM PROBLEM In this section, to resolve the EA-VNM problem, we will provide deep insights into our designed algorithm called EKM-ENM, from the point of view of the topology and resource-aware NRM stage, node migration stage based on maximum weight matching, and energy-aware link migration stage. Importantly, the computation process of NRM value, taking network topology attributes and global network resources into account simultaneously, has critical effects on the VNM technology due to the relative importance of each substrate FiWi node. To save more bandwidth resource and transmit the storage capacity of the virtual node u from the source substrate node s to the target substrate node t, the enhanced KM algorithm is proposed to realize the maximum weight matching in the customizing bipartite graph. In order to minimize the migration time, the unoccupied substrate link l st with the shortest path and maximum bandwidth in the substrate path P st is selected to implement a greedy path selection strategy.

A. Topology and Resource-Aware NRM Stage
To embed more VNs into the substrate FiWi access network, the relative importance of each node should be first quantified in this section, with the introduction of five fundamental topology attributes into the NRM stage. We have done a deep analysis of the topology-aware VNE strategy based on multiple characteristics, e.g., node degree centrality Deg m , node strength centrality Str m , Euclidean distance between any two nodes Dis (m,n) , node farness centrality Far m , node closeness centrality Clo m , as well as link interference index Int(l mn ). Next, we will present how to compute network topology attribute and NRM value. However, there must exist at least one straight path and loop free between nodes m and n, which has a profound impact on the quantification of network topology attributes and it can be expressed as follows: where the functions link and bandwidth represent the total amount of effective links and link bandwidth adjacent to the node m in the given network, respectively. The Cartesian coordinates (x m , y m ) and (x n , y n ) denote the relative location of nodes m and n, respectively, and N represents the set of the whole nodes in the SN or VNs. Both farness attribute Far m and closeness attribute Clo m of node are wrapped up in the Euclidean distance between any two nodes, and the larger the farness of the node Far m is, the lower its closeness Clo m is, and the more dispersive the nodes are. The minimum interference theory of the traffic engineering, rerouting the traffic to the future traffic with minimum link interference Int(l mn ), has been first studied in the multiple protocol label switching (MPLS) context. As we can see in (15), the link interference Int(l mn ) is positively related to the node farness centrality Far m or Far n , and it is negatively related to the node degree centrality Deg m or Deg n . To integrate the interactions of network topology attributes with global network resources, including node capacity, node location, and link bandwidth, the resource block (RB) value of the given node m, which is represented by RB m , can be first computed to highlight the resource quantification and the attribute measurement Int(l mn ) (16) where C(m) defines the mth node CPU capacity. To ensure the convenience of computing network topology attributes and limit the quantitative results within a certain range, the normalized RB of the given node m can be calculated as As we know, the interaction strength of any two discrete entities can be measured with the help of Newton's law or Coulomb's law. Leveraging on such premises, this commonality, e.g., interaction between OLT and ONU, ONU and wireless gateway, as well as wireless gateway and WR, can be devoted to quantifying the importance of any prescribed nodes, which consider global network resources and network topology attributes. Therefore, a novel variable mapping factor (m,n) , which can estimate the mapping ability of each underlying node, can be defined as follows: Obviously, the mapping importance degree of any given node m, covering the remaining nodes in the substrate FiWi network, can be expressed as m = n =m,n∈N (m,n) . The constraint condition n = m ensures that the denominator of the (18) is not zero. Similarly, the normalized mapping factor m of the given node m can be calculated as More specially, the variable m (nor) increases along with the increase of node CPU capacity C(m) and node strength centrality Str m , and decreases with the increase of node degree centrality Deg m . The direct NRM approach based on network topology attributes and global network resources can be directly calculated by a novel metric NRM m , just as shown in (20), which covers the normalized RB RB m (nor) and the normalized mapping factor m (nor). Both α and β are weight factors that balance the importance of the variables RB m (nor) and m (nor) in the given substrate FiWi access network

B. Node Migration Stage Based on Maximum Weight Matching
After the NRM strategy based on network topology attributes and global network resources, it is necessary to accomplish node embedding and node migration in the time period [0, d]. In particular, the best threshold value θ is first ensured to identify the set classification between the larger virtual node attributes and the smaller virtual node attributes according to the resource consumption ratio, i.e., . The virtual node set with a larger NRM m value is mapped onto the larger substrate physical node set, represented by the candidate target substrate node set T s , via the greedy L2L embedding approach. The other node sets in descending order of NRM m value will be regarded as the candidate virtual node set U v , which can be embedded into the candidate target substrate node set T s in descending order of NRM m value, i.e., by the use of the greedy node S2L embedding approach instead of the greedy node S2S embedding approach. In other words, the substrate nodes with smaller embedding metric NRM m can be powered off as many as possible to reduce energy consumption.
The optimal matching problem with two disjoint data sets is also known as the maximum weighted matching problem. For the sake of description, the nonnegative weighted edge ω ij can be quantified as the favorability, and assigned to each edge of the bipartite graph G(V, E), where V represents the two candidate node sets, and E describes the matching edge. In addition, the initial node label value ω i (or ω j ) can be called as expected value, i.e., ω i denotes the maximum favorability of the candidate virtual node set. As for the candidate target substrate node set, ω j can be set to zero at the very beginning. Then, with the help of the label update, the promising solution of the KM algorithm is adopted to resolve the maximum weight matching problem. In general, the two disjoint subsets X and Y have the certain number of nodes, usually |X| ≤ |Y|, and the optimal matching problem is also completed, i.e., each node is successfully matched. If the total number of two subsets is not equal, each element of which can be converted by adding a weighted edge with a value of zero.
However, in the VNM problem, our final objective is to find a maximum weight matching in a customizing bipartite graph with the weighted edges ω ut , so that the sum of the weights of the matching edges needs to be maximized, which means the more bandwidth resource is saved. The subgraph for an optimal matching may be nonunique because different edges may have the same weight value. Meanwhile, more than one virtual node in the set U v , originating from different VNs, can be embedded into the same substrate node in the set T s with a smaller network resource, i.e., many-to-one scheme, not one-to-one scheme is adopted in the maximum weight matching. Besides, the two disjoint subsets X and Y are replaced with the two candidate subsets U v and T s , and each node attribute of which has NRM value, node CPU capacity, and node storage capacity. Moreover, the weighted edge can be further represented by the link bandwidth difference ω u st , because migrating the virtual node u from the source substrate node s to the target substrate node t can save the link bandwidth, which similarly leads to migrate the virtual link l uv from the original substrate link l sp to the target substrate link l tp . The initial node label value in the candidate virtual node subset U v is represented by the maximum link bandwidth difference, i.e., ω u = max{ω u st ∀u ∈ U v ∀s ∈ S s ∀t ∈ T s }, but the initial node label value in the candidate target substrate node subset T s is 0, i.e., ω t = 0. The link bandwidth difference can be expressed as follows: While att < att max do 9: Visit the U v (1) = 1 10: For each substrate node t in the T s do 11: If E ut == 1&&V u [u] = t then 12: Visit T s (t) = 1; 13: .add(u); 15: Return to step 6; 16:

29: End
However, in order to distinguish the maximum weight matching problem, from the EA-VNM problem, we proposed the EKM-ENM algorithm to achieve the customizing maximum bandwidth resource saving in the bipartite graph, which consists of the candidate virtual node set U v and candidate target substrate node set T s , and its core idea is to find a new augment path iteratively. Because of the critical effect of the migration sequence on the energy efficiency and the VNR acceptance ratio, both the virtual nodes and the substrate nodes should be first sorted in descending order of NRM value. To achieve the maximum weight matching, the pseudocode of the EKM-ENM algorithm is shown in Algorithm 1.
More specially, the function Attribute_check determines whether the target substrate node t provides the virtual node u with enough resource capacity, e.g., CPU capacity and storage capacity, and the virtual nodes in the same VNR can be embedded into the different substrate nodes. If the augment path cannot be found, we will update the node label value.

Algorithm 2 Dijkstra Shortest Path Selection Strategy for Link Migration
Require: G s , G v , U v , and T s ; Ensure: A feasible path P tp for the virtual link l uv ; 1: Sort the virtual nodes in U v decreasingly according to their storage capacity demand M(u); 2: for each virtual node u in U v do 3: Compute the shortest path via Dijkstra shortest path algorithm; 4: Select the leisure path with maximum link bandwidth; 5: if the greedy path exists then 6: Record this path for virtual node u; 7: end if 8: Record no available path for virtual node u; 9: end for The function Label_update first checks that which nodes in set U v and set T s have been traversed in the maximum weight matching process via the Hungary algorithm. Importantly, the label update can be performed by a loose variable, which is represented by slack. The minimum value of slack can be calculated via slack = min{ω u + ω t − ω ut }, where ω u and ω t denote the node label values of the visited nodes in U v and the unvisited nodes in T s , respectively. After accomplishing one node label update, the new node label value can be achieved. Once the node label update is completed, a new promising edge can be added to the matching. Finally, the virtual nodes in U v will find a perfect matching, which can save more link bandwidth resource until it reaches the maximum number of attempts. However, the time complexity plays a dominant role in the EKM-ENM algorithm. A new augment path in the bipartite graph G(V, E) can be found in the time complexity of O(|N s ||N v |). Therefore, the overall time complexity of the EKM-ENM algorithm is O(|N s | 2 |N v | 2 ).

C. Link Migration Stage Based on the Dijkstra Shortest Path
After accomplishing the virtual node migration from the source substrate node s to the target substrate node t, we can achieve the migration results for each node of the VNRs. Next, we will reassign the unoccupied substrate path P st to transfer the storage capacity of all the virtual nodes. The specific calculation process is shown in (5), and it can be found that the maximum bandwidth of the path P st can minimize the migration time of storage capacity. The universal shortest path link migration is applied in the EA-VNM problem to minimize the bandwidth of the path P tp . Particularly, the virtual links provided by the SP are mitigated one after another, with the emergence of service interruption and bandwidth consumption. Therefore, the shortest path selection strategy, L2L and S2S (L2S2), is designed for node migration. First, the virtual nodes in U v should be sorted decreasingly according to the storage capacity demand M(u). Then, the substrate path with small (large) unoccupied bandwidth P tp is assigned to the virtual link l uv with small (large) storage capacity M(u). The pseudocode of the Dijkstra shortest path selection strategy for the link migration algorithm is shown in Algorithm 2.

V. PERFORMANCE EVALUATION
In this section, the experimental settings of substrate FiWi network and forthcoming VNs are first described, and then the key performance metrics consisting of long-term average InP revenue, substrate resource utilization, and energy consumption are developed. However, the existing studies consider the VNE paradigm over the FiWi from the viewpoints of total revenue of InP, average VNR acceptance ratio, remaining node CPU/link bandwidth variance, and average node CPU/link bandwidth utilization, instead of an economic viewpoint of the energy efficiency. Although the comparison of the EKM-ENM algorithm with the prior arts is difficult, both the traditional EA-VNE scheme [36] and CEVNE algorithm [30], as well as the proposed EKM-ENM algorithm have been considered in terms of energy cost.

A. Parameter Setting
To achieve the evaluation of discrete-time event mode for VNM, the physical network setting based on computing, network, and storage resource constraints is generated in the GT-ITM tool. The substrate FiWi access network consists of 55 nodes, in which the index of 1˜50, 51˜54, and 55 describes the wireless gateway/WR, ONU, and OLT, respectively. Each ONU can accommodate two wireless gateways. The node geographic locations of both eight wireless gateways and 42 WRs can be acquired via the k-means clustering algorithm, i.e., select 50 node locations from the 500 random node locations in the square area of 400 m×400 m. In order to distinguish the VNM over FiWi access networks from the VNE over the traditional access network, the node CPU capacity, the node storage capacity, and the link bandwidth capacity have different values as shown in Table I. Specifically, the initialized node CPU capacity for node types of olt, onu, and wr is 1000, uniformly distributed between 200 and 500, and uniformly distributed between 50 and 100, respectively. Then, the initialized node storage capacity for the corresponding node types is 800, uniformly distributed between 200 and 500, and uniformly distributed between 40 and 100, respectively. Next, the energy consumption of the corresponding node types is uniformly distributed between 2 mw and 100, 10, and 50 mw, respectively. In addition, the initialized link bandwidth capacity for the link types of fiber, cable, and wireless is 1000, 100, and 54 Mb/s, respectively. However, the VN setting is measured in time window of 3000. The arrival time and the service time of the VNs follow the Poisson distribution (i.e., λ a ∈ [2,8]) and the exponential distribution (i.e., λ s = 10), respectively. Each VN can have at most four virtual nodes, the CPU capacity demands, and storage capacity demands of which are uniformly distributed between 5 and 10 and between 20 and 30, respectively. The virtual link bandwidth demands are uniformly distributed between 4 and 6. As for the long-term average InP revenue, the revenue per unit of node capacity demand and link capacity demand is 5, and the cost per unit is 1. In the NRM stage, both α and β are set to be 0.4 and 0.5, respectively.

B. Results and Discussions
In terms of minimizing long-term average energy consumption, in order to verify that the EKM-ENM algorithm is superior to the existing algorithms, e.g., the CEVEN algorithm and the EA-VNE algorithm, the relationship between long-term average energy consumption and time window is described in detail in Fig. 2. In order to represent the total number of VNs, the different VN arrival rates are used to describe the variation tendency, i.e., the greater the value of λ a , the greater the number of VNs provided by SP. Conversely, the smaller the number of the VNs. As can be seen from Fig. 2, on the one hand, under the same VNs arrival rate, the longterm average energy consumption of the EKM-ENM algorithm is much lower than that of both CEVNE algorithm and EA-VNE algorithm, which further indicates that the EKM-ENM algorithm turns off more physical nodes in the substrate FiWi broadband access network than that of both the CEVNE algorithm and EA-VNE algorithm. On the other hand, as the arrival rate of the VNs increases, the long-term average energy consumption of EKM-ENM, CEVNE, and EA-VNE algorithms also increases significantly. More computing, network, and storage resources need to be mapped to the SN, so that more physical nodes can be started. When evaluating the minimum energy consumption of the EKM-ENM algorithm and other algorithms, the time unit of 3000 is selected for the purpose of comparing the maximum energy consumption of the EKM-ENM algorithm with that of other algorithms. For example, for the VN arrival rate of λ a = 2, the long-term average energy consumption of the EKM-ENM algorithm (99.99 W) is about 22 % and 28 % lower than that of the CEVNE algorithm (121.99 W) and the EA-VNE algorithm (128.41 W). It is owing to that the EKM-ENM algorithm remaps VNs to the fewer physical nodes through the active migration technology, which effectively utilize the physical node resources. In addition, a prereplication strategy is used for the substrate physical nodes during migration, and the switching time of the substrate physical nodes that has a negative impact on the migration time is ignored.
To be specific, the long-term average energy consumption of the EKM-ENM algorithm (148.39 W) is about 5.5 % and 8.2 % lower than that of the CEVNE algorithm (156.53 W) and the EA-VNE algorithm (160.56 W), respectively, when the VN arrival rate λ a = 5 is reached. Similarly, the longterm average energy consumption of the EKM-ENM algorithm (165.53 W) is about 2.3 % and 5.5 % lower than that of the CEVNE algorithm (169.33 W) and the EA-VNE algorithm (174.60 W), respectively, when the VN arrival rate λ a = 8 is reached. It can be anticipated that with the increase of the VNs arrival rate (i.e., λ a varying from 5 to 8), the energy saving advantage of the EKM-ENE algorithm compared with the CEVNE algorithm and the EA-VNE algorithm gradually dwindles, decreasing from 5.5 % to 2.3 % and from 8.2 % to 5.5 %, respectively. The reason is that as the number of VNs sharply increases, the availability of SN resources decreases, which results in a decrease in the number of virtual nodes to be migrated.  Next, Figs. 3 and 4 verify the long-term average substrate resource utilization in terms of physical node resource utilization and physical link resource utilization, respectively. The physical node resource utilization, a function of the resource capacity of the substrate node, refers to the ratio of the occupied physical node resources to the total physical node resources. Under the same CCF, the EKM-ENM algorithm has higher physical node resource utilization than that of the CEVNE algorithm and the EA-VNE algorithm, because more physical nodes are available to accommodate the embedded VNRs. As shown in Fig. 3, the average node utilization of the three algorithms is almost the same at the outset. When the saturation is achieved, it can be clearly seen that the node resource utilization of the proposed EKM-ENM algorithm (0.3) is significantly higher than that of the CEVNE algorithm (0.2) and the EA-VNE algorithm (0.1). The reason is that the proposed EKM-ENM algorithm adopts the maximum weight matching node migration, so that the receiving rate of the VNs is also high. Fig. 4 shows the physical link resource utilization for the three algorithms. Different from the physical node resource utilization, the average link resource utilization of the EKM-ENM algorithm quickly shows the advantage in complementary cumulative distribution function (CCDF) = 0.05. At the saturation point, the link resource utilization rate of the EKM-ENM algorithm is 50 %. Meanwhile, the average link resource utilization rate of the CEVNE algorithm and the EA-VNE algorithm is 36 % and 32 %, respectively. This is in line with our expectations, since the proposed EKM-ENM algorithm uses the shortest path link migration to facilitate the balance of link resource, which contributes to the higher receiving rate and link resource utilization of VNRs. Fig. 5 depicts the VNRs acceptance ratio of the three algorithms. It can be clearly seen from Fig. 5 that the proposed EKM-ENM algorithm always has the highest acceptance ratio, which maintains above 80 % from the beginning to 1800 time units. However, the acceptance ratio of the CEVNE algorithm and the EA-VNM algorithm retains above 70 % from the beginning to 1800 and 2200 time units, respectively, and the acceptance ratio of the EA-VNM decreases gradually all the time. More specially, the EA-VNE algorithm tends to select the shortest link path, resulting in the uneven resource consumption. In other words, some key substrate nodes and substrate links may be used for many times, leading to the rapid exhaustion of substrate resources and embedding failure. In addition, the CEVNE algorithm improves the EA-VNE algorithm to some extent via selecting the path with the most remaining bandwidth resources from all the shortest paths, to balance the network load, alleviate the link congestion, and improve the request acceptance ratio. When the migration time is minimized, the proposed EKM-ENM algorithm adopts the maximum weight matching to realize node migration and the shortest path to achieve link migration, so as to make full use of SN resources and obtain the highest acceptance ratio. Fig. 6 presents the VNRs acceptance ratio under the different VN arrival rates. It can be seen that in the case of a given arrival rate of the VNs, with the increase of the average arrival rate of VNs, the VNRs acceptance ratio of the three algorithms gradually declines and eventually tends to be stable. However, due to the utilization of the NRM to measure node importance and maximize weight to match node migration, the acceptance ratio of the EKM-ENM algorithm is higher than that of other two algorithms, especially in the case of higher arrival rate of the VNs. Among them, when the VN arrival rate λ a = 8 is reached, the VNRs acceptance ratio of the EKM-ENM algorithm (0.63) is 30.2 % and 34.9 % higher than that of the CEVNE algorithm (0.44) and the EA-VNE algorithm (0.41). In conclusion, by introducing appropriate node importance measurement and maximum weight matching, the  EKM-ENM algorithm proposed in this article can achieve a greater VNRs acceptance ratio than other algorithms.
The obvious comparison of the long-term average InP revenue of the three algorithms, i.e., the EKM-ENM algorithm, the CEVNE algorithm, and the EA-VNE algorithm, is shown in Fig. 7. Without loss of generality, the arrival rate of VN varies between 2 and 8. With an increasing of VN arrival rate, the three algorithms show a gradual upward trend in the long-term average InP revenue. The EA-VNE algorithm consists of node mapping and link mapping. The node mapping stage further includes two substeps: 1) router node mapping and 2) host node mapping. In router node mapping, the location of electricity price and time-varying diversity is used to save energy cost. Here, the long-term average InP revenue of the EA-VNE algorithm and the CEVNE algorithm can be regarded as the performance benchmark of the proposed EKM-ENM algorithm. Because the EKM-ENM algorithm effectively reuses the residual resources (e.g., node CPU, node memory, and link bandwidth) that are available in the candidate target physical node set, the InP can receive more VNRs, resulting in higher revenues. In particular, the EKM-ENM algorithm can reduce the resource waste of SN nodes by constructing candidate virtual node sets and candidate target physical node sets. Compared with the CEVNE algorithm and the EA-VNE algorithm, the EKM-ENM algorithm uses maximum weight matching in node migration, providing higher revenues for the InP. For example, when VN arrival rate λ a = 8 is reached, compared with the CEVNE algorithm and the EA-VNE algorithm, the EKM-ENM algorithm can increase the revenue of InP by 4.5 % and 12.7 %, respectively.

VI. CONCLUSION
This article optimizes the VNE from the perspective of active migration. In particular, an EA-VNM problem is described and a new heuristic EKM-ENM algorithm is designed. The algorithm solves the problems of how to build alternative virtual nodal sets and alternative target physical nodal sets, how to quantify link bandwidth differences, and how to update node labels. In addition, a shortest path link migration algorithm is proposed, which reduces the migration time of VNs nodes to a large extent. A large number of experiments have confirmed that the EKM-ENM algorithm can save 22% and 28 % energy cost compared with the CEVNE algorithm and EA-VNE algorithm, respectively, while obtaining attractive long-term average InP benefits.