Employing FAMTAR and AHB to Achieve an Optical Resource-Efficient Multilayer IP-Over-EON SDN Network

With the global Internet traffic continuously growing, network operators face more and more challenges related to the management of their networks. Efficient utilization of the available network resources becomes crucial to maintain the desired Quality of Service level and control the upsurge in operational expenses. The combination of the Software-Defined Networking concept with the multi-layer network architecture can simplify the process of control and management of the network, its layers, and resources. In this paper, we propose a solution to enhance resource utilization in software-defined multi-layer optical networks. The proposed solution takes the advantages of the Software-Defined Networking, Flow-Aware Multi-Topology Adaptive Routing, and Automatic Hidden Bypasses mechanisms to ensure simultaneous, multi-path data transmissions in both IP and optical layers. The Software-Defined Networking controller manages both mentioned mechanisms, selects the best possible bypass, and allocates lightpaths to ensure that the optical spectral efficiency is optimal. The evaluation shows that the proposed solution for multi-layer software-defined network increases the overall network performance and resource utilization.

optical networks. Since that time, many researchers covered 99 the topics of multi-layer network control with standard pro-100 tocols [3], [4], [5], recently there is also some momentum in 101 using artificial intelligence for that control [6], [7]. 102 In [3] authors present cooperation between Segment Rout-103 ing (SR), SDN and optical bypasses. Custom SDN solution 104 is used to control edge node label stacking configuration and 105 optical bypasses, which are used upon load variations. The 106 routing policy for the optical bypass is based on the prede-107 fined threshold and does not require signaling protocols. Seg-108 ment Routing is also explored in [4], where authors make use 109 of it in two situations for a multi-layer network. Firstly com-110 bined with SDN and dynamic optical bypasses and secondly 111 used to effectively load balance the traffic also among non-112 ECMP routes. Research in [5] also employs Segment Routing 113 technology, however, in a 5G multi-layer, multi-domain net-114 work. The authors validate SDN-based network slicing for 115 disaggregated 5G transport networks, with slices defined at 116 multiple layers and provisioned over multiple domains. 117 Regarding artificial intelligence, authors in [6] present 118 fully distributed multi-layer routing policies based on 119 BIO-inspired ant colony optimization algorithm with online 120 control for the optical and IP/MPLS layers. The algorithm 121 presented by the authors assumes disjoint control planes for 122 both optical and IP layers with the only local routing infor-123 mation in each network node, which represents a different 124 approach than SDN, where network control is centralized. 125 A reinforcement learning algorithm implementation is shown 126 in [7]. The algorithm is used in SDN controller to provide a 127 proper virtual multi-layer network resource allocation with 128 fine service isolation. The introduced control loop, allows to 129 improve resource utilization over the network nodes. 130 Some solutions [8], [9], [10] focus on network resilience 131 problems rather than optimal resource allocation. Refer-132 ence [8] addresses the problem of cross-layer orchestration to 133 address IP router outages in IP-over-EON. The authors pro-134 pose a set of multi-layer restoration algorithms which aim to 135 minimize the operating expenses. Reference [9] addresses the 136 problem of the survivability for IP over EON networks. The 137 authors proposed a proactive restoration method for a joint 138 multi-layer network, which was shown to achieve efficient 139 resource usage and outperform the single-layer protection 140 methods. Moreover, integer linear programming formulations 141 were presented to provide survivability in the case of link or 142 node failure in the network. Authors in [10] introduce a new 143 SR scheme to recover traffic flows after a network failure 144 event dynamically. They employ SDN controller to obtain a 145 network topology but only when a failure occurs. The solution 146 allows reducing the failure recovery time.

147
There is also a hardware-based approach for multi-layer 148 network control. In [11] authors take advantage of the P4 149 switches rather than the SDN controller. Traffic is forwarded 150 to the optical bypass once the predefined threshold is reached. 151 Interestingly, the authors consider two cases for the bypass 152 usage -reroute all packets or reroute just the portion which 153 exceeded the threshold.
Even though the number of solutions stated above exists, they mainly introduce one-layer optimization (like bypasses) 156 and show how to perform the network control over multi-

168
This section provides the necessary background for the 169 multi-layer network concept as well as mechanisms which 170 were used, FAMTAR and AHB.  SDN controller at the proper layer, while keeping the resource 209 usage as efficient as possible. With the global visibility of 210 the network, SDN controller can perform simultaneous, inte-211 grated control of multiple layers in the network to fulfill the 212 QoS requirements of the incoming traffic. Finally, SDN con-213 troller can dynamically decide which traffic demand is trans-214 mitted through which layer in order to increase the resources 215 utilization even more.

216
The concept of multi-layer SDN raises new challenges 217 which we define as a questions in Table 1.

219
FAMTAR [15] is a multi-path adaptive routing mechanism 220 based on the concept of flows. FAMTAR can work with every 221 routing protocol being responsible for finding the best possi-222 ble path between two endpoints, since it operates above the 223 intra-domain routing protocol (IGP). In a scenario when there 224 are no congestions in the network, all transmissions between 225 those endpoints use the best path. However, when conges-226 tion occurs, flows which were already active remain on their 227 primary path, while new flows are pushed to an alternative 228 path. Therefore, the optimal paths change according to the 229 congestion status of the links -FAMTAR uses the best path 230 provided by the routing algorithm and in case of congestion 231 automatically triggers finding new paths.

232
To accomplish that, a FAMTAR router stores Flow For-233 warding Table (FFT) together with a classic routing table. 234 In FFT each flow has a corresponding entry which represents 235 the interface to which packets of this flow are forwarded. This 236 information is taken from the current routing table when the 237 flow is added to the FFT, i.e., when its first packet appears. 238 For flows that are present in the FFT the routing table is not 239 consulted, therefore FFT is used to execute the majority of 240 the packet routing tasks. Entries in the FFT are static and do 241 not reflect routing table changes.

242
Once a state close to congestion is noticed on one of 243 the links, the adjacent router updates the cost of this link 244 with a predefined high value. This link is then perceived as 245 congested. Updated link cost appears as a standard change 246 in the routing protocol, which spreads this information as 247 a standard topology change message. When routers receive 248 this information, they compute new paths which are likely 249 to avoid congested links. Routing tables are updated with 250 the newly computed paths. However, this update affects only 251 new flows. The flows which were active before that event are 252 VOLUME 10, 2022 should use it. The analysis presented in this paper shows 285 that AHB can provide lower delays and higher throughput. 286 The mechanism yields excellent results in both low and high-287 loaded networks. Bypasses can be created manually by net-288 work operators or automatically in centralized or distributed 289 systems.

290
The mechanism was also introduced into IP-over-EON 291 architecture in [18]. In EON, the optical spectrum used for the 292 transmission is divided into narrow frequency slices (slots). 293 The slots are reserved by setting an end-to-end path between 294 the optical network devices. The frequency of the slots used 295 over the path has to be static over all-optical hops. The band-296 width of the created path is defined by the number of selected 297 slots and a modulation format.

298
An example of bypass creation is shown in Figure 2. 299 In Figure 2 a simple multi-layer network architecture is 300 considered. The network comprises two layers: IP, where 301 IP routers reside, and the optical (EON) layer where opti-302 cal (fiber) links physically connect optical nodes (cross-303 connects). Additionally, we assume that each IP router is 304 bound with an optical cross-connect -this is typical for 305 existing carrier networks. Thanks to the bypass mechanism 306 only the selected optical resources (slots) are revealed to 307 the IP layer. The remaining spectrum is denoted as hidden 308 resources and can be used when congestions occur. There-309 fore, a new lightpath is established without creating a vir-310 tual link when a request cannot be served in the IP layer 311 due to the lack of resources. This lightpath is then used to 312 offload new traffic. Once the transmission on the bypass 313 ends, the lightpath is removed and optical resources are 314 released.

340
IP layer processing starts with a crucial question of whether 341 the examined link is in the optical or IP layer (link is a 342 bypass or not). If a link is not a bypass y, then the FAMTAR 343 mechanism is employed. If the examined link's load is greater 344 than the FAMTAR's activation threshold and the link's cost 345 wasn't already increased, then the cost of that link is set 346 to a predefined high value z. This message is then spread 347 across the topology as explained in III-B. Otherwise, when 348 the examined link's load is not greater than the FAMTAR's 349 activation threshold, the controller performs a check if the 350 cost of that link was already increased and if a load of that link 351 is lower than the FAMTAR's deactivation threshold. If that 352 condition is true {, then the controller sets the cost of the 353 link to its original value.

354
The optical layer control mechanism starts with a question 355 of whether the examined link's load is greater than the Bypass 356 Creation Threshold (BCT). If the answer to that question 357 is true, then the controller requests the FFT statistics from 358 the network node |. The algorithm for bypass calculation 359 presented in the Algorithm 1 is executed right after the FFT 360 statistics are successfully gathered. The algorithm aims to 361 find the best source-destination bypass to deal with the over-362 loaded link. Based on the flow entries gathered from the FFT 363 of the overloaded link, the algorithm returns the best possible 364 source-destination pair based on our custom metric computed 365 for all of the source-destination pairs. This custom metric is 366 expressed as a float value which is a result of the multiplica-367 tion of the values of the following parameters: 368 metric = hops · rate · nodes_usage · modulation 369 As it is known, multiplication will only work if there is an 370 agreement that low or high values of all parameters are better. 371 In this scenario, the metric is better if all parameters have 372 high values -it is more likely that the pair for this metric will 373 be treated as the best option for bypass. For a more in-depth 374 explanation, we define those parameters as:

375
• hops -the number of hops between source-destination 376 pair, we prefer that the bypass omits as many nodes in 377 the IP layer as possible,

378
• rate -current flow rate, value in Mbps, we would like to 379 feed the bypass with more significant flows, rather than 380 many little ones,  • modulation -the maximum modulation order value that 392 is possible to reach between source-destination nodes.

393
It is important as the proposed system doesn't use optical 394 regenerators.

395
Once the metric_by_pair container is filled with the met-   The allocation mechanism can be divided into two cases.

404
In the first case, when a bypass for a given source-destination we use the Generic Dijkstra algorithm [19], which finds 410 optimal solutions to dynamic RMSA problem, at the same 411 time being considerably faster than other algorithms [20].

412
By optimal solution we mean the shortest possible path taking 413 into account spectrum continuity and continuity constraints 414 enabling supporting a given request. We use the open-source 415 Python implementation of the algorithm provided in [20].

416
In the second case, when a bypass for a given source-417 destination pair already exists~, an additional lightpath for 418 that bypass will be created if the demand will not fit in that 419 bypass's remaining bandwidth. In this way, we make sure that 420 bypass is used to its full potential and that slots in the opti-421 cal layer are not over-allocated. After allocation, we reroute 422 flows from the overloaded link that passes by bypass 423 edges.

424
On the other hand, if the answer to the first question 425 whether the examined link's load is greater than the BCT 426 is false, and the currently processed link is a bypass, then 427 the proposed release mechanism is executed . Because of 428 its design, bypass loses rather than gains traffic. Therefore, 429 it only takes a portion of the traffic that existed at the exact 430 moment of its creation, and new flows are not forwarded 431 through it. A bypass should be removed and therefore release 432 resources once the transmission on it ends. However, in order 433 to boost the efficiency, it is more than reasonable to unset 434 the bypass earlier, i.e., when the utilization of its resources 435

Algorithm 1 Bypass Calculation
Input: F -list of flows on overloaded link, N -node Output: Metrics for all possible (src,dst) pairs Release mechanism, similarly to allocation mechanism, 438 aims to maximize resource utilization, and it can also be 439 divided into two scenarios. As mentioned earlier, the bitrate  IP layer interfaces between nodes directly connected in the 463 emulated topology and bypass interfaces between all pairs 464 of nodes in the network. Direct interfaces were used in the 465 calculation of IP layer routing table and for FAMTAR oper-466 ation. Bypass interfaces were not visible for IP layer routing 467 purposes and initially had their bandwidth set to 0. When a 468 new lightpath was created by the controller on a particular 469 source-destination pair, bandwidth of a corresponding bypass 470 interface was being increased accordingly. After allocating a 471 new lightpath, the most significant flows on that relation were 472 individually redirected to the bypass from direct IP links by 473 changing their outgoing interface entries in FFT. Similarly, 474 when a lightpath was removed by the controller, bandwidth 475 of a corresponding bypass interface was decreased and flows 476 leftover on that interface were redirected back to direct IP 477 interfaces. 478 We evaluated the performance of the proposed solution 479 under various scenarios. We show that the presented bypass 480 algorithm implementation is able to outperform standard 481 FAMTAR implementation in given cases, as well as, save 482 network resources.    each direction, that represent frequency slices described in The results in Figure 5 depict that increasing the num-541 ber of direct slots between nodes in the network leads to 542 increased traffic carrying capacity of the entire system. How-543 ever, we can make two further observations. First, when 544 85 and more slots are allocated for direct IP layer connection, 545 the traffic rate shows no or little change for the non-saturated 546 network scenario. Second, the allocation of 5 slots for future 547 bypass (95 for direct slots), introduces no additional loss even 548 for over-saturated network case.  Figures 6 and 7 show average UDP loss and average HTTP 551 degradation parameter, respectively. The UDP loss is the per-552 cent of sent traffic that could not be carried by the network 553 and did not reach the destination. In the case of HTTP traffic, 554 which is a TCP-based protocol, traffic loss would not be 555 an appropriate measure, as TCP limits its rate in order to 556 maintain a constant loss. Instead, we define a HTTP badness 557 parameter, which is a value that presents the level of degrada-558 tion of the HTTP connection. The HTTP badness is calculated 559 as a deviation of observed flow rate from its nominal rate, 560 traffic loss. 566 We can observe that the allocation of 90 and more slots for 567 the direct IP layer does not change the level of both UDP loss 568 and HTTP badness for every tested network throughput.    One of the key advantages of the hidden bypasses solu-588 tion is that it allows keeping optical resources unallocated 589 until the heavy load occurs. Figure 9 presents average num-590 bers of slots usage. We can observe that the slots usage 591 depends more on the initial slots allocation, rather than the 592 network usage. As by our EON bypass creation algorithm 593 definition, the bypass requires at least two direct links and 594 uses the same slot (λ) over all segments. That creates gaps 595 over the segments spectrum that cannot be allocated for 596 future demands. Because of that the number of slots that can 597 be allocated decreases together with the initial slots alloca-598 tion parameter and reduce possible network throughput for 599 these parameters. That behavior does not impact results with 600 85 and more slots, as the number of possible bypasses is 601 lower. The spectral efficiency for bypasses is presented in 602 figure 10. We can observe that in every bypass configura-603 tion, we exchange more data per slot than in a non-bypass 604 environment. However, even though, the maximum value is 605 for 25 and 50 initial slots, we have to reject that network 606 configuration because of huge network losses for these input 607 parameters.