LBRO: Load Balancing for Resource Optimization in Edge Computing

Mobile cloud computing and edge computing-based solutions provide means to offload tasks for resource-limited mobile devices. Mobile cloud computing provides remote cloud solutions while edge computing provides closer proximity-based solutions. Remote cloud solutions suffer from network latency and limited bandwidth challenges due to distance and dependency on the Internet. However, these challenges are addressed by edge-based solutions since the edge node is available in the same network. The use of Internet of Things-based solutions considering future Information Communication Technology infrastructure is on the rise resulting in the massive growth of digital equipment increasing the load at edge devices. Hence, some load balancing mechanism is required at the edge level to avoid resource congestion. The load balancing at the edge must consider the user’s preferences about edge resources such as personal computers or mobile devices. A user must declare which resources can be spared for other devices to avoid overprovisioning essential resources. We present Load Balancing for Resource Optimization (LBRO), a collaborative cloudlet platform to address load balancing challenges in edge computing considering users’ preferences. A comparative analysis of the proposed approach with the conventional edge-based approach yields that the proposed approach provides significantly improved results in terms of CPU, memory, and disk utilization.

problem at the mobile end. The cloud computing paradigm 23 offers a resource-rich environment to these mobile devices 24 for resource sharing and load balancing. The concept of 25 virtualization is used to share the resources of a physical 26 machine. Various service-oriented architectures are offered 27 The associate editor coordinating the review of this manuscript and approving it for publication was Huaqing Li . by the cloud computing paradigm namely, Infrastructure as 28 a Service (IaaS), Platform as a Service (PaaS), and Software 29 as a Service (SaaS) [1]. This paper considers the use case of 30 the IaaS model where a task is bundled as a virtual machine 31 (VM) and is placed on a physical server (cloudlet). 32 The mobile cloud computing (MCC) model is used to 33 offload a compute-intensive task from a mobile device to 34 a cloud environment, thus addressing resource shortage [2]. 35 In the MCC model, a mobile device directly communicates 36 with the remote server using wireless Internet services [3]. 37 However, the challenges of latency, limited bandwidth, and 38 seamless connectivity pose a major hindrance to the usage 39 of this model. Edge-based solutions such as mobile edge 40 [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], 66 [26], [27], [28], [29], [30]. Moreover, existing edge-based 67 solutions do not consider users' resource preferences which 68 are very vital for the node's performance. These preferences 69 include resource priority and percentage to spare for sharing.  This paper is structured as follows. Section II contains the 97 discussion of important works related to this study. Section III 98 follows the collaborative model, and the design of the pro-99 posed LBRO framework while Section IV provides perfor-100 mance analysis. Lastly, the conclusion and future directions 101 are given in Section V.

103
This section provides insight into state-of-the-art edge com-104 puting techniques. The related work is reviewed from the 105 perspective of load balancing and resource optimization in 106 an edge federated environment.

107
A queuing network scheme has been proposed in [9] using 108 a multi-edge and user-based model. This scheme efficiently 109 works on user-to-edge device mapping and edge device place-110 ment. The target of this scheme is constantly moving mobile 111 users and the scope is limited to a Metropolitan Area Net-112 work (MAN). The scheme proposed in [10] has considered 113 load balancing in the form of a VM to an edge device with 114 adequate resources. The proposed scheme is more focused 115 on total migration time rather than download time. The min-116 imum total time is achieved by adapting to Wide Area Net-117 work (WAN) bandwidth and loading on the edge device. The 118 state of the VM on the destination edge device is compared 119 with the source edge device and the difference is calculated 120 to maintain the same state before the source VM can be shut 121 down. Delta encoding scheme is used to calculate the dif-122 ference that is de-duplicated and compressed before transfer. 123 The user VM is moved in closer proximity to the source edge 124 device to minimize latency. An SDN-based solution named 125 MobiScud has been proposed in [11]. A mini cloud in the core 126 of Radio Access Network (RAN) is established to host users' 127 Virtual Machines (VMs). These VMs assist users to execute 128 compute-intensive tasks and control messages from mobile 129 devices that are monitored by MobiScud to keep the VM 130 moving along user to keep it in closer proximity. MobiScud 131 also optimizes the flow rules for migrating VM to keep the 132 transition phase smooth with less disruption of services to 133 users. However, users tend to use WiFi more often when 134 indoors, and RAN services are thus needed to be adjusted 135 accordingly.

136
An ad hoc scheme is proposed in [13] that allows peer 137 devices to lend and acquire resources from each other. 138 Two devices in communication are called master and slave 139 devices. A device offering free resources to others is treated 140 as a slave device and the other device borrowing resources 141 is treated as a master device. This cooperative scheme also 142 supports a smaller scale network assuming there is no inter-143 ruption in task offloading. Another ad hoc scheme is proposed 144 in [14] using short-range radio communication technology to 145 form a peer-to-peer (P2P) network of mobile devices. Mobile 146 devices participating in the proposed scheme are divided 147 into two categories i.e. a computational service provider 148 that has ample resources to offer and a client who requires 149 resources. An opportunistic approach is used by devices to 150 find appropriate peers and lend services. This scheme is 151 97440 VOLUME 10, 2022 useful for a smaller-scale network with a short span of service requirements. A scheme named DRAP is proposed in [15] 153 that uses middleware between mobile and edge devices. The 154 devices with resources can form a group and will be treated 155 as edge devices whose resources can be acquired by any other  and 2) the maximum distance between any two edge nodes 208 is 2 hops. The objective of this scheme is to achieve higher 209 throughput with minimum delay time. The proposed scheme 210 assumes that nodes are static, and the maximum number of 211 cloudlet hops is two for mobility. It has also been reported that 212 if these assumptions are not fulfilled, the proposed scheme 213 will provide poor results as compared to any standard cloud 214 computing solution. So, as a remedy to the challenge of 215 meeting the assumptions a combination of edge-based and 216 cloud-based schemes is recommended. The experimentation 217 is performed only on interactive mobile applications.

218
A centralized Enterprise Cloud (EC) based scheme is pro-219 posed in [24]. All the participating edge devices are reg-220 istered with EC which maintains complete information of 221 all the edge devices. Moreover, any mobile device requiring 222 resources is also registered with EC. The request for resources 223 first goes to EC which is responsible for allocating an appro-224 priate edge device. The advantage of this scheme is that a 225 mobile device moving away from an edge device may resume 226 the same task on any of the edge devices registered under the 227 same EC thus saving cost, time, and energy. However, this 228 scheme is highly dependent upon internet bandwidth since 229 EC is not a part of the local network. A similar approach 230 using a centralized root server is proposed in [25]. The root 231 server maintains all sorts of information including connected 232 edge devices and services provided by them. A request is 233 forwarded to the root server that routes it to a suitable edge 234 device. A suitable edge device receiving the request from the 235 root server either executes the task by employing its resources 236 or can share resources with other edge devices. It also has 237 the capability to break the task into smaller proportions and 238 distribute it among various edge devices. Being a centralized 239 service model scalability can be an issue if a larger network 240 model is to be considered.

241
A cloning technique has been proposed in [26] that main-242 tains a clone of the mobile device in the core of RAN. The 243 clone is based on a VM that is kept in closer proximity to 244 the mobile device and is migrated along the mobile device to 245 maintain minimum distance thus reducing latency. However, 246 these frequent migrations may result in increased network 247 traffic between various edge devices maintained in the core of 248 the RAN. SDNs are used to optimize and manage these traffic 249 flows thus improving performance and energy consumption. 250 The implementation of this technique with the existing RANs 251 requires some fundamental changes in the core cellular net-252 work. A mesh network-based solution named MeshCloud 253 using Wireless Mesh Networks is proposed in [27]. The fun-254 damental property of a mesh network is high robustness due 255 to multiple paths leading to a single destination. The proposed 256 scheme is highly dynamic as new edge nodes can be added 257 and removed at any time. Mesh topology lacks scalability and 258 is thus not suitable for larger networks.

259
A novel task offloading scheme is proposed in [28] that 260 caters to DDoS attacks and considers sustainability and secu-261 rity issues of the cloudlet networks. A collaborative task 262 offloading mechanism for mobile cloudlet networks named 263 VOLUME 10, 2022 CTOM is proposed in [29]. An online algorithm is proposed  at the broker containing the required resources, the broker 298 matches the requirements with all member cloudlets and 299 dispatches the request to the optimal cloudlet having adequate 300 resources with minimum latency, and updates the resource 301 state of that cloudlet. Similarly, when a VM is to be migrated 302 due to load or a more optimal location, the occupied resources 303 are released from the source server and an updated state of 304 resources is maintained.

305
Besides managing resource information of member 306 cloudlets, the broker performs various operations for cloudlet 307 federation such as cloudlet registration, keeping track of 308 resources, and optimal cloudlet selection. The resources 309 include CPU, memory, storage, and bandwidth, whereas opti-310 mal selection includes decisions regarding cloudlet and VM 311 for migration [1].

312
Existing cloudlet-based models support resource sharing 313 and load balancing based on local knowledge about the other 314 cloudlet nodes (fixed or mobile) in closer proximity, prefer-315 ably within the same LAN. The proposed cloudlet federation 316 extends the range of closer proximity to MAN and WAN with 317 97442 VOLUME 10, 2022

357
The problem of finalizing the optimal cloudlet is challenging 358 due to dependency on multiple variables including resources 359 such as CPU, memory, storage, and bandwidth. Weights are 360 assigned to each resource by the owner of the edge device to 361 segregate spare resources for sharing. For example, an owner 362 'x' wishes to spare 20% of the CPU, 30% of the memory, 363 10% of his disk storage, and 5% of the bandwidth to take 364 part in the sharing process for some other user 'y' to acquire 365 these resources for the execution of some task. The owner 366 can simply assign weights 2, 3, 1, and 5, respectively to each 367 available resource for sharing.

368
The resource calculation process can be divided into two 369 phases i.e., resource index calculation and resource level 370 calculation. The resource index calculation phase provides 371 a single cumulative value based on selected resources, their 372 quantities, and assigned weights that are used to rank the 373 cloudlets in the federation. Let W = {w 1 , w 2 , w 3 , . . . , w n } be 374 set of weights assigned by the owner against each resource, 375 C = {s 1 , s 2 , s 3 , . . . , s n } be a set of cloudlets, A = 376 {b 1 , b 2 , b 3 , . . . , b n } and T = {r 1 , r 2 , r 3 , . . . , r n } represents 377 set of available and total resources at a cloudlet respectively. 378 The value of the total resource index can be calculated using 379 the following equation.
Algorithm 1 is designed for resource index calculation and 382 is presented below.

383
The resource level calculation phase helps to initiate load 384 balancing on a cloudlet having a critical resource level. 385 There are two levels of resources. One is ''normal'' and 386 the other is ''critical''. A cloudlet is considered in a normal 387 state if available resources are above the minimum thresh-388 old level, whereas a cloudlet is considered in the critical 389 state if available resources are below the minimum threshold 390 level. The value of the threshold is identified by the resource 391 requirement of the host OS i.e. in our case the minimum 392 Let U = y 1 , y 2 , y 3 , . . . , y n be a set of utilized resources,  Table 4.

401
The value of ''1'' represents a critical state and ''0'' repre-409 sents otherwise. Algorithm 2 presents resource level calcula-410 tion. Two phases of the optimal cloudlet selection include filtration 413 of eligible cloudlets having enough resources to execute the 414 job that is identified by the following condition: In the second phase, the optimal cloudlet is selected 418 based on the resource index value calculated in algorithm 3. 419 The cloudlet with the maximum resource index value and 420 non-critical resource level is selected as optimal from the 421 eligible cloudlet list. The maximum value of the resource 422 97444 VOLUME 10, 2022 index indicates that the cloudlet has the maximum available 423 resources in the federation. Let Q = q 1 , q 2 , q 3 , . . . , q n repre-424 sents the set of requests at the broker. In case a cloudlet is in a critical state, a load balancing 427 mechanism is initiated that requires a VM to be migrated 428 from the critical cloudlet to ease up the load. The problem of 429 finalizing the optimal VM is challenging due to dependency 430 on multiple variables such as VM size and required resources.

431
The selection process of optimal VM starts with the filtration  The load balancing mechanism is performed offline.

438
A VM is considered eligible if by removing it from the 439 cloudlet, the resource level becomes normal. In the second 440 phase, an optimal VM is selected from the list of eligi-441 ble VMs. A VM occupying minimum resources and size 442 is selected as it can be migrated in a very short amount 443 of time having a larger solution space of eligible cloudlets. 444 Algorithm 4 presents the mechanism for optimal VM selec-445 tion for migration. This section discusses performance metrics, experimental 448 setup, and testbed results. In these experiments, we record the 449 resource level of a cloudlet with and without the implemen-450 tation of our proposed approach and present a comparative 451 analysis. Since we are using cold migration, there is no over-452 head of memory pre-copying operation consuming more CPU 453 cycles and network bandwidth [32]. An Open Virtualization 454 Appliance (OVA) file is transferred from a heavily loaded 455 cloudlet to a light-loaded cloudlet having maximum available 456 resources.

458
Several metrics such as CPU, memory, disk utilization, and 459 latency are considered for the evaluation of proposed algo-460 rithms. The considered metrics reflect system resources that 461 are independent of application type and nature. The main 462 objective is to launch enough requests that the system is 463 forced to either forward the request to the remote cloud in 464 the case of the conventional cloudlet model or collaborate 465 with peer cloudlets in the case of the proposed approach. Real 466 load in terms of VMs has been used to exhaust the system's 467 resources.  Table 5. Memory is considered a critical resource of a computer sys-510 tem. Often this resource creates a bottleneck for the sys-511 tem performance due to continuous read, write, and paging 512 operations. A comparative analysis of the memory utilization 513 trend for both models is shown in Figure 4. The results 514 clearly show an elevated level of memory utilization by the 515 conventional model completely utilizing the memory causing 516 performance degradation. The load balancing feature of the 517 proposed model admits limited requests according to avail-518 able memory, thus avoiding the critical resource situation.

520
Storage resource is not considered critical resource now a day 521 due to the availability of larger capacity at a low cost. How-522 ever, the increased rate of reading and writing requests from 523 storage might cause performance degradation. The results of 524 storage utilization for both models are presented in Figure 5. 525 The results clearly show an elevated level of disk utilization 526 by the conventional model as compared to the proposed 527 model due to load balancing features. The load balancing 528 module admits a limited number of requests according to 529 available memory space. No request is admitted if there is 530 no free memory available avoiding page swapping between 531 97446 VOLUME 10, 2022

553
In future work, a software platform that supports resource 554 collaboration is to be developed for commercial purposes 555 considering cost and energy. This model will serve as a base 556 and other parameters such as latency, hop-count, throughput, 557 response time, execution time, and offload time will be incor-558 porated as future enhancements to the presented algorithms. 559 Additionally, instead of manually assigning resource-specific 560 weights to segregate spare resources of a cloudlet for sharing, 561 optimal weights per resource will be dynamically predicted 562 using unsupervised learning methods and neural networks.