Demystifying Usability of Open-Source Computational Offloading Simulators: Performance Evaluation Campaign

Along with analysis and practical implementation, simulations play a key role in wireless networks and computational offloading research for several reasons. First, the simulations provide the ability to easily obtain the data for a complex system’s model evaluation. Second, simulated data provide a controlled environment for experimentation, allowing models and algorithms to be tested for robustness and identifying potential limitations before deploying them in real-world applications. Choosing the most appropriate tool for simulation might be challenging and depends on several factors, such as the main purpose, complexity of data, researcher skills, community support, and available budget. As of the time of the present analysis, several system-level open-source tools for modeling computational offloading also cover the systems’ communications side, such as CloudSim, CloudSim Plus, IoTSim-Edge, EdgeCloudSim, iFogSim2, PureEdgeSim, and YAFS. This work presents an evaluation of those based on the unique features and performance results of intensive workload- and delay-tolerant scenarios: XR with an extremely high data rate and workload; remote monitoring with a low data rate with moderate delays and workload requirements; and data streaming as a general human traffic with a relatively high bit rate but moderate workload. The work concludes that CloudSim provides a reliable environment for virtualization on the host resources, while YAFS shows minimal hardware usage, while IoTSim-Edge, PureEdgeSim, and EdgeCloudSim have fewer implemented features.


I. INTRODUCTION
T HE rapid growth of network services has forced the creation of new methods to offload the computations and data.The architecture of the Internet of Things (IoT)driven scenarios, e.g., smart home and health monitoring, consists of the entities to sink and process data [1], [2].The idea to utilize the resources of remote computers improved info-communication technologies, allowing one to store and process large amounts of data without wasting own machine's resources.
Historically, Mobile Cloud Computing (MCC) implies the computational offloading from mobile devices to cloud servers via the communication link [3].This paradigm allows to use an enormous computing and storage capacity.Even though datacenters have high performance, they also consume a lot of power, which can cause global problems.The distance to the nearest datacenter greatly impacts the latency.As of February 2023, there are only 24 datacenters in Finland, where 18 are located in the capital region [4].The connection time from northern Finland to the nearest Cloud server might be counted by several seconds.In contrast, some delaysensitive applications would require orders of magnitude less [5].
Shifting the offloading to the edge of the network aims to assist in resolving the latency issue.An edge is a server near the gateways that can process data with lower communication latency than in the cloud because of its proximity to the user.Like edge, fog computing is another paradigm that processes big tasks not far from the user with minimum delay [6].Nowadays, there are more than six emerging paradigms for optimal processing data in remote servers, such as mobile edge, cloud computing, and fog computing, that have their own advantages according to the particular use case and its requirements [7].The growing attention to the computing paradigms correlates with the growing number of emerging toolkits for computing paradigms simulations since actual implementations of testbeds appear to be close to impossible.There is a wide range of tools for validating, generating, transmitting, and offloading data, as shown in Fig. 1.The left pillar represents the nature of the environment, the central part of figure shows the tool's name, and the right part represents the research topics for which the tool was used.In 2022, MATLAB became the most popular tool among researchers applying it from wireless connections and IoT to offload scenarios due to its mathematical nature and significant base of toolboxes.Python-based Integrated Development Environment (IDE) gained the most popularity for modeling over the years because of its implementation simplicity and high availability of resources.Network Simulator 3 (ns-3) is a widely used network simulator that is keeping popularity over the years.Recently developed tools are gaining popularity in various research fields, e.g., satellite networks and connected vehicles.
The practical reason for simulations is the allowance to test the robustness of new models and algorithms without using real-world data, which might be challenging to obtain, and to identify potential issues and limitations before deploying them in real-world applications.Choosing the right tool for data simulation depends on several factors.The first and principal is the purpose of the simulation and what kind of data you need to obtain.Tools might have unique features suitable for specific scenarios only.Another factor is the user's skills and his/her preferences in the programming language, Operating System (OS) or Graphical User Interface (GUI).Last but not least is the amount of dedicated budget for purchasing.
The main contribution of this work is the evaluation of the existing open-source simulators used for modeling cloud, fog, and edge computing scenarios from the systems' communications side based on implemented and unique features and their performance.We analyzed the simulators used in the computational offloading, Mobile Edge Computing (MEC), and MCC.The work includes the evaluation of the following tools: CloudSim, CloudSim Plus, IoTSim-Edge, EdgeCloudSim, iFogSim2, PureEdgeSim, and YAFS.
Notably, not all tools shown in Fig. 1 under computing/offloading research topics were included in the comparison.MATLAB is a matrix programming language suited for analytic model validation by conducting extensive simulations.It has applicability in any engineering research and does not specialize in computing simulations; therefore, it was not included in this overview.OMNeT++ Discrete Event Simulator (OMNeT++) and ns-3 are also widely used network simulators among researchers.ns-3 is not suitable for the IoT simulation at the edge level since it does not support the scheduling and application composition features, while OMNeT++ does not support edge communication protocols.Therefore, they were not included either.GreenCloud is a packet-level simulation tool, which can measure the energy consumption of datacenter components.This simulator only focuses on the calculation of energy consumption to ensure energy-aware placement [8].
The rest of this article is organized as follows.Section II contains the introduction to the computing paradigms, the review of related works, the list of criteria, and the simulators' descriptions.Section III presents their performance comparison.This article ends with a discussion and recommendations.

II. FEATURES EVALUATION
This section introduces the fog-edge-cloud computing paradigms based on [7] and delves deeper into the literature review of works that compare tools.Then, it presents the developed set of metrics and the simulator's description.

A. Background on Computational Continuum
In 2011, the National Institute of Standards and Technology (NIST), USA, published an official paper providing a comprehensive definition of cloud computing.According to NIST, cloud computing is characterized as "a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction" [9].Cloud computing is a powerful datacenter comprising several interconnected nodes through high-speed channels.The cloud architecture consists of two hierarchical levels: the end user and the datacenter (user cloud).A stable connection to the Internet is a key requirement for cloud service providers.Their primary goal is to allocate the appropriate node to complete the user's task computation while ensuring data security.
The cloud architecture's physical layer includes servers, network equipment, and storage devices.The lowest layer of the cloud consists of physical infrastructure drivers and cloud drivers, facilitating communication with hardware and other external clouds.The core of the cloud OS encompasses a Virtual Machine (VM) manager, network manager, storage manager, and other relevant components [10].The top layer of the OS features various management tools, such as administrator tools, service managers, schedulers, and cloud interfaces.Clients connect to the server through this cloud interface.
The initial computing paradigm aimed to forward data to the cloud for analysis.In addition, cloud computing has enabled big data analysis, accessibility from multiple platforms, and fast computational speed due to its high computational power.However, the proliferation of wearables, sensors, and IoT devices has posed more stringent requirements in terms of mobility support, geodistribution, location awareness, and latency.Consequently, new paradigms, such as edge and fog computing, have emerged, aiming to reduce the distance between end devices and the central server.
Fog computing is a distributed computing infrastructure that brings computational capabilities closer to the user while retaining cloud-like features.This approach allows for data storage and processing with lower latency, better location awareness, and higher Quality of Service (QoS) for real-time applications [11].On the other hand, edge computing locates the datacenter at the actual network edge providing computing offloading, data storage, and data processing services [12].These paradigms share similarities but disagree regarding processing location and types of used hardware [13].Fog and edge computing aims to bring data processing nodes closer to the user, but the key difference lies in the role of the first node in the network.Edge devices are positioned as the first node, while fog nodes' proximity depends on the availability of servers.The hardware in edge computing may include lower end devices, whereas fog computing relies solely on server-based hardware.Edge computing provides computation on the end network servers, while fog computing processes data at the Local Area Network (LAN) level.
In summary, cloud computing was precisely defined by NIST in 2011 as a model for ubiquitous access to configurable computing resources.Since then, computing paradigms have evolved, leading to the emergence of fog and edge computing.These new paradigms focus on reducing latency and proximity between users and servers, see Fig. 2, presenting new opportunities and challenges for Information and Communication Technology (ICT), especially in the medical domain [7].

B. Related Works
A comparative evaluation of simulators is necessary to demonstrate the superiority of a particular tool in a specific scenario becoming a subject of many works.
Aljabry and Al-Suhail [14] introduced a brief survey on network simulators for Vehicular ad hoc Network (VANET).The authors reviewed many popular simulators but provided no simulation results or performance evaluation.Kang et al. [15] presented a comprehensive survey on network simulators Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.for Airborne ad hoc Network (AANET) or Flying ad hoc Network (FANET) and Underwater Sensor Network (UWSN).Patel et al. [16] presented a comparative study on network simulators.The conclusion showed that OMNeT++ and Network Simulator 2 (ns-2) were the most appropriate network simulators for large-scale network simulators.The promising tool ns-3 was gaining popularity as an easy-to-use tool for simulating wireless networks.Anyway, the work did not provide any performance comparison between the simulators.The same comment applies to work by Toor and Jain [17], where they presented a survey on open-source network simulators, e.g., ns-2, ns-3, J-Sim, OMNeT++, OPNET, QualNet, and MATLAB.Bakni and Moreno [18] proposed an evaluation approach to describe the network simulator's behavior, capacity, and performance.Bakni et al. [19] applied the proposed approach for the Cisco Packet Trace network simulator and extended their work by applying the evaluation approach to several additional Wireless Sensor Network (WSN) modeling tools.
Sundani et al. [20] provided a comparison on WSN simulators based on key features, limitations, scalability, Central Processing Unit (CPU), and memory usage on more than ten simulators.Weingartner et al. [21] proposed a methodology for performance evaluation, which provides the comparison between network simulators.However, the information in the work is partly outdated already.Sarkar and Halim [22] provided the comparison based on the deployment mode, type, supported protocols, and network impairments.A strong contribution of this work was the presented comparison and the provided recommendations.
Simulators specialized in computational offloading have gained popularity recently and started to develop rapidly.Kunde and Mann [23] worked on theoretical and practical comparison of fog computing simulators, e.g., iFogSim, MyFogSim, and YAFS, showing the impact on simulation run time and other parameters.Fakhfakh et al. [24] analyzed the most popular simulators for cloud computing and compared them based on the supported modules (energy, mobility, and so on).Qu et al. [25] presented a new simulation platform for edge computing with distributed learning and blockchain models and introduced a comprehensive literature review on the existing cloud and edge simulators, focusing on the lack of the federated learning concept implementation.Unfortunately, this simulator is not available in open source.

C. Set of Criteria and Methodology
There is no standardized method to evaluate the simulator, but the proposed set of criteria exists in the literature [18].Following the authors' example, we present our criteria for the simulator's evaluation.At any rate, the purpose of the simulation might differ from work to work.That is why we leave it to the readers to decide which tool is the best for them, based on the preferred programming language, OS, and the research aims.
1) Open Access: This criterion shows the code available in open access, which means that it is online and free of charge.
2) Programming Language: This characteristic highlights the programming language used for writing scripts and modules.
3) User Interface: Simulating tools can have a GUI to provide a user-friendly environment.
4) Documentation Availability: This criterion represents the availability of the related documentation: manuals, tutorials, videos, presentations, setup instructions, and so on.
5) Ease of Setup: This characteristic shows the experience of the tool's initial configuration.
6) Ease of Use: This characteristic shows the experience of the tool's usage.It could vary from easy to hard.
7) Scalability: This characteristic shows how scalable the tool is.The number of nodes in IoT scenarios could be more than 100, so it is critical to answer whether the tool allows scaling easily on such a number.We assume that the simulator is scalable if it allows to connect up to 100 end devices.
8) Supported Features: This characteristic represents the tool's key features in the implemented modules, e.g., mobility, orchestration, or networking.
Sections II-D and II-E introduce the investigated simulators.Since simulators work in the different abstraction levels, see Fig. 3, the tools were divided into two groups and introduced in separate subsections.The first subsection emphasizes the tools for simulating virtual cloud environments.They show the service availability, energy consumption, allocation of tasks, etc.The first group includes the following tools: CloudSim, EdgeCloudSim, IoTSim-Edge, and CloudSim Plus.The second subsection introduces the second group of tools that work with the network architecture.It describes the system from the user to processing server.They show task response time, which combines server processing time and delivery time and provides the choice of data processing location.This group includes iFogSim2, PureEdgeSim, and YAFS.Table I provides a comparable analysis of mentioned tools.

D. Simulators Focused on Computing Infrastructure and Application Services
1) CloudSim: The simulator was developed by the Cloud Computing and Distributed Systems (CLOUDS) Laboratory at the University of Melbourne, Melbourne, VIC, Australia, in 2009.The primary idea of this project was to develop a toolkit for modeling and simulating recently emerged cloud computing infrastructure and their services [30].CloudSim is an open-source simulator assisted with documentation and installation guides.
The simulation environment contains the following entities: a host, i.e., simulated hardware, VM, and datacenter; cloudlets, i.e., tasks, services from the user, and broker.The entity broker is responsible for negotiating between the user (cloudlet) and the cloud provider (datacenter) and allocating the resources there.The output results show the status of the cloudlet proceeding, start time and end time, processing time, and id of the datacenter and the VM where the cloudlet was sent.Cloudlet is defined by the following properties: length, file size, and output size.VM, the real cloud-based VMs, is defined by Million Instructions per Second (MIPS), image size (Mb), Random Access Memory (RAM) (Mb), bandwidth, and several CPU.
CloudSim supports virtualization, i.e., the creation of multiple VM on the physical server (host).Furthermore, it supports VM migration, which means that it allows simulation of the movements from one physical host to another.In real practice, this feature will enable us to adaptively allocate the workload on the servers for better system performance or to maintain the failed server without user interruption.Among the key features, CloudSim supports cloud datacenter network topology, which includes the wireless and physical interconnection of datacenter components, such as storage, computing entities, servers, and switches.
2) CloudSim Plus: CloudSim Plus is a new Java-based framework for modeling a cloud computing environment first released in 2016.Based on CloudSim but working as an independent project, CloudSim Plus improved its performance and simplified its usage by reducing the amount of duplicated code and restructuring the modules and packages [31].The tool has a related project, CloudSim Plus, which is an opensource simulator with lots of available documentation on its webpage, white paper, and discussions on the Google forum.The online documentation is one of the most detailed explanations of the implemented features.
The CloudSim Plus project is similar to CloudSim but has improved structure, reducing the complexity of the scripts.The simulator consists of the physical layer, which includes servers, the logical layer, i.e., the datacenter network topology, and the virtualization layer, used for creating VM.The modules support the VM migration and vertical/horizontal VM scaling.It uses the same entities-hosts, datacenter, cloudlets, and broker, responsible for communication between the user and the datacenter.
3) EdgeCloudSim: It is an open-source tool for edgespecific modeling based on CloudSim and was developed in 2017.The main uniqueness of this project is that it considers both computational and networking resources compared to its predecessor CloudSim.The project's scripts can be run on Linux-based systems, including Mac OS via the preferable IDE for compilation [32].
The tool inherited CloudSim module for VM allocation in the datacenter and other computing features.Nevertheless, EdgeCloudSim includes unique features for edge computing architecture described further.The edge orchestrator module is responsible for making critical decisions on allocating or terminating the VM on the available resources and offloading tasks to the cloud or edge server to increase the overall system performance.The networking module is responsible for LAN or Wireless Local Area Network (WLAN) communication delay for both directions Uplink (UL) and Downlink (DL).EdgeCloudSim supports mobility, so the devices' locations and, consequently, the delay are updated according to this module.
Running the simulation requires defining the application, edge device, and configuration parameters.To avoid the overloaded script, the mentioned parameters are stored in three separate files named accordingly.The application file contains information about application data size, battery level, usage distribution on the devices, active and idle duration, and so on.The configuration file sets the cloud and simulation parameters, such as the number of mobile devices, orchestration policy, and architecture.It is possible to set the computing capabilities to mobile devices or use them as sensors without a processing unit.The edge device file includes edge datacenter parameters.It defines the number and location of the edge server, computing and storage capabilities, and the number of VMs deployed on it.The number of edge servers is scaled-the tool allows linking more than ten servers and deploying multiple VMs on each of them.Still, the specific of such structure needs to define each server separately and, thus, not user-friendly.
EdgeCloudSim benefits in investigating the Quality of Experience (QoE) for the edge-based computing scenarios.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

TABLE I MAIN SIMULATORS CHARACTERISTICS
The supported modules simulate scalable scenarios and provide information on package delivery in the dedicated server.
4) IoTSim-Edge: This tool was developed in 2019 and is based on CloudSim project.It is focused on IoT-driven offloading scenarios such as planning the capacity of Road-Side Unit for the intelligent transportation system or sensor deployment in the smart building scenario [8].
The archive file is available in GitHub and complemented with the user manual.Unfortunately, the file was not updated for a long time, and no recent file is available.The project built under Maven contains the pom.xml, but the old versions and missing dependencies could fail the Maven build process.
The IotSim-Edge simulator consists of the following entities Edgelet, i.e., a generated task from the IoT sensor; MicroElements (MEL), i.e., an abstract component of the application that represents the services in the form of microservice; edge devices, i.e., a laptop, smartphone, or other devices that host MEL; edge datacenter, i.e., the core edge infrastructure; and EdgeBroker, which allocate users' requests with accordance to their requirements.After setting the required parameters (MIPS, RAM, battery capacity, location, and so on), the simulator allocates MEL for edgelets and outputs their execution time.The simulator supports such features as energy consumption, mobility, and networking.

E. Simulators Focused on Network Architecture and Resource Allocation
1) YAFS: It is a SimPy-based highly configurable simulator designed on a complex network theory for analysis of fogdriven computing scenarios developed in 2019.It allows the creation of a scalable, dynamic network and simulates the request in it [33].
The installation guide and project documentation are available in the open source on the official webpage.
Installation is roughly simple.If it fails to find the matching "YAFS," the distribution must upload manually to the Python home file.Up-to-date documentation, user guides, referred papers, and solutions to solve Python errors are available online.
Execution requires defining the network and application.Network topology is modeled as a graph that contains computing capacity and bandwidth information for each node and data rate and propagation information for each link.The communication time, i.e., latency, is calculated as the ratio between message size and set bandwidth plus the propagation speed.The service time is the time that is needed to process the packet.
The network architecture is presented as an orgraph with the forest of trees topology.A node represents a cloud server connected to the proxy server linked to a set of gateways.Each gateway could be linked to the defined set of mobile devices, including sensors and actuators.There is no separate entity for them; sensors and actuators are mobile devices with no computing capacity that generate or consume data and connect to the main mobile device.The application is set as a group of modules that can generate or process a packet.Modules could be deployed on the cloud server or in the group of edge servers, depending on the orchestration policy.
One of the advantages of YAFS is a highly configurable and flexible architecture that allows running a scalable network in terms of the number of nodes and modules.It supports virtualization, microservices, and other features easily defined as logical relationships by customized configurations.
2) iFogSim2: It is a tool to simulate the fog computing environment developed upon the CloudSim and measures the impact of resource management techniques in network congestion, latency, cost, and energy consumption [34].It was updated in 2021 and inherited features from the old version (iFogSim).The rising number of IoT large-scale scenarios There are guidelines available on the GitHub page.Also, the CloudSim tutorial page contains several guidances of iFogSim2 Project Structure for beginners as it is one of CloudSim's related projects.
The simulation entities consist of FogDevice, representing the actual fog computing resource; fog broker, responsible for the task distribution; and sensor class, i.e., cameras and temperature sensors.The output file shows the execution time of the proposed topology and energy consumption on each node (device and server).The topology allows the creation of nodes in different layers.
The iFogSim2 shortcomings are partly solved in the extension MobFogSim, whose primary purpose is to simulate the mobility in the IoT gateways and cloud datacenters.In turn, MobFogSim limits the scope of creating clusters in edge/fog computing environments and lacks documentation.

III. SIMULATORS PERFORMANCE COMPARISON
The performance evaluation is a numerical characteristic for comparison of the scripts.The script performance highly depends on the programming language, used libraries, project architecture, and other parameters.Even though all projects are evaluated together in this work, it is important to mention that the evaluated tools differ in the initial aim and their structure, and it is better to take in mind the difference between the following two groups that were already introduced earliertools that are major for virtualization or task allocation in the servers/datacenter and those that are focused on packet's arriving time and describe the network architecture.The first group includes CloudSim, IoTSim-Edge, and EdgeCloudSim, while the second group includes PureEdgeSim and YAFS.CloudSim Plus and iFogSim2 was not included in the performance evaluation due to compilation problem.

A. Test Scenarios
As one of the motivations, emerging resource-hungry applications should meet rigorous communication requirements set by the standardization bodies.In contrast to traditional light use cases, e.g., remote sensing or monitoring, 3GPP TR 26.928 "Extended Reality (XR) in 5G" considers the media delivery bitrate >1 Gb/s to provide a sufficient media quality and low latency, mentioning the need for potential standardization [28], see Table II.The end-to-end latency for the XR environment, referred to as immersive motionto-photon, including rendering and decoding, states around 20 ms or less [29] for a smooth user experience.Nonetheless, the online video stream, 3GPP TR 26.925 "Typical traffic characteristics of media services on 3GPP networks," refers to the 150-ms max packet delay budget that the user will barely notice.Today, the recommended bitrate for Full HD video lies between 3 and 12 Mb/s, while for the 4K UHD, 5-25 Mb/s [27], which is smaller than the XR.
Following the standardized recommendations, we further focus on the three scenarios taken as examples of intensive workload and delay-tolerant scenarios to make the test scenarios closer to real-life implementation.They also include the standards of different modalities, static or moving, based if the user is moving or not while delivering/accepting the service.
XR scenario corresponding to remote Augmented Reality (AR)-assisted telesurgery is a used example of extremely high data rate with intensive workload and static modality, detailed in Table III, based on the standardization summary [35].AR applications process the video data generated by the laparoscope, or 3-D ultrasound probe, equipped with a small camera, and process it on a server (private server, edge, or cloud), according to 3GPP.The laparoscope and robotic medical instruments (trocars, graspers, and scissors) are inserted through tiny incisions in the patient's body.Since all organs function during the operation, the video is transmitted to the console monitor with ultrasmall delays to prevent healthy tissues' perforation.The telesurgery scenario assumes that the patient and the operating doctor are physically located on different continents operating remotely.In that case, the IEEE standard defines the performance requirements Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.[26], [36], AND [37] for multicast video traffic for medical applications via Public Land Mobile Network (PLMN) [36].The 3-D ultrasound probe augments the main anatomical image with the 3-D volume data, producing a data stream above 1 Gb/s.The AR image from a 3-D ultrasound probe requires a precise robotic instrument's location in the patient's body.The images are exchanged in a total of 240 images/s over cellular communication.

TABLE III QOS METRICS COMPOSED WITH
The monitoring scenario numerically corresponds to cardiac telemetry, i.e., a low data rate with moderate delays and workload requirements, and moving modality, detailed in Table III.A wireless wearable telemetry device includes body sensors, e.g., ECG, respiratory rate, and SpO 2 , which provides 24/7 monitoring of the patient's health.Due to the on-body way of wearing, the device must be small and energy-efficient.Cardiac telemetry devices require to keep devices alive for at least a month without recharging.From the performance side, it requires a highly reliable, always-on connection with the hospital to process the patient analytics and raise the alarm in an emergency.The number of devices varies on the patient location: up to 1000 wearables/km 2 in the hospital area or about ten devices per km 2 in suburban areas [26].
The streaming scenario is an example of general human traffic with a relatively fast bit rate but moderate payload, as detailed in Table III.The application aims to provide access to the user's favorite online in Full HD to watch from a mobile device.User location could be static or moving with the user, e.g., if the user is sitting in the train, the speed could reach up to 500 km/h.The packet size is 500 B, and the endto-end delay requirement is set as 150 ms [37].The number of devices varies up to 500 active devices in city areas.
To sum up, all three scenarios differ regarding payload and required bit rate.The intense XR scenario, i.e., ARassisted surgery, requires transmissions of the uncompressed captured images 256 × 256 × 256 voxels 24 bits 10 frames/s that is 4 Gb/s.Monitoring scenario, i.e., cardiac monitoring, measures data with a frequency of up to 1000 samples/s, thus requiring up to 500 kbps bit rate.Streaming scenario, i.e., online video streaming, transmits compressed images from/to the user device.Full HD video resolution 1920 × 1080 24 bits 60 frames/s and H.264 gives up to 9 Mb/s.The recommended performance metrics of the use cases are presented in Table III.

B. Overall System Model
The system contains 80 sensors/actuators implemented in the IoT device for the AR telesurgery scenario and 1000 sensors as wearables devices for cardiac monitoring.The  devices are wirelessly connected to the gateways, which can offload data to the edge or cloud datacenter over a 5G network.The datacenter allocates resources for VM or MEL to process cloudlet or edgelet, which refers to the application workload.
Assume that the channel bandwidth for mobile devices over cellular network is 1 MHz where the system bandwidth of 20 MHz and 20 users are simultaneously connected [38].The transformation of channel bandwidth (Hz) to bit rate (bps) depends on many factors, including the chosen technology, coding rate, and modulation.Let us say that the average 20 MHz is approximately 100 Mb/s; consequently, 1 MHz is 5 Mb/s.The proxy server is connected to the datacenter via optical fiber, which can reach 1 Gb/s.
The processor speed, usually measured in MIPS, represents the number of million instructions (MIs) required in one cycle, where by "instruction" means a specific hardware operation to process the task.Specification of the widely used ARM platform in the embedded IoT states that ARM Cortex-M4 has 100-MHz processor frequency and 128-kB RAM; ARM Cortex-M3 has 72 MHz and 20 kB RAM [39].Cortex-M3 and M4 cores achieve 1.25 MIPS/MHz [40], which is approximately 125 MIPS at 100 MHz [41].Compared to the datacenter capacity, the server processor, e.g., Intel Xeon [42], has 38.5-MB cache and 2.50-GHz processor frequency with up to 28 cores built to process up to 100 000 MIPS [43].The reference simulation for Group 1 models the allocation of the received tasks (cloudlets) and VMs in the datacenter that was generated with the required size and task frequency.Other simulation parameters are presented in Table IV [44].
All simulations were conducted on Ubuntu OS deployed via the VirtualBox with dedicated four CPUs and 12 GB of memory.Simulation scripts were launched in the terminal via a bash script.The bash script runs the process after a slight delay (5 s), fixating the start and end time of the running and measuring the CPU utilization and memory every 1 s via -ts Linux command.The exception was made for IoTSim-Edgedue to the Maven compilation issues launched via the cmd line, the script was run in Eclipse.The results were collected the same way-measuring the CPU utilization and memory every 1 s via -ts Linux command by process name.CloudSim Plus and iFogSim2 did not participate in the performance evaluation due to compilation failure.

C. Evaluation Results
The main aim of this section was to understand much hardware resources are utilized while executing the script, but not explain the behavior of their performance, as those are implementation-specific and could not be affected.Assuming that simulators received the same scenario according to their capabilities, we compare each of them regarding CPU, RAM usage, and the simulation execution time.
1) Maximum CPU and RAM Values: Table V presents the maximum CPU and RAM values of each simulator.The built-in Python simulator YAFS showed the best (minimum) usage in CPU and memory for XR scenario (see Fig. 4).It is explained by the nature of languages and their different methods [45].However, it showed the worst results for monitoring and streaming scenarios in Figs. 5 and 6.The last two scenarios had relatively smaller workloads but applied devices on a bigger scale.The simulator's peculiarity is that it creates the link for each sensor/device.Thus, a growing  number of devices will add to their network graph complexity and reflect on the hardware usage.The same happened with the EdgeCloudSim; even though it works in the virtual abstraction, it still requires specifying the links for the sensor nodes (see Fig. 3), thus the growth of the hardware load.CloudSim showed the opposite, and its CPU and RAM usage stayed the biggest during XR scenario, as the biggest simulated workload was applied.
IoTSim-Edge showed the best (minimum) CPU usage and worst (maximum) RAM usage from Group 1 for all scenarios.Figs.4-6 show that the IoTSim-Edge slope rises sharply during processor and softly during memory load's peaks.The longest raise corresponds to the highest CPU or RAM.CloudSim was the best in terms of RAM usage from Group 1. PureEdgeSim showed the worst RAM usage from Group 2 for all scenarios and the worst CPU usage for XR and streaming scenarios.YAFS was the best in terms of RAM usage from Group 2.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.2) Load Over Time: Absolute values are sometimes not enough to estimate the processor and memory load.Dedicating many resources could increase the processing speed but overload the processor in the case of extended use.We integrated data over time to evaluate the tool's performance, where the higher values represent the most load.Fig. 7 presents the cumulative normalized processor and memory load for XR scenario, Fig. 8 presents the cumulative normalized processor and memory load for monitoring scenario, and Fig. 9 presents the cumulative normalized processor and memory load for streaming scenario.YAFS and CloudSim showed that the RAM curves have a slow elevation over time, but the CloudSim CPU curve increased rapidly.EdgeCloudSim showed a slow peak of the CPU curve over time but a sharp curve of RAM usage.Summing up, the results conclude that the different ways of code implementation influence the CPU and RAM usage on the hardware.
3) Simulation Run Time: The results about tool's run-time behavior could be derived from Table V. IoTSim-Edge made the fastest run from Group 1 in all scenarios.Python-based YAFS from Group 2 worked very fast during the XR scenario and was overloaded with the increased graph complexity during monitoring and streaming scenarios, where the best result was showed by PureEdgeSim.

IV. CONCLUSION AND DISCUSSION
Developing a good simulation tool for modeling networks is considered as a valuable contribution to academic work.On one hand, most of the existing reliable network simulators do not consider cloud entities, such as datacenter, host, VM, or broker.On the other hand, computational simulators do not count network delays and mobility highlighting the need for Edge-and Fog-specific simulators forced by emerging computing paradigms.
CloudSim was one of the first open-source simulators, which implemented avant-garde technologies.It has been a validated and trusted tool for many years.Fortunately for humanity and unfortunately for the developers, technologies change fast, but this tool lacks upcoming modules.Notwithstanding the mentioned limitations, CloudSim inspired other independent projects that used it as a base project for more improved simulators, i.e., CloudSim Plus.In turn, CloudSim Plus inherited many entities and features CloudSim, but the reengineered project improved the performance and deployed new modern features, i.e., event listener, VM migration, and parallel computing.Developers advanced the scripts made them more human readable to improve the usability.
EdgeCloudSim is an easy-to-set-up and easy-in-use tool, which simulates the task execution in various edge scenarios (edge cloud and mobile edge) and supports orchestration and networking models.A significant disadvantage of this tool is lack of energy model and task migration.IoTSim-Edge suits primarily if the research evaluates IoT devices and their interaction with edge devices.Despite its more user-friendly appearance, it lacks essential features such as virtualization and orchestration.
PureEdgeSim and iFogSim2 are focused on offloading to the edge.iFogSim2 allows to simulate complex IoT scenarios considering the delays between sensors and IoT device.It allows to connect the servers into N -tier hierarchy and allows to specifies the delays between them.The tool allows to model latency-sensitive applications and, thus, suits to the AR-assisted surgery simulations.In contrast, PureEdgeSim comprises a range of orchestration architectures, including mist computing, i.e., offloading on mobile devices, mist edge, mist cloud, and edge cloud, which is applicable to telemetry scenarios in the medical domain.The implemented energy model shows the energy consumption and task failures due to battery run out.
YAFS is a perspective project, which provides full support for the user in the open source.It plans to implement geolocalization and other features to the existing ones and solve the Python compatibility issues; hence, some limitations might be irrelevant in the near future.Project YAFS works on implantation modern features for fog computing architecture, and it allows to model VR scenarios; thus, it is suitable for VR-assisted remote medical scenarios as well.
Summarizing the above, there are many tools available in the open source designed for modeling cloud, fog, and edge computing scenarios.Each of them has its limitations and advantages that could suit the research specific aim.Table VI summarizes the simulation tools investigated in this work.Choosing the best simulator for the research questions could take some time, so this article aims to help in minimizing this time overviewing the most popular tools for modeling the offloading scenarios for medical applications.

Fig. 1 .
Fig. 1.Most used environments in the telecommunication, computer science, and engineering area as of 2023.

Fig. 3 .
Fig. 3. Diversity of simulation tools for offloading scenarios and their applicability according to different abstraction levels.

3 )
PureEdgeSim: It is an open-source simulator for studying dynamic and highly heterogeneous networks based on CloudSim Plus.It was developed in 2018 for deploying edge, fog, and cloud It allows the connection of thousand of devices and supports their mobility with the location manager.Many research papers are available open source as well as the official GitHub pages containing the setup instructions and step-by-step video with examples.The main simulation file takes the parameters of cloud and edge servers and mobile devices stored in the .hmlfiles.Simulation configuration consists of simulation time, device count, orchestration algorithms, and other parameters that could be changed according to the scenario.It allows for collecting the energy consumption, task execution time, and task success rate.

TABLE IV SIMULATION
PARAMETERS FOR SCENARIOS IN

TABLE VI SUMMARY
OF SIMULATORS ADVANTAGES AND DISADVANTAGES