Advanced Co-Simulation Platform for UAV Simulations Under Virtual Wireless Network Environments

In this paper, we propose an advanced co-simulation platform that performs the unmanned aerial vehicle (UAV) simulation and the wireless network simulation concurrently. As UAV-based applications become more advanced and complicated and the commands and data transmitted between UAV and ground control station significantly increase, it is vital to safely and robustly transmit the packets carrying commands and data in a wireless network environment. However, the conventional UAV simulation platforms do not take into account the wireless network conditions encountered in the real-world flight so they cannot sufficiently simulate packet transmission. To address this issue, co-simulation platforms have been developed to verify UAV-based applications under a virtual wireless network environment. However, they still cannot effectively support the co-simulation needed to develop the UAV-based applications. In this paper, we have derived a list of improvements to the conventional co-simulation platform based on analysis of the conventional ones. Based on these improvements, we have developed an advanced co-simulation platform consisting of a UAV simulation part, a wireless network simulation part, and an integration part. In this platform, a two-channel approach was introduced to parallelize the synchronization required for performing the UAV simulation and the wireless network simulation simultaneously. The experimental results carried out on three test scenarios demonstrate that the proposed platform accurately performs co-simulation by efficient interoperation of its three parts and by remarkable reduction of the processing time delay, and that it is extendible due to its modularized software architecture.


I. INTRODUCTION
In recent years, unmanned aerial vehicle (UAV) has drawn 21 great attention as an innovative technology driving the indus- 22 trial revolution. UAV has been employed for a broad range 23 of applications including surveillance, search and rescue, 24 cargo transport, precision agriculture, inspection, environ-25 mental monitoring, disaster relief and management, etc [1]. 26 The associate editor coordinating the review of this manuscript and approving it for publication was Guillermo Valencia-Palomo . Furthermore, UAV-based applications have become more 27 advanced and complicated by incorporating multi-UAVs with 28 the technologies of big-data, artificial intelligence, and vir-29 tual reality [2], [3], [4], [5], [6], [7]. When developing 30 these advanced UAV-based applications, UAV simulation 31 platforms based on virtual UAVs and virtual environments 32 have been utilized to verify and evaluate the mission plan-33 ning, the flight of UAV, the performance of application, etc. 34 The verifications and evaluations performed on simulation 35 platforms ultimately reduce the risk of accidents during actual 36 an up-down link between UAV and GCS and to parallelize 93 synchronization. The WNETSIM part carries out a wireless 94 network simulation with packet of location data under a 95 virtual network environment. In this simulation, packet trans-96 mission from UAV to GCS and its communication status are 97 examined and transferred to the ITG part. Both the UAVSIM 98 and the WNETSIM parts are interconnected with the ITG 99 part. The ITG part efficiently manages packet transmission 100 between UAV and GCS by a synchronization scheme based 101 on parallel processing of packets. Through the interoperation 102 of these three parts, the proposed co-simulation platform can 103 simultaneously execute UAV simulation and wireless net-104 work simulation. The experimental results demonstrate that 105 the proposed platform effectively performs the co-simulation 106 under various conditions without simulation errors and time 107 delays. 109 UAV simulation platform has been diversely used from mod-110 eling the aerodynamics and functional aspects of UAV con-111 trol system to validating the mission fulfillment and related 112 performance of multi-UAVs. Typically, UAV simulation plat-113 form must support virtual environments and virtual UAVs that 114 reflect the characteristics of the specific UAV-based applica-115 tion to be developed. For this reason, open-source software 116 packages such as PX4, Gazebo, X-Plane, and AirSim have 117 been employed to construct UAV simulation platform [12], 118 [13], [14], [15], [16], [17], [18]. AirSim has been employed 119 to test autonomous flights of UAVs because it provides a 120 flight simulation environment as well as the machine learning 121 data needed for deep-learning technologies [14], [17], [19]. 122 Especially, PX4 and Gazebo are so useful for building UAV 123 simulation platforms because they provide various APIs and 124 commands and have an active community. If these software 125 packages are incorporated with an open-source GCS, it is eas-126 ier to observe and evaluate missions and functional behaviors 127 of UAV. However, conventional platforms based on PX4 and 128 Gazebo can only test the functionalities and performances of 129 an application without considering network environments.

130
The reliability and usability of UAV-based applications 131 are dependent on wireless network environments because the 132 wireless network conditions greatly influence the transmis-133 sion of packets containing command and data. Hence, it is 134 essential to simulate the packet transmission and related oper-135 ations between UAV and GCS under various wireless network 136 conditions. For this purpose, open-source network simulation 137 tools such as NS-3 and OMNet+ have been introduced to 138 UAV simulation platforms [20], [21]. NS-3 is one of the 139 commonly used simulation tools in the network field and it 140 is useful for simulations of packet delivery in virtual envi-141 ronments for Wi-Fi wireless communications. For instance, 142 this tool has been utilized to verify video transmission in 143 UAV systems and build virtual network environments for 144 supporting simulations of swarm UAVs [22], [23]. However, 145 it cannot visualize the flight of UAV or evaluate the perfor-146 mance of the application. In order to successfully simulate an 147 VOLUME 10, 2022 UAV-based application, the functional behaviors of UAV and 148 the packet transmissions between UAV and GCS should be 149 simultaneously tested.   can be compiled into Table 1.

197
As shown in Table 1 FlyNetSim, and CORNET. Even the packet transmission 202 between UAV and GCS is not supported in the CUSCUS. 203 In particular, a single data packet containing both the flight 204 information and location data of the UAV is transmitted over a 205 single channel. This packet is sequentially processed for UAV 206 simulation and wireless network simulation on the conven-207 tional co-simulation platforms. Fig. 1 represents the synchro-208 nization scheme based on the single channel and sequential 209 processing in the conventional platforms. Note that only the 210 location data included in the packet is utilized for the wireless 211 network simulation indicated by the dotted-box in Fig. 1, and 212 the others are used for UAV simulation. As shown in Fig. 1, 213 the packet to be transmitted to GCS for UAV simulation is 214 determined in the packet management stage based on the 215 time delays d p and d n . In Fig. 1, d p represents the processing 216 time required to receive and interpret a packet and perform 217 a wireless network simulation and d n indicates the network 218 propagation time obtained from the NS-3 tool. If d p is smaller 219 than d n , the packet transmission to the GCS is delayed to 220 synchronize the UAV simulation with the wireless network 221 simulation. Otherwise, the packet is discarded in the packet 222 management stage. Packets discarded during synchronization 223 can cause inaccurate simulation results. For these reasons, 224 UAV simulation and wireless network simulation cannot be 225 successfully performed on the conventional co-simulation 226 platforms.

227
Based on the analysis and comparison mentioned above, 228 the following issues should be addressed to improve conven-229 tional co-simulation platform:  In order to perform the co-simulation successfully, we pro-241 pose an advanced co-simulation platform composed of 242 an UAVSIM part, a WNETSIM part, and an ITG part. 243 Fig. 2 illustrates the internal configuration and operation flow 244 of the proposed co-simulation platform. It also shows how 245 these parts relate to and integrate with each other.    from the virtual UAV module of the UAVSIM part, and the 298 distance between UAV and GCS is obtained and sent to the 299 WNETSIM part. In the P_Mgt module, which is constructed 300 with UDP sockets and queues, the FC-packets received from 301 the UAVSIM part are forwarded to the P_Sync module, and 302 then the FC-packets are re-transmitted from the P_Sync mod-303 ule and sent to the GCS module or the virtual UAV module 304 in the UAVSIM part. Since wireless network environment is 305 not taken into account in the FC-packet transmission between 306 UAV and GCS in the UAV simulation, it is desirable to reflect 307 the wireless network state while transmitting the FC-packets 308 to perform the UAV simulation under a virtual wireless net-309 work environment. For this purpose, the re-transmission of 310 FC-packets in the P_Sync module is determined by networks 311 parameters, which are periodically updated with the simula-312 tion results obtained from the WNETSIM part. In general, synchronization scheme has significant impacts 316 on the performance of the co-simulation platform. However, 317 conventional co-simulation platforms have inefficient syn-318 chronization schemes. Because data packets are transmitted 319 through a single channel and are sequentially processed in 320 the conventional synchronization schemes, the NS-3-based 321 network simulation stage is always executed before the packet 322 management stage as shown in Fig. 1. This makes it impos-323 sible to precisely synchronize the UAV simulation with the 324 were introduced to develop the virtual environment mod-359 ule, the virtual UAV module, and the GCS module, respec-360 tively. The WNS module was constructed using NS3.34 to 361 support the Wi-Fi 802.11a/b/g/n/ac wireless network environ-362 ment. In addition, MAVSDK [24] was adopted to transmit the 363 LO-packet in the Info-Ext module. The other modules in this 364 paper were implemented using C++ with QT5. In particular, 365 the P_Sync module was implemented based on the multi-366 threads to support concurrent queuing of FC-packets in the 367 P-Mgt modules. The graphical user interface (GUI) module in 368 Fig. 4 consists of six panels. These panels visualize the UAV 369 flight and the network parameters generated by the WNS 370 module, monitor the status of NS-3 and PX4 SITL, interface 371 with the GCS, and set the simulation parameters.

373
In order to verify the proposed co-simulation platform, 374 we carried out some experiments based on an Intel 375 Core i7-3770 3.40 GHz CPU, DDR3 12 GB RAM, NVIDIA 376 GTX 1060, and Ubuntu 20.04 LTS. In these experiments, 377 three test scenarios were applied to evaluate the performance, 378 effectiveness, and extendibility of the proposed co-simulation 379 platform. In order to validate the functionalities of the proposed 383 co-simulation platform, we set up the 1 st test scenario in 384 which a single UAV flies over a square-shaped flight path 385 and the proposed platform simultaneously performs UAV 386 simulation and wireless network simulation. In this exper-387 iment, adjacent waypoints were spaced 200 m apart and 388 TxPower, default delay, and data rate were set to 40dbm, 389 0µs, and 10Mbps, respectively. In addition, two propagation 390 loss models, Logdistance and Frils, were applied to the Wi-Fi 391 VOLUME 10, 2022  It is meaningful to investigate the wireless network simula-409 tion of the proposed platform in detail according to the Wi-Fi 410 model. Fig. 6 shows the network parameters generated by 411 co-simulation, in which the 1 st test scenario of Fig.5 is applied 412 to the Wi-Fi 802.11g model and the Wi-Fi 802.11ac model. 413 In Fig. 6, the Wi-Fi 802.11g model and the Wi-Fi 802.11ac 414 model are represented by blue and red lines, respectively. 415 Fig. 6 (a) shows the distance change between the GCS and the 416 UAV where the two lines completely overlap as the flight path 417 are the same. As the distance between the GCS and the UAV 418 increases, Fig. 6 (b) shows how the packet transmission time 419 increases for each Wi-Fi model. In addition, Fig. 6 (c) and (d) 420 illustrate how the amount of data transmission per second and 421  This clearly shows that the wireless network simulation 444 works correctly on the proposed platform.

445
Since co-simulation performance typically depends on the 446 synchronization scheme, it is desirable to observe the effect of 447 parallel processing applied on the synchronization scheme of 448 the proposed co-simulation platform. Based on the 2 nd test 449 scenario where the UAV maintains a constant position at 450 a fixed distance from the GCS, the processing time delay d p 451 of the proposed platform was compared to that of FlyNetSim. 452 Fig. 8 shows the processing time delays measured in the 453 proposed platform and FlyNetSim for a sequence of packets 454 transmitted from the UAV to GCS. The average processing 455 time delays of the proposed platform and FlyNetSim are 456 0.186ms and 2.617ms, respectively. It means that the process-457 ing time was saved by approximately 92% due to the synchro-458 nization scheme of the proposed platform. This clearly shows 459 that the two-channel approach with parallel processing of the 460 proposed platform is more effective for UAV simulation than 461 a single-channel approach.   for UAV-related simulations and researches [14], [17], [19].

472
In this experiment, co-simulation was performed according to 473 the 1st test scenario and simulation conditions of Fig. 5 (a).

474
Comparing Fig. 9 with Fig. 5 (a), it can be easily seen that  In order to further examine the extendibility of the pro-480 posed platform, the proposed platform was applied to the 481 development of UAV-related control system. In this experi-482 ment, QgroundControl, which was the GCS in the proposed 483 platform, was replaced with a customized GCS implemented 484 with MATLAB, and a co-simulation was carried out to track 485 a simple guidance command of an UAV. For this simulation, 486 we set up the 3 rd test scenario in which the UAV makes a turn-487 ing flight with a radius of 50m by receiving a velocity com-488 mand from the customized GCS. Let p be a two-dimensional 489 location vector of UAV transmitted to the GCS. Then, the 490 velocity command generated by the GCS is formulated as 491 follows: where r and ω denote the desired turning radius and the angu-494 lar velocity vector, respectively. Note that p and v(x, y) are 495 transmitted between UAV and GCS during the turning flight. 496 Fig. 10 represents co-simulation results on the proposed plat-497 form for the 3 rd test scenario and the data rate of 4Mbps. 498 From Fig. 10, we can identity that the flight path of the 499  application. For this reason, co-simulation platforms based on 516 a synchronization scheme have been suggested as a solution 517 to this problem. However, the conventional co-simulation 518 platforms still are unsuitable for performing simultaneous 519 UAV simulation and wireless network simulation due to crit-520 ical shortcomings.

521
In this paper, we propose an advanced co-simulation plat-522 form capable of verifying and evaluating UAV-based appli-523 cations under a virtual wireless network environment. Based 524 on the analysis of the conventional co-simulation platforms, 525 we have derived a list of improvements for the conventional 526 co-simulation platform and developed the proposed platform 527 consisting of UAVSIM, WNETSIM, and ITG parts. In the 528 proposed platform, a two-channel approach was applied to 529 support various types of missions and associated UAV flights 530 and to parallelize synchronization. From the experimental 531 results for three test scenarios, it has been shown that the 532 VOLUME 10, 2022 proposed platform efficiently provides the functionalities 533 required for co-simulation based on efficient interoperation 534 between the three parts. In addition, the processing time 535 delay in the wireless network simulation was reduced by 536 approximately 92% due to the synchronization scheme based 537 on the two-channel approach. This allows the co-simulation 538 to be performed more precisely on the proposed platform.

539
Moreover, the proposed platform can support various simula-540 tions because it is extendible due to its modularized software 541 architecture.