RE-ORA: Residual Energy-Aware Online Random Access for Improving the Lifetime of Slotted ALOHA-Based Swarming Drone Networks

In this paper, to maximize the lifetime of drone swarms by resolving the battery exhaustion problems caused by undesired retransmissions, we propose a residual energy-aware online random access scheme (RE-ORA) by adjusting the packet transmission opportunities based on the residual energy of a drone in S-ALOHA-based swarming drone networks. We aim to improve the battery lifetime of the drone with the smallest energy among the drone swarms. In addition, we analyze the success, collision, and idle probabilities of the swarming drone networks. In particular, we analyze the intradrone swarm collision and interdrone swarm collision probabilities. Through intensive simulations, we show that the proposed RE-ORA scheme outperforms the conventional algorithm with respect to the average lifetime of drone swarms and successful packet transmission probability according to the number of drones and the residual energy.


I. INTRODUCTION
Future cellular networks aim to support explosively increasing mobile data traffic and to consider three-dimensional (3D) network coverage [1]- [4]. Recently, 3D communication networks considering drones have been widely regarded as an inevitable solution for beyond 5G (B5G) and 6G cellular networks [5]- [8]. In particular, the utilization of unmanned drones has been gradually expanded in our daily lives, such as in environmental monitoring, disaster relief support, logistic delivery, and agricultural pesticide spraying. However, one of the most severe problems encountered when utilizing drones is the extremely short flight time because of their battery constraints. When a drone swarm performs its mission, all of the drones that comprise the drone swarm work together for one mission, but the tasks of each drone may be somewhat different. Here, a 'mission' is a goal that the drone swarm must achieve, and the work each drone must do to achieve The associate editor coordinating the review of this manuscript and approving it for publication was Ton Duc Do . this mission is defined as a 'task'. That is, all drones included in the swarm should complete their tasks perfectly to finish the mission of the drone swarm. If the battery of a specific drone is exhausted relatively early, it experiences difficulty in performing its task completely. Consequently, it results in mission failure. Because the energy consumed when transmitting control and data packets is much greater than the energy consumed while sleeping, we consider the residual energy of drones as a determinant of obtaining a packet transmission opportunity. Specifically, when they send packets to a ground control station (GCS) using random access, we try to make drones with low energy quickly succeed and switch to sleep mode.
In random access-based swarming drone networks, several drones may transmit their packets to serving GCSs simultaneously to successfully access the shared wireless medium [9], [10]. The simplest and most representative random access schemes are pure ALOHA (P-ALOHA) and slotted ALOHA (S-ALOHA). In P-ALOHA, the time a drone transmits is continuous. In other words, drones send their packets whenever they want. In S-ALOHA, the drones can start transmitting their packets at the beginning of the time slot. Therefore, perfect synchronization is required in the S-ALOHA protocol. The following three channel events can occur after packet transmissions: success, collision, and idle. Additionally, we classify two types of collisions, namely, intra-and interdrone swarm collisions.
In the S-ALOHA protocol, the optimal packet transmission probability (P opt (t)) of each drone for maximizing the throughput is known as P opt (t) = 1 N act (t) [9]. Here, N act (t) is the number of active drones that have packets to send at the current time slot t. However, when considering the maximization of the lifetime of the drone swarm, 1 N act (t) might not be the optimal value. In this paper, we use this value, P opt (t), as the packet transmission probability of the conventional S-ALOHA-based random access scheme. Fig. 1 shows a detailed example of the operation of the conventional S-ALOHA scheme in the swarming drone network. There are four active drones, and the packet transmission probability of each drone (P conv (t) = P opt (t)) is 1 4 . In this figure, we assume that drone 4 has the lowest residual energy among the drone swarms. If drone 4 fails its packet transmission at the second and fourth time slots, it could experience a battery exhaustion problem due to the successive packet transmissions. Thus, if a certain drone's battery exhaustion occurs, this drone swarm has difficulty fulfillng its mission completely. From this example, we can see that reducing the energy consumption of drones with the lowest residual energy is extremely important for improving the lifetime of the drone swarm because of the battery limitation problem of the drone.
To resolve the collision problem in UAV swarms, [11] proposed a potential field method-based collision avoidance algorithm to avoid inter-UAV collisions. However, [11] enforcedly changes the trajectories of drones which possibly collide. In [12], L. Ruan et al. proposed a game theoretic framework to guarantee optimal energy-efficient coverage deployment by adjusting the location of relay UAVs and their transmission power. In addition, [13] derived a theoretical model on the propulsion energy consumption of fixed-wing UAVs as a function of the UAV's flying speed, direction and acceleration for optimizing the trajectory of UAVs.
In [14], J. Zhang et al. tried to maximize the spectrum efficiency as well as energy efficiency of the UAVenabled mobile relaying system by jointly optimizing the time allocations for relaying of UAVs together with their flying speed and trajectory. This paper addresses a new fundamental trade-off between spectrum efficiency versus energy efficiency maximization by exploiting the UAV trajectory design when considering the UAV-enabled mobile relaying with the propulsion energy. Reference [15] proposes a method for evaluating the energy consumption of the mobile base station at any point on the drone's flight path and thus selects the hovering location of minimum energy consumption as the optimal drone's location. This paper aims at maximizing the energy efficiency of relay aerial robotic networks while not reducing network throughput. In addition, in [16], S. Ahmed et al. designed energy-efficient UAV relaying communications, which jointly optimizes both throughput and UAV propulsion energy consumption. Here, an iterative algorithm was also proposed to solve the energy efficiency maximization problem. Moreover, [17] proposed a UAV-assisted adaptive-learning approach for energyefficient wireless-powered public safety IoT networks. The wireless-powered communication system is a new networking paradigm in which the battery of wireless communication devices can be remotely charged by means of microwave wireless power transfer (WPT) technology. That is, WPT is the technology that directly charges the device's battery, but we herein aim at proposing the technology to improve the lifetime of the drone swarm by adjusting the drone's transmission opportunity. Although these two technologies have clearly different research goals, these are very important technologies that can be utilized together for energy-efficient drone communications.
In swarming drone networks, adopting the optimization framework-based energy efficiency maximization scheme is almost impossible in practice due to the large computational complexity. Additionally, if we do not consider the residual energy of the drone when calculating the packet transmission probability, it may cause a significant reduction in the lifetime of the drone swarms. That is, in the case of a drone swarm, if the battery of a specific drone runs out relatively quickly, the drone will have difficulty completing its task. To perfectly fulfill the mission of the drone swarm, improving the lifetime of the drone swarm is inevitable. Therefore, to improve the battery lifetime of the drone with the smallest residual energy in the current time slot, we propose the residual energyaware online random access (RE-ORA) for S-ALOHA-based swarming drone networks.
The main contributions of this work are as follows.
• The proposed RE-ORA scheme controls the packet transmission probability by considering the residual energy of drones to improve the lifetime of the drone swarms and successful packet transmission (SPT) probability (P ST (t)). Here, P ST (t) can be calculated using where N S is the number of successfully VOLUME 9, 2021 transmitted packets and N T is the total number of transmitted packets.
• In consideration of the operational complexity of the proposed RE-ORA scheme, we utilize two kinds of static and dynamic sigmoid threshold values. In this paper, RE-ORA-S and RE-ORA-D are RE-ORA with static and dynamic threshold values, respectively. In RE-ORA-S, we use the maximum amount of energy of a drone as the sigmoid threshold value regardless of the variation in the residual energy of the drone swarm according to their transmissions. In addition, RE-ORA-D exploits the average residual energy of a drone swarm as the sigmoid threshold value to dynamically control the packet transmission opportunities of each drone.
• We mathematically analyze the success, idle, and intraand interdrone swarm collision probabilities. We compare the lifetime performance of the S-ALOHA-based swarming drone networks and the SPT probability of the proposed RE-ORA schemes (RE-ORA-S and RE-ORA-D) with those of a conventional scheme. The rest of this paper is organized as follows. In Section II, we propose a residual energy-aware online random access scheme based on a sigmoid function for improving the lifetime of swarming drone networks. In Section III, we analyze success, idle, intradrone swarm collision, and interdrone swarm collision probabilities. Moreover, the simulation results in terms of the average lifetime of drone swarms and the average SPT probability are shown in Section IV. Finally, the conclusions are drawn in Section V.

II. PROPOSED SCHEME: RESIDUAL ENERGY-AWARE ONLINE RANDOM ACCESS (RE-ORA)
In the proposed RE-ORA scheme, the packet transmission probability of drone i in state M i at time slot t (P i (t)) can be calculated by the following: where η i (t) is the residual energy of drone i at the current time slot t, η on (t) is the threshold value at the time slot t, and w is the weighting factor of a sigmoid function used to adjust the packet transmission opportunity according to the residual energy of each drone in the RE-ORA scheme.
Considering the operational complexity of the proposed RE-ORA scheme, we consider two kinds of static and dynamic η on (t) as follows.
where N tot is the total number of drones. In Equation (2), RE-ORA-S and RE-ORA-D are RE-ORA with static and dynamic η on (t), respectively. In RE-ORA-S, we use η max as η on (t) regardless of the variation in the residual energy of the drone swarm according to their transmissions. η max is  (1) to dynamically control the packet transmission opportunities of each drone. As shown in Equation (1), we utilize a sigmoid function to control the packet transmission probability for random access attempts. Fig. 2 shows the packet transmission probability versus the amount of residual energy according to the different weighting factors in the proposed RE-ORA-S scheme. As shown in Fig. 2, the lower the residual energy is, the higher the packet transmission opportunity. Drones with low residual energy have a higher probability of success in a packet transmission at an early time, and energy-saving effects can also be obtained by switching to sleep mode more quickly. In the proposed RE-ORA-S and RE-ORA-D schemes, drones with relatively lower residual energy have a chance to obtain a higher packet transmission probability as w increases. This can result in lower packet collisions and higher successful transmissions for drones with relatively lower residual energy. Consequently, the lifetime of drone swarms can be improved in the random access-based swarming drone networks. Fig. 3 shows a detailed example of the operation of the proposed RE-ORA scheme. In this figure, drone 4 with the lowest residual energy succeeds in a packet transmission in the second time slot. This drone can switch to sleep mode at the beginning of the third time slot, and this drone can save its battery until the beginning of the next frame. With regard to the weighting factor of the proposed RE-ORA scheme, the packet transmission probability for the amount of residual energy may vary slightly. As mentioned above, the proposed RE-ORA scheme determines the packet transmission probability based on the current residual energy of each drone, unlike the conventional scheme that considers only the number of active drones. If the drone with the lowest energy fails in packet transmission, it should transmit its packet again in the current frame. However, consecutive failures in packet transmissions can result in faster drone battery exhaustion. The lifetime of the corresponding drone swarm may decrease. In contrast, RE-ORA focuses on increasing the SPT probability of drones with a relatively low residual energy. Thus, these drones can reduce unnecessary energy consumption and the lifetime of the drone swarm can increase significantly. The most substantial difference between the proposed scheme and the conventional scheme is that the proposed RE-ORA scheme uses the drone's residual energy. At the beginning of each frame, the GCS transmits the drone's average residual energy, and each drone sends its residual energy at the end of each frame. This information can be simply piggybacked by using beacon and ACK packets in RE-ORA. Therefore, the required power consumption to exploit the proposed scheme is not severe compared to the conventional scheme. In other words, because the conventional algorithm needs the information for the active number of drones, it should also utilize simple handshaking as in the proposed scheme.
Algorithm 1 shows the overall operation procedures of the proposed RE-ORA-S and RE-ORA-D schemes. Also, P tx and P sp denote the power consumption of drone in transmission and sleep modes, respectively. In this algorithm, LT_Cnt denotes the lifetime of the drone swarm, and T slot is the length of the time slot. In the proposed RE-ORA scheme, P i (LT_Cnt) can be calculated by considering the residual energy of a drone. Here, P i (LT_Cnt) can be represented as P i (LT_Cnt) = 1 − Drone i transmits its packet at time slot t. Drone i defers its transmission at time slot t.
In Equation (3), N denotes a set of drones. To calculate the success, collision, and idle probabilities of the proposed RE-ORA scheme, we define the battery state i as M i , and M as the set of battery states of the drone. In addition, the number of drones in state M i is denoted as N i . By using the binomial distribution, the success probability (P S i (t)) of drone i in state M i can be represented as Next, the idle probability (P I (t)) at time slot t can be calculated as P I (t) = Pr(No drone attempts to transmit a packet) As mentioned above, the idle probability means that none of the drones included in N transmit packets. Moreover, we calculate the intradrone swarm and interdrone swarm collision probabilities of our proposed RE-ORA scheme. The In addition, the interdrone swarm collision probability of the proposed RE-ORA scheme (P IC-2 (t)) can be represented through the following: Using Equations (6) and (7), the total collision probability (P TC (t)) is given as follows: Moreover, using Equations (4) and (8), the average SPT of the proposed RE-ORA scheme (ξ (t)) can be described in the following: In summary, from Equations (4)-(7), we can compare the success and collision probabilities in accordance with the residual energy states. In addition, we can find that the smaller the residual energy is, the higher the probability of success.
In the case of an intradrone swarm collision probability, because the packet transmission probability of the drone with high residual energy is relatively smaller than that of a drone with low residual energy, the intradrone swarm collision probability is also smaller. In the proposed RE-ORA scheme, a drone with low residual energy has a higher probability of success early, and energy-saving effects can be obtained by quickly switching to sleep mode. It can be seen that the energy efficiency of the entire drone swarms can be improved by considering the residual energy. In other words, the lifetime of the drone swarms can be improved in S-ALOHA-based swarming drone networks. Fig. 4 shows the numerical results for the success, idle, and intra-and interdrone swarm collision probabilities. To obtain the numerical results, we assume four residual energy states, M = [0.25, 0.5, 0.75, 1.0]. The total number of drones is from 4 to 400, where N 1 : N 2 : N 3 : N 4 = 1 : 1 : 1 : 1.

IV. SIMULATION RESULTS
To obtain the simulation results, we considered four dif- , [19]. For example, in the case of Env1, the drone consumes 0.1 W when transmitting a packet at each time slot and 0.01 W when waiting without transmitting a packet. In addition, we assume that P max is 10 W and N tot is 40. As shown in Fig. 5, the average lifetime of drone swarms is defined as the total number of frames used by the drone swarm until the residual energy of even one of the drones reaches zero. In each frame, if a drone succeeds in packet transmission, it switches its mode into sleep until all drones included in the drone swarm succeed packet transmissions or the drone swarm completes its mission. If all drones successfully transmit packets, the corresponding frame ends and the next frame starts. When the next frame starts, all drones will be out of sleep mode.  That is, drones with relatively lower residual energy can have a chance to obtain a higher packet transmission probability as w increases. This results in lower packet collisions and a higher rate of successful transmissions for drones with relatively lower residual energy. In addition, we can see that Env1 has the greatest average lifetime among the three combinations of power consumption. When the ratio of power consumption in the transmission mode to the power consumption in sleep mode increases, the gap between the proposed RE-ORA-S scheme and the conventional scheme becomes larger. When w = 4, compared to the conventional scheme, the gain of the average lifetime in the proposed scheme is 8%, 22%, and 23% for Env1, Env2, and Env3, respectively. This is because the difference between the amount of power consumed in the transmission mode and the amount of power consumed in the sleep mode is the largest in Env3.
From Fig. 6, we can see the SPT probability, success probability, idle probability, and total collision probability versus the number of drones. As described above, as the weighting factor in RE-ORA-S w increases, the difference in transmission probabilities between drones with high and low energy increases. Thus, the collision probability decreases and, conversely, the probability of successful transmission increases. Consequently, the P ST of the proposed RE-ORA-S scheme is higher than that of the conventional scheme. That is, the greater the value of w is, the higher P ST is for the drones with lower energy. When w = 4, the proposed RE-ORA-S has 9% gains in P ST and 8% gains in P S compared to the conventional algorithm. Additionally, regardless of the number of drones, we can see that the P ST and P S in the proposed scheme are greater than that of the conventional scheme, and the P TC and the P I are smaller. Fig. 7 shows the packet transmission probability according to the variation in the average residual energy of the drone swarm when w = 7 and the number of drones is 10 in RE-ORA-D. In the legend box, each number represents the average residual energy of the drone swarm over each frame when the initial energy is 10 W. Additionally, FN means a frame number. That is, 9.946(1) denotes the average amount of the residual energy of 10 drones at the beginning of the 2nd frame. As shown in Fig. 7, we can see that the smaller the η on (t) in Equation (1) is, the smaller the number of drones that have packet transmission opportunities. We can find such a trend from Fig. 7. Therefore, by using this trend, we can see that the proposed RE-ORA-D can obtain an additional performance improvement compared to RE-ORA-S and the conventional algorithm, as shown in Fig. 8. In the case of the Env1 and the Env4, RE-ORA-D has 12% and 9% gains for the average lifetime of drone swarms compared to RE-ORA-S, respectively, because RE-ORA-D exploits the average residual energy of the drone swarm as the threshold value of Equation (1) to dynamically adjust the packet transmission opportunities of each drone. This shows that RE-ORA-D performs better than RE-ORA-S with respect to the average lifetime of drone swarms.
To show the performance results for the realistic scalability scenario in the case of Env1 and Env2, we have obtained the simulation results for up to 100 drones with respect to the average lifetime of drone swarm, as shown in Table 1.  This table shows that the proposed RE-ORA-S and RE-ORA-D schemes outperform the conventional scheme when the number of drones increases up to 100. Especially in Env2, RE-ORA-D with w = 7 achieves a 203.03% lifetime gain compared to the conventional scheme when the number of drones is 100. These results mean that the proposed scheme is scalable and could be easily deployed in practical environments.

V. CONCLUSION
This paper aims to improve the lifetime of drone swarms by controlling the packet transmission probability online when considering the amount of the residual energy of the drones. Given the battery limitation problem of a drone, maximizing the battery life of the drone with the lowest residual energy is one of the most important research items in swarming drone networks. In the proposed RE-ORA scheme, each drone determines its packet transmission probability by using a sigmoid function to generate the difference between drones with high and low residual energy. In particular, we considered the operational complexity of the proposed RE-ORA scheme with static and dynamic threshold values of the sigmoid function. In addition, we numerically analyzed the probabilities of a success, idle, and intra-and interdrone swarm collisions, and through intensive simulations, we demonstrated that our proposed RE-ORA could improve the overall lifetime of a swarming drone network. We have shown that the larger w is, the greater the lifetime of the drone swarm because the largest difference in packet transmission probabilities occurs in RE-ORA. For further work, we will take into account centralized and distributed reinforcement learning-based packet transmission probability adjustment scheme for maximizing the average lifetime of drone swarms considering air-to-ground channel model. In addition, to maximize the lifetime of drone swarms, it is expected that the proposed scheme and the WPT technology can be used together through the design of a new MAC frame structure that considers the random access phase and energy harvesting phase.