Intelligent Hello Dissemination Model for FANET Routing Protocols

The routing mechanisms in flying ad-hoc networks (FANETs) using unmanned aerial vehicles (UAVs) have been a challenging issue for many reasons, such as its high speed and different directions of use. In FANETs, the routing protocols send hello messages periodically for the maintenance of routes. However, the hello messages that are sent in the network increase the bandwidth wastage on some occasions and the excessive number of hello messages can also cause the problem of energy loss. Scarce works deal with the problem of excessive hello messages in dynamic UAVs scenarios, and treat several other problems, such as bandwidth and energy wastage simultaneously. Generally, the existing solutions configure the hello interval to an excessive long or short time period originating delay in neighbors discovery. Thus, a self-acting approach is necessary for calculating the exact number of hello messages with the aim to reduce the bandwidth wastage of the network and the energy loss; this approach needs to be low complex in terms of computational resource consumption. In order to solve this problem, an intelligent Hello dissemination model, AI-Hello, based on reinforcement learning algorithms, that adapts the hello message interval scheme is proposed to produce a dense reward structure, and facilitating the network learning. Experimental results, considering FANET dynamic scenarios of high speed range with 40 UAVs, show that the proposed method implemented in two widely adopted routing protocols (AODV and OLSR) saved 30.86% and 27.57% of the energy consumption in comparison to the original AODV and OLSR protocols, respectively. Furthermore, our proposal reached better network performance results in relation to the state-of-the-art methods that are implemented in the same protocols, considering parameters, such as routing overhead, packet delivery ratio, throughput and delay.


I. INTRODUCTION
C URRENTLY, unmanned aerial vehicles (UAVs) have been used extensively in several applications [1]- [4], in which either human intervention is not required or required at some extent [5]. However, some problems must be avoided such as the energy loss which is considered as one of the major concerns in UAVs networks [6]. In [7], an adaptive hello messaging scheme was proposed for saving energy however, the study focused only on energy consumption without considering the network throughput. The UAVs routing protocols face other problems, for instance, the periodic exchange of hello messages for neighbor detection and maintenance. This occurs because the UAVs are required to exchange data packets frequently with each other. Additionally, the link breakage recovery, the scalability and allocated bandwidth are other problems that must be dealt accordingly [8]. The EE-Hello scheme is proposed in [9], reducing the energy utilized by the UAVs, but the proposed method depended on many characteristics that must be configured manually, for example, the numbers of UAVs. Therefore, there is a lack of works that propose an automatic solution for treating the diverse problems found in UAV scenarios.
Traditional hello messaging studies [10], [11] present different problems to configure an adaptive time interval for generating hello messages. This fact can cause unnecessary bandwidth usage and energy consumption in Flying Ad-Hoc Networks (FANETs). Moreover, the hello interval can lead to a trade-off in which a shorter interval facilitates the quick detection of new neighbours or current link breaks, but conversely, it produces higher overhead and consumes more energy. In the case a longer interval of the hello messages is set up, the overhead and energy consumption are reduced, but it limits neighbor discovery and link break detection capability. Thus, an optimized and dynamic hello interval is essential for FANETs.
The low-altitude drones [12] are commonly accomplished with sensors, cameras, on-board monitors and other devices for communication systems [13] all these devices consumed the energy of drones. Therefore, studies focused on producing energy-efficient green UAVs [14], [15]. These studies also focused on the requirements which UAVs need to maintain their communication links. Thus, the network connectivity has become a critical issue in UAVs which is leading towards the field of FANETs research [16], [17].
In all existing networks, a communication protocol cannot generate excessive overhead [18]; thus, in FANETs this fact has to be avoided for not consuming more energy than the requirements [19]. Another aspect is that the property of detecting the neighbor UAVs [20] in ad-hoc mode has to be studied and explored by avoiding a large energy drain [21]. Currently, few works [22] focus on the energy drain problems for traditional routing protocols in the context of FANET.
Commonly, UAV can quickly move or change directions in a FANET [23] which affects the route maintenance [24]. This fact can cause different throughputs and delays in data dissemination. Thus, it is relevant to consider the threedimensional (3D) topology in UAV [25]. In other situations, the devices can create disconnected UAV groups or totally disconnected groups [26]. Thus, for establishing a route, the UAV needs to discover its neighbors through sending hello messages or other feedback mechanism for the communication links [27].
For the neighbor detection or maintenance, a signaling exchange of hello messages is sent by the routing protocol [28]. In Mobile Ad-Hoc Networks (MANETs) [29], many hello messages are sent for determining the dynamic network topology which requires that all the devices should exchange hello messages or beacons periodically when they are in ON mode [30]. However, many times no adaptive interval conditions are defined [10], that cause unnecessary bandwidth usage and excessive energy consumption. A shorter interval can facilitate a quick detection of neighbors or new link breaks [27], while it can also produce a higher overhead and consumes more energy [31]. Thus, it is very useful to determine an optimized hello interval which can dynamically adapt in different circumstances within FANETs. Currently, scarce works [9] study this problem but no one uses an intelligent and automatic solution for disseminating the hello messages in FANETs.
In [32], an innovative dynamic neighborhood-based algorithm for the broadcast storm problem (DNA-BSP) is proposed, in which it can mitigate the broadcast storm problem in FANETs which reduces the message redundancy by more than 98%, and additionally makes the message delivery superior to 95% faster in comparison to a flooding scenario. Thus, it outperforms the broadcast storm mitigation techniques. However, the power consumption problem is not part of its proposal scope.
Recent studies have used deep learning mechanisms [33]- [35] including convolution neural networks and long shortterm memory to improve the performance of the network layer [36], [37]. However, they are complex systems that do not reduce in the energy consumption, and either they do not include a mechanism to reduce the hello dissemination in UAVs. For reducing the energy consumption traditional solutions such as Dijkstra's shortest path algorithm [38], are being used for FANETs [39]. However, in [39], the proposal is not a dynamic and robust solution.
In this paper, we propose an intelligent adaptive hello interval algorithm based on Artificial Intelligence (AI) that generates a power saving solution. The AI algorithm calculates the network density based on a network history and it is automatically adapted in accordance to the changes red in the network. Our proposed method for determining the hello interval uses the information regarding permitted airspace, number of UAVs, transmission and speed range while an intelligent timeout timer value is calculated for better network throughput.
In the simulation tests, a 3D space scenario [40], [41] is used such as a practical situation, and the results showed that the proposed algorithm decreased the energy consumption by reducing unnecessary overheads [42], without having degradation in network throughput.

A. CONTRIBUTIONS OF THE PRESENT STUDY
In this paper, we propose an intelligent hello dissemination model based in the context of FANET routing protocols, named AI-Hello, to address the drawbacks previously described for existing position-based routing protocols in FANETs [43]. Our proposed model seeks the neighbors path between the source and destination by considering a reduced number of hello messages. For adapting to a fast change in topology, we proposed an adaptive and intelligent technique, where the learning rate and reward factor parameters, [44] are automatically adjusted based on the network conditions. In dynamic networks, the topology changes rapidly [45]. Thus, a high learning rate focuses more on new information, due to this reason it is better that the adaptive mechanism could be dynamic and not fixed. In this context, the number of hello messages are reduced and consequently, the energy consumption is decreased. Thus, in this paper different factors are studied, such as the dynamic network scenarios of UAVs and energy consumption by using lightweight models of deep learning.
The main contributions of this paper are summarized as in the following: • An Intelligent and adaptive Hello dissemination model based on Reinforcement Learning, called AI-Hello, which is able to suppress unnecessary hello messages without degrading the overall network throughput. Thus, the proposed model is able to improve the network performance conditions, in terms of increasing the throughput and packet delivery ratio while reducing the end-to-end delay, routing overhead and energy consumption. • The proposed AI-Hello model is obtained by considering different UAV characteristics in the context of FANET routing protocols, reaching a good performance in high-density acceleration scenarios.
In order to compare the results, the AI-Hello is implemented in both AODV and OLSR routing protocols. Later, these implementations are compared with different and widely adopted versions or solutions implemented in the same routing protocols. Furthermore, different FANET characteristics are considered in the test scenarios. In general, experimental results show that our proposed implementation obtained the best network performance results including energy consumption in relation to state-of-the-art methods, for instance, the proposed AI-Hello model implemented in AODV and OLSR protocols reduced by 30.87% and 27.57%. the energy consumption, FANET dynamic scenarios of high speed range, in relation to the native AODV and OLSR protocols. The proposed model presents a good performance by using appropriate task representation with suitable initial Qvalues that produces a dense reward structure for facilitating the network learning.

B. OUTLINE OF THE PAPER
The remainder of this paper is structured as follows. We describe the related work in Section II. Section III illustrates the proposed Hello dissemination model in the context of FANET. The methodology is presented in Section IV. In Section V, performance of the proposed algorithm is evaluated which is based on simulation results through metrics of network overhead, network throughput, end-to-end delay, and energy consumption. Finally, Section VI summarizes our conclusions.

II. RELATED WORKS
Several schemes have been proposed for determining the hello interval for the traditional MANET routing protocols. In this section, some of the main ideas of related works are discussed.
An adaptive hello messaging scheme was proposed in [7] for saving energy and suppressing the unnecessary hello messages. In [7], if a node does not participate of an event for a period of time, then it is not necessary to maintain the status of the link and the hello messages are suppressed. However, in FANETs there are some situations in which a number of UAVs do not participate in the communication but they move away from each other. Thus, the dynamic nature of FANETs requires knowledge of the entire network to route packets. In the case two non-communicating UAVs get silent for a period of time then neighbors will not recognize each other. Another problem occurs due to their high speed which they maintain in some cases for instance false links, generating an unnecessary overhead and spending energy to detect the real links.
In [29], a scheme for determining the hello interval based on the impact of node speed and transmission range spent on the hello interval for MANETs was investigated. However, other factors must be considered in FANETs, such as the airspace and the number of UAVs [9].
In [32], a dynamic neighborhood-based algorithm for the broadcast storm problem is proposed, called DNA-BSP, using outdoor experiments and computer simulations. The algorithm can mitigate the broadcast storm problem in FANETs, and it reduced the message redundancy by more than 98%, and the message delivery presented values superior to 95%. The authors studied in depth the problem of the broadcast storms in FANETs. However, their analysis lacks the energy-saving metric because the power consumption is a stochastic process, and for this reason it is not treated.
The EE-Hello scheme for determining an adaptive hello interval in FANETs was proposed in [9], which is able to save about 25% of the energy currently used, because it suppresses unnecessary hello messages and the overall network throughput is not impaired. However, in [9], the total amount of energy consumed by all the UAVs were measured, except individual energy. Moreover, in [9] the method is not automatized for sending the Hello messages, depending on other conditions such as the learned behavior, variation in the number of UAVs, malfunctions and other factors that will be addressed in our work.
Other works that do not consider the hello messages are proposed. In [46], a routing protocol was proposed which included two ways of routing, the delivering packets of data between vehicles with the help of UAVs using a protocol named VRU_vu, and the routing packet of data using a protocol named VRU_u. Results showed that the protocol [46] decreased end-to-end delay by an average of 13% and overhead by 40%. However, the saving energy of the devices is not treated.
A Q-learning-based [47] fuzzy logic was proposed in [39] for the FANET routing protocol. The proposed algorithm facilitates the selection of the routing paths in which an optimal routing path to the destination was determined by each UAV using a fuzzy system with link and path-level parameters. The link-level parameters use the transmission rate, energy state, and flight status between neighbor UAVs, and the path-level parameters use the hop count and successful packet delivery time. The reinforcement learning method was used for updating the path-level parameters [48]. The results showed that the method maintained low hop count and energy consumption as well as prolonged the network lifetime. VOLUME 4, 2016 However, important parameters, such as packet delivery and throughput are not measured.
A Q-learning-based topology-aware routing (QTAR) protocol was proposed in [22] for FANETs to provide a reliable combinations between the source and destination. The QTAR improves the routing decision by considering twohop neighbor nodes in which those decisions are dynamically adjusted according to the network condition. Results revealed that QTAR outclassed the existing routing protocols with respect to various performance metrics under distinct scenarios. However, the study did not deal the problem of energy consumption for FANETs.
Therefore, there are several works that consider the dynamic topology of UAV and disseminate hello messages in an automatic and intelligent way. Thus, there are scarce studies that work on the network throughput, and power consumption, which dynamically generate the adjustment of the hello interval in FANETs.

III. THE PROPOSED MODEL
In this work, we propose a new model based on the Deep Deterministic Policy Gradient (DDPG) [49], [50], the AI-Hello, which is a deep reinforcement learning approach for generating the hello messages.
The proposed DDPG refers to an improved version of the actor-critic algorithm, using the DNN for choosing the best value of the policy function instead of finding the action which is based on a specific distribution. Calculating the better Hello messages in the routing protocol, the AI-Hello manages continuously the energy in network communication.
The model dynamically generates the adjustment of the hello interval and also the timeout period with respect to the individual UAVs and the network. Thus, in the case of a highspeed UAV the model generates small values for the hello interval which reflects the changing of the network topology. In the case of a low-speed UAV, a higher value of the hello interval is generated. In these situations, the link changes occur frequently for an adequately-dense network. Whereas, the link changes are infrequent for a low-density network, decreasing the hello interval. The kernel of the DDPG code can be found in [51].
The new model is applied in a FANET as shown in Fig. 1. According to Fig. 1, the network ranges the topology (1) and generates a new traffic matrix [52] in (2), then a preprocessing is generated in (3), which reduces the number of parameters of the network. Furthermore, data are processed through the 2-D convolution layer [53] and the DDPG agent generates a new model, AI-Hello, in (4). Thus, a decision of the hello interval is defined in (5). Finally, an action in the network regarding the hello interval is performed that is represented by (6). Periodically, the traffic data is directed to the DDPG for finding better link weights and minimizing the end-to-end delay in the network, in accordance to the state vector and the hello messages.
As far as the processing of traffic data is concerned, it uses a convolution layer with the 2-D Convolution. A matrix represents the network traffic usually which summarizes the intensity of the number of packets for each source and destination. Traffic features are extracted from the matrix in order to reduce the size of input for the DDPG using convolution layer. The output vector represents the state vector of the network for generating the interval of hello messages.
Thus, the DDPG in the network represents the intelligent packet routing through the hello messages to maximize the network performance. The DDPG agent obtains the network state for determining the hello interval. Thus, a set of link weight [l1, l2, . . . ln] is determined for the state vector which is based on the hello messages. The paths are calculated, and the routing protocol acts to generate new paths. Though, the optimized hello interval discovers the route path and the performance of the FANET network is also optimized.
The pseudocode for the proposed DDPG is shown in Algorithm 1 in which an ensemble of P Q-Functions randomly and independently is used for reducing the variance in the Q-Function. The algorithm works with three hyper parameters, G, P , and Q. In which, G represents the training sample length, P represents the epoch length, and Q is the number of distinct indices used by Q-table. The number of epochs is important to run and train the DDPG agents,while the actors model is related with the training sample length.
The network conditions are carried into n variables with the transmission range T x , the allowed airspace V M , the number of UAVs U u , and the speed ranges v of the devices.
In the Algorithm 1, initialize the approximative Q-table characterized by variable θ, the buffer D with , reward r t , network state equal to s t and the new state s t+1 , F ← F ∪{(s t , a t , r t , s t+1 )}, and the mini-batch M . By the end the Hello interval is calculated according to the policy's updating gradient ∇ θ .
In the training process of the DDPG, the parameters of the critic net to approach the Q-table and respective parameters of the actor net with the aim to train the policy are updated continuously. The time complexity for the state normalization is N (s), in which N (s) represents the number of the variables for the state set. The space complexity is calculated according to the number of variables in the state in which the algorithm records the means and standard deviations for avoiding repeated calculation. Thus, the experience replay buffer occupies some space to store the state sets in DDPG, hence, the space complexity is N.

Algorithm 1 Proposed DDPG Algorithm
UPDATE the HELLO interval

IV. METHODOLOGY
In this section, the simulation parameters value are described. The proposed model the AI-Hello which is tested in terms of energy consumption, overhead requirements, network throughput, and network trade-offs. The model was applied into two routing protocols, the Ad-hoc On Demand Distance Vector (AODV) that is a reactive protocol, and the Optimized Link State Routing Protocol (OLSR) that is a proactive protocol.
In this research work some hello schemes are used for comparison typically well known schemes [54], [55], the adaptive hello messaging scheme [7], the Park's adaptive hello scheme [29], the EE-Hello scheme [9] and our proposed model. The parameter values of these schemes are chosen according to the related problems and they provide better results.

A. SIMULATION ENVIRONMENT
In the simulation step, the network simulator NS-3 version 3.30 is used. A python code called"main.py" runs the proposed model DDPG through the "multi-run.cc" script on the NS-3, as explained in [51].
The scenario of simulations is in the area of 600 m × 600 m, in which the UAVs are free to fly into the airspace of 600 m × 600 m × 150 m. Fixed-wing UAVs are considered, with minimum and maximum air speed constraints. In a step of 0.5 s a UAV could be changed its speed by using a maximum acceleration and deceleration of 5 m/s 2 and 7 m/s 2 , respectively. The horizontal direction could be changed by a maximum of 6.3 degrees and in the vertical direction by a maximum of 3.15 degrees. On the other hand, movements of the devices use Gauss-Markov 3D mobility model in the same scenario which is presented in Fig. 2, with a length of 600m and width of 600m and with swinging UAV height, h. The UAVs move at a random speeds in a range of   Table 1 presents more details about the simulation scenarios, in which 50 independent simulations are used.

B. PROPOSED MODEL CONFIGURATION
Through the experiments many values of the hyperparameters are tested and the variable Q = 4 presents a better functioning as the variables P = 1600 and G = 150.
It is important to note that the chosen hyperparameters helped into the exploration strategy of the environment.
For the DDPG, the training sample number is represented by 150 with length of the training epochs set to 1600, replay VOLUME 4, 2016 buffer's capacity set to 40 000 and the size of the mini-batch equal to 90. The learning rate of the actor and critic net are set to be 4×10 −4 . The discount factor γ is set to 0.909, target subnet τ is 0.01, noise's decay factor κ is equal to 0.999, and scale factors for normalization λ 1 = 120 and λ 2 = 4.
The learning rates of the actor and critic net are both 4 × 10 −4 and discount factor γ is set to 0.9 in Q-table.

C. PERFORMANCE METRICS
In performance metrics, the Packet Delivery Ratio (PDR) is used for determining the effect of the proposed model on the network. The PDR is defined as the following.

P DR =
Rec_Data_P ackets Sent_Data_P ackets (1) The number of Constant Bit Rate (CBR) flows, called CBRn, are varied to verify the network performance. Thus, the throughput per CBR flow (T /CBR) is used instead of conventional network throughput where the total network throughput T is used by CBRn.
The Network Overhead is also used for measuring the performance of the protocols. It is calculated through the difference between the amount of data transmitted (KB) that is known and controlled by network configuration, and the amount of data received (KB), considering a lossless transmission network.
For comparing the energy consumption, the total amount of energy consumed by all the UAVs is used. There are many methods for measuring the energy consumption [56], [57]. According to related studies [7], the energy consumption for each byte of overhead transmission is set to 200 µW and for the reception it is set to 150 µW.
According to [7], the energy consumption measured is represented by the sum of energy consumed (E y ) in data reception (R x ), Data transmission (T x ), idle and information sensing states. Therefore, for each state the energy consumed is calculated as: Where, p y represents the power dissipated in state y, and t y represents the time spent in state y. Thus, the state y can be T x , R x , idle and sensing. The value of p y is specific to each sensor.
All the bytes of data involved in receiving (Rx) or transmitting (Tx) information, such as bytes of payload, packet header, and trailer, including the sensing phase, IDLE times, transmission of acknowledgments are accounted for obtaining the total consumed energy. Thus, the total consumed energy, derived from the counted bytes, is measured through the Network Interface Card (NIC) of each device, and the energy consumed of each device is summed to obtain the total consumed energy.
The end-to-end delay is also used to measure the performance of all protocols. It is calculated through the time required to transmit a packet all over a network from the source towards the destination node [46]. Thus, the end-toend delay is calculated as: where p i represents the arrived packet, and T A (p i ) and T D(p i ) represent the arrival time of packet p i and the delivery time of p i , respectively.

V. RESULTS AND DISCUSSION
In this section, the main results about the impact of the model in the network density, network overhead, throughput, overhead efficiency, and energy consumption are shown and discussed.

A. NETWORK DENSITY
In this work, the PDR is measured in the routing protocols using our proposed model AI-Hello and the other models. Fig. 3 and Fig. 4 show the measure of the PDR for AODV and OLSR, respectively using with and without our proposed model, and considering U u =20 and U u =40.
It can be noted that the proposed model AI-Hello is applied to AODV and OLSR has a better performance in comparison with the other protocols. The proposed model AODV presents low PDR even with the increased acceleration of UAVs for random speeds in a range [5s] m/s, where speed takes values from the set 10, 15,20,25,30,35,40,45,50 and with two populations of UAVs of 20 and 40. The AODV-Park also presents high values of PDR. However, the number of dropped packets is superior to the values obtained by the proposed solution. The good performance of the AI-Hello for the PDR is due to the correct value of the hello messages performed by the reinforcement learning algorithm.
For OLSR, in general the PDR is higher than the AODV protocol because of the acceleration of the devices. The AODV presents better performance for high mobility of the devices. Results show that the proposed model works for both routing protocols. Fig. 6 show the network overhead behavior for the different protocols and models implemented in the test scenarios. The highest overhead reduction is reached by our proposed model.

Fig. 5 and
The adaptive hello messaging scheme proposed in [7], the AODV-Hans causes a pause in the scenario until the UAV to receive new messages. However, in some situations, the UAVs remain highly active and trying to maintain the neighbourhoods. Whereas the proposed model saves a small but significant amount of overhead when compared to others because unnecessary hello messages are avoided. In the case of the AODV-Park and OLSR-Park the overhead is worst in the high-density scenarios because it sends hello messages at two second intervals and the UAVs' speed range increases with the overhead consumption caused by the large numbers of link changes. Conversely, the proposed model and the EE-Hello use a higher interval, generating lower overhead. Thus, the EE-Hello scheme shows promising results for AODV and OLSR protocols, in which the hello interval reduces the overhead in terms of UAVs scenario. However, it produces higher overhead than the proposed model.
The EE-Hello scheme presents good results because it determines the appropriate distance at which hello messages transmit periodically. However, the automatic solution used by our proposed model presents the lowest values of overhead in the network.

C. THROUGHPUT PER CBR FLOW
In Fig. 7 and Fig. 8, we can observe that all the schemes present similar trends with respect to the T /CBR for AODV and OLSR. The proposed model identifies the link changes more quickly than the other schemes and obtains good values of T /CBR. The initial Q-values are identified by the proposed algorithm which produces a dense reward structure as well as facilitates to identify the link changes. Whereas, there are delays in the link changes by other models specifically VOLUME 4, 2016  with the AODV or OLSR and AODV-Hans or OLSR-Hans model.
In the experiments, random sources and destinations of CBR are chosen among all UAVs. Thus, each UAV become a potential traffic source or a traffic destination. The number of source and destination, and the pair among them are set randomly and automatically.
In comparison to AODV protocol, the proposed method presents an increase of the throughput around 11,22% for the scenario of 40 UAVs and a speed range of 5-50. In comparison to OLSR protocol, the proposed method presents an increase of the throughput around 18,96% for the scenario of 40 UAVs and a speed range of 5-50.
Results of all schemes are similar to some extent for the AODV protocol. However, improvements due to the proposed model are more evident for the OLSR protocol.

D. ENERGY CONSUMPTION
In this research work, the total amount of energy consumed by all the UAVs per second is calculated. Thus, the final result of energy consumed represents the value spent by all the UAVs and not a single UAV. From Fig. 9 and Fig. 10 can be noted that the proposed model consumes a significantly smaller amount of energy than the other models used for comparison purposes. Although the proposed model is based on reinforcement learning algorithms, there is no higher   energy consumption than other models proposed in related works. This fact occurs because the proposed algorithm is tractable for using the appropriate task representation with suitable initial Q-values, which produces a dense reward structure and facilitating the learning. Hence, the proposed model present excellent characteristics for energy efficient green UAVs. The lower energy consumption is observed in both AODV and OLSR routing protocols.
In Fig. 9, the results of two network scenarios for the AODV protocol are depicted, in which U u =20 and U u =40 are considered. In the case of U u =20 and for the speed range of 5-50, the proposed model presents an energy consumption of 11.9 J and the AODV presents an energy consumption of 16 J; thus, the proposed model presents a reduction of 25.62% of energy consumption. These values show how much of the energy consumption was reduced. In the case of U u =40, the percentage of energy consumption reduction was 30.86% . Similarly, the results corresponding to the OLSR protocol are presented in Fig. 10. The percentages of energy consumption reduction were 25.01% and 27.57% for U u =20 and U u =40, respectively.
It is worth noting that in other speed ranges of the UAVs, the energy consumption reduction also occurs for both routing protocols (AODV and OLSR). Furthermore, results show that the proposed model can save a substantial amount of energy without degrading the key network performance parameters.
Additionally, the processing power for running the pro-   posed model is measured and compared to the other models. Fig. 11 presents the experimental results regarding the processing power analysis, considering a network scenario with four UAVs, the implementation of both routing protocols AODV and OLSR, and a speed range of 5-10 m/s. The simulation time length was until 400 minutes. It is important to note that a simple scenario configuration is used only to show the complexity of the models. As can be observed from Fig. 11, the proposed model presents low values of consumed energy because the algorithm facilities the network learning by decreasing its complexity. The sum of energy spent by each UAV was measured, in which the total value of processing power has a relation with the number of Hello messages generated by each algorithm. For this reason, our proposed method presents low energy consumption results in comparison to the others, even compared to the AODV protocol. Table 2 and Table 3 present the variation rate of the end-toend delay of the protocols used in the test scenarios taking as comparison reference the AODV and OLSR protocols, respectively. In all these scenarios 20 and 40 UAVs were considered. For better visualization of the end-to-end delay, only the scenario of the speed range of 5-50 is presented, because it represents the highest difference in the end-toend delay compared to the other protocols. The other speed range scenarios presented small differences in relation to other protocols, and they are not presented. It can be noted that the proposed method, AI-Hello, presents better results of decreasing the end-to-end delay for both reference protocols, reaching a reduction of the end-to-end delay around 17.6% and 19.0% in comparison to AODV and OLSR, respectively.

VI. CONCLUSION
In this research paper, an intelligent adaptive hello interval algorithm, called AI-Hello, based is proposed to deal with the problem of high energy consumption in FANETs. It is important to note that the proposed model can be implemented into existing protocols or it can be added as an independent module, permitting an easy implementation. Furthermore, it is important to note that the proposed model automatically adapts the hello messages according to the changes on the network. In FANETs network conditions can change very fast; then, the learning rate and reward factor parameters of the proposed adaptive model based on reinforcement learning algorithm are automatically adjusted according to the network conditions. In simulation experiments, the proposed AI-Hello model is implemented into two default protocols AODV and OLSR. For performance comparison purposes other state-of-the art-methods or schemes are implemented in the same routing protocols. Experimental results show that the proposed model reduces the network overhead and the energy consumption getting better performance than other existing schemes. Regarding the energy consumption that is one of the most critical parameters in FANET context, our proposal obtained a reduction between 25% to 30% in relation to both AODV and OLSR protocols, considering high speeds, outperforming the other methods implemented in this work. In future works, we intend to test the proposed model in other routing protocols and simulate different FANET scenarios.
As future works, authors pretend to test diverse scenarios, with a higher number of UAVs. Additionally, other routing protocols will be explored and compared to our proposed method, using network models different from the lossless model. Currently, his main research interests are related to deterministic and stochastic combinatorial optimization problems applied to a diverse range of engineering domains including wireless communications, signal processing, network design, and energy problems.
DICK CARRILLO received the B.Eng. degree in electronics and electrical engineering from San Marcos National University, Lima, Perú, and the M.Sc. degree in electrical engineering from Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro, Brazil, in 2004 and 2008, respectively. Since 2018 he is a researcher at Lappeenranta-Lahti University of Technology, where he is also pursuing the Ph.D degree in electrical engineering. His research interests are mobile technologies beyond 5G, energy harvesting, intelligent meta-surfaces, and Cell-free mMIMO.
RENATA LOPES ROSA received the M.S. degree from the University of São Paulo in 2009 and the Ph.D. degree from the Polytechnic School of the University of São Paulo in 2015 (EPUSP). She is currently an Adjunct Professor with the Department of Computer Science, Federal University of Lavras, Brazil. She has a solid knowledge in computer science based on more than ten years of professional experience. Her current research interests include computer networks, telecommunication systems, machine learning, quality of experience of multimedia service, social networks, and recommendation systems.