HAP-Enabled Communications in Rural Areas: When Diverse Services Meet Inadequate Communication Infrastructures

The high altitude platform (HAP) network has been regarded as a cost-efficient solution for providing network access to rural or remote areas. Apart from network connectivity, rural areas are predicted to have demands for diverse real-time intelligent communication services, such as smart agriculture and digital forestry. The effectiveness of real-time decision-making applications depends on the timely updating of sensing data measurements used in generating decisions. As a performance metric capable of quantifying the freshness of transmitted information, the age of information (AoI) can evaluate the freshness-aware performance of the process of updating sensory data. However, unlike urban areas, the available communication resources in rural areas may not allow for maintaining dedicated infrastructures for different types of services, e.g., conventional non-freshness-aware services and freshness-aware real-time services, thereby requiring the proper resource allocation among different services. In this article, we first introduce the anticipated services and discuss the advances of rural networks. Next, a case study on the efficient resource allocation across heterogeneous services characterized by AoI and data rate in HAP networks is presented. We also explore the potential of employing the free-space optical (FSO) backhaul framework to enhance the performance of multi-layer HAP networks. To strike a balance between the AoI and data rate, we develop both static and deep reinforcement learning (DRL)-based dynamic resource allocation schemes to allocate the communication resources provided by HAP networks. The simulation results show that the proposed dynamic DRL-based method outperforms the heuristic algorithm and can surpass the performance ceiling achieved by the proposed static allocation scheme. In particular, our presented method can improve performance by nearly 2.5 times more than the ant colony optimization (ACO) method in terms of weighted sum performance improvements. Some insights on system design and promising future research directions are also given.


I. INTRODUCTION
W HILE the urban communication networks have been continuously developed to cope with the growth of data traffic, it is estimated that there are roughly 34% of the world's population lives in the rural and remote areas without Internet coverage at all [1].The potential revenues from the sparsely populated areas place restrictions on investments in rural communication infrastructures by the profit-oriented network operators.Due to their potential for providing cost-effective connectivity to large geographic areas, HAP networks have attracted increasing attention from both academia and industry [2], [3], [4].For instance, one HAP at 20 km altitude can addressing the connectivity issue for a tens-of-kilometers-radius area.
In contrast to the urban area, the rural residential areas are located far away from one another and isolated by land-based economic areas (e.g., agriculture and forestry).Therefore, the majority of communication devices, as well as service demands, in the residential and economic areas are naturally different.In general, the communication devices can be categorized into machine-type communication (MTC) devices and human-type communication (HTC) devices.Although the density of users in the rural areas is smaller than that in the urban areas, the digitalization transformation in the rural areas is not being left behind [5], [6], [7].The anticipated digitalization trends will bring diverse demands for information and communication services to rural areas, including: • Real-time communication: Real-time communication services are crucial for a diverse set of services in rural areas, such as emergency response, eHealth, and teleeducation.For instance, remote and rural areas may lack medical professionals, and eHealth can provide a high-quality medical consultation and diagnosis.• Internet access: Internet access plays a key role in improving the economic development of rural areas and enriching the quality of life for rural residents.The Internet can serve as a powerful tool for expanding e-commerce opportunities, fostering greater social inclusion, and unlocking new opportunities for entrepreneurship and innovation.• Intelligent information processing: Intelligent information processing can improve efficiency and productivity in various domains, such as digital agriculture and forestry.Generally, digital agriculture and forestry involves making intelligent actions based on the sensing data.The success of intelligent information processing depends on the effectiveness and quality of the data gathered from Internet of Things (IoT) devices • Cloud-based services: Cloud-based services can provide cost-effective and flexible solutions for rural areas, such as cloud storage and software as a service (SaaS).Cloud-based services can also enable rural areas to access cutting-edge technologies that may be otherwise unavailable or unaffordable.Intuitively, the QoS for the above services cannot be measured by the same performance metric.For example, the QoS of the real-time communications service is affected by the latency and throughput.Round-trip time (RTT) is a common performance indicator used to evaluate the QoS for surfing Web pages.As for intelligent information processing systems, the AoI is an important metric to evaluate the freshness of data available to the decision controller.AoI was first proposed in [8], it refers to the time elapsed since the generation of the freshest update packet among the packets that have been received by the monitor node.For example, in the digital agriculture, a monitoring system with a low AoI can provide farmers with timely and accurate information of soil moisture levels, temperature, and other environmental factors, allowing them to make precise decisions about irrigation, fertilization, and other farming practices.

A. ENABLING TECHNOLOGIES FOR CLOSING THE DIGITAL GAP IN RURAL AREAS 1) COVERAGE ENHANCEMENT IN RURAL AREAS
Providing reliable and high-quality coverage in rural areas is challenging due to the characteristics of environmental factors.HAP networks are a promising solution for addressing this issue.However, single-layer HAP networks may experience severe path loss in their long-range single-hop links, thereby rendering low coverage and connectivity.With the placement of unmanned aerial vehicle (UAV)-mounted base stations (BSs) or terrestrial BSs (TBSs), multi-layer HAP networks can convert long-range links into short-range multihop links, which can reduce overall propagation delay and improve coverage performance.Moreover, by utilizing the inter/intra-layer link, multi-layer HAP network has the potential to achieve excellent load distribution, reduce the overall transmission energy consumption, and increase the access data rate [9].The operating time of conventional UAV is restricted by the capacity of its on-board battery.By introducing a physical link from the ground power station to the UAV, tethered UAVs have a stable power supply.This allows them to hover at a desired position and keep their transceiver active for a significantly longer time.In comparison to conventional untethered UAVs, tethered UAVs offer advantages in terms of providing reliable long-term communication services [10].

2) BACKHAUL ENHANCEMENT IN RURAL AREAS
In order to guarantee connectivity for the rural areas, the backhaul network should be capable of dealing with the resource limitations constraints, such as the lack of electricity and the difficulty for fiber deployment.While optical-fiber plays an important role in current commercial networks, wireless backhaul is predicated to account for the majority of global backhaul links [11].Specifically, by partitioning the total radio resource into two parts for wireless backhaul and access, the integrated access and backhaul (IAB) technique enables network operators to increase BS density without the expensive cost of deploying optical-fiber.By leasing spectrum resources, operators can build the RF backhaul networks to provide the services over the air.The continued usage of RF backhaul will require the evolution toward higher frequency waves to support wider channels, such as millimeter and terahertz waves.In comparison to the RF links, FSO communication links can reach higher capacity performance without spectrum licensing requirements.However, the signal transmissions of FSO links are vulnerable to environmental factors, which include turbulence loss, absorption loss, and scattering loss [12].

3) COMPUTING CAPACITY ENHANCEMENT IN RURAL AREAS
Edge cloud computing is an effective approach for enhancing computing capacity in rural areas.By migrating some or all data processing tasks of mobile applications from mobile devices with limited resources to cloud servers, computation offloading can significantly decrease the energy consumed by mobile devices.Implementing edge computing servers on aerial and terrestrial vehicles allows users in rural areas to perform highly demanding computing tasks within strict delay constraints [13].In addition, enabling caching on cloud servers can help reduce latency in rural areas by storing frequently accessed data closer to users.This approach can be particularly effective when combined with edge computing, as it allows data to be processed and delivered more quickly.

4) POWER SUPPLY ENHANCEMENT IN RURAL AREAS
Rural areas commonly experience difficulties in obtaining dependable and reasonably priced electricity, yet the importance of having reliable power sources to facilitate communication cannot be overstated.In this regard, wind, solar, and RF energy harvesting are viable options for enhancing power supply in rural areas.These renewable energy sources can provide a reliable and sustainable source of power for wireless communication equipment, even in areas where access to the grid is limited.Wind turbines can be installed in rural areas to generate electricity, which can then be stored in batteries for use during periods of low wind activity [14].Similarly, solar panels can be installed to capture sunlight and convert it into electricity, which can also be stored for later use.

B. MOTIVATIONS AND CONTRIBUTIONS
Unlike the urban networks with abundant communication infrastructures, all services provisioning in rural areas rely on the limited available communication resources.Consequently, insufficient communication resources may not allow for the maintenance of dedicated infrastructures for different services in rural areas.In this context, designing and optimizing wireless networks while disregarding competition for limited resources by multiple services may not be applicable to rural areas in practice.To provide diverse services with different QoS requirements in rural areas, flexible and efficient allocation of communication resources is necessary.In this article, we explore the potential of deploying HAP networks to meet the heterogeneous service demand from rural areas.After introducing several enabling technologies for rural wireless networks, we present a brief review of studies on the design of resource allocation and IoT networks in the rural areas.Then, we investigate not only static but also dynamic resource allocation schemes for rural HAP communication systems.Our main contributions are summarized as follows.
• We study a practical scenario where the HAP network needs to simultaneously fulfill conventional nonfreshness-aware and freshness-aware real-time service requests from devices in rural areas.Specifically, the QoS of these two services are measured by data rate and AoI, respectively.
• In addition to radio frequency (RF) links, we explore the usage of FSO links to improve the backhaul capacity of HAP networks.Apart from the single-layer HAP network, we also consider multi-layer HAP networks with the aforementioned backhaul frameworks.We obtain several system design insights by comparing the data rate and AoI performance of different architecturebackhaul framework pairs.• We design both low-complexity static and DRL-based dynamic mechanisms to allocate the insufficient communication resources of the HAP network with heterogeneous service requests.Simulation results reveal that the DRL-based dynamic resource allocation mechanism achieves significant performance improvement comparing with the static mechanism and baseline methods.

II. A BRIEF OVERVIEW OF RELATED WORK A. RESOURCE ALLOCATION IN RURAL AREAS
The issue of resource allocation in rural areas has been studied from various angles.The authors in [15] proposed a channel assignment and routing algorithm to maximize the network throughput in rural mesh networks.In [16], a heuristic algorithm with polynomial time complexity was proposed to minimize the deployment cost in long-range wireless mesh networks under the throughput constraint.
Different from [15] and [16] that focus on static channel assignment strategies, the authors in [17] developed a congestion-aware dynamic channel allocation and routing protocol with the objective of maximizing the throughout.In [18], Alonso et al. presented a framework to identify the minimum number of BSs required to satisfy the convergence requirement, and addressed the problem of minimizing power consumption.The authors in [19] addressed the joint access channel assignment and backhaul topology formulation optimization problem for wireless heterogeneous mesh networks in rural areas.In [20], the authors first formulated a novel indicator to measure the telecommunications service imbalance and then developed an optimal BS placement strategy to minimize their proposed imbalance index.By incorporating UAVs into rural wireless networks, the authors in [21] solved the problem of minimizing deployment costs for multi-UAV networks with backhaul capacity and user coverage requirements.
In [22], the authors investigated the effect of drone charging stations on the coverage performance of UAV-assisted rural wireless networks.However, the aforementioned works only focus on conventional non-freshness-aware performance metrics, which overlook the emerging trend of freshnessaware sensing and automatic IoT applications in rural areas.

B. IOT IN RURAL AREAS
The authors in [23] studied the coverage performance of deploying IoT networks in rural areas based on Longterm evolution (LTE) and low power wide area networking (LPWAN) technologies.In [24], the authors proposed a cross-layer scheduling solution to improve the delay and throughput of the rural IoT networks.The authors in [25] proposed a HAP-aided IoT network architecture and investigated several potential applications.In [26], a novel design of the system configuration was proposed to improve the coverage performance of massive IoT networks in rural areas.The authors in [27] addressed the joint deployment and resource allocation problems for HAP-aided IoT network so as to maximize the overall system throughput.Timely status updating is indispensable for many real-time IoT applications in the rural areas.Apart from the weather forecasting and climate monitoring systems, the advent of data-driven agriculture has enabled more IoT sensor data from rural areas.The transmission of sensory data needs to be timely with the objective of expanding the IoT into smart processing capabilities.AoI is a widely used indicator for measuring the freshness of information packet transmission [28], [29], [30].To optimize the freshness of information transmission in satellite-based IoT networks, a hybrid automatic repeat request (HARQ) transmission protocol with the objective of reducing AoI was proposed in [31].In [32], the authors considered applying non-orthogonal multiple access (NOMA) technique to satellite-terrestrial networks and proposed a coalition formation game based algorithm to minimize the AoI.However, the above studies solely focus on one performance indicator and neglect the fact that the insufficient infrastructure in rural areas is shared between different services.

III. CASE STUDY: EFFICIENT RESOURCE ALLOCATION IN RURAL AREAS WITH MULTIPLE SERVICE DEMANDS
In this section, we consider a rural network consisting of one HAP, M MTC small cells, and H HTC small cells.In the following, we use 0, {1, . . ., M}, and {M + 1, . . ., M + H} to index the HAP, MTC small cells, and HTC small cells.It is assumed that there are multiple machine-type devices (MTDs) and multiple human-type devices (HTDs) randomly distributed in MTC small cells and HTC small cells, respectively.The HAP has requests for the freshness-aware information generated by the group of MTDs M, and can provide conventional Internet access services to the set of HTDs H.The numbers of total MTDs and HTDs are N M = |M| and N H = |H|, respectively.We assume that time is slotted and the length of each time slot is .To ensure the freshness of the received update packets at the HAP, MTDs must frequently generate and transmit the status update packets of their sensing tasks.The update packets are the time-stamped measurements.The considered communication system has three architecture variants, depending on whether UAVs or TBSs are deployed to assist the transmissions from terrestrial devices to the HAP.In particular, each small-cell is served by at most one TBS or UAV.As shown in Fig. 1, the network architectures include: • TBS: The terrestrial devices first forward their data packets to the TBSs.The TBSs act as the relay to assist the transmission from terrestrial device to the HAP; • LAP: In this architecture, unlike deploying TBSs on the ground, the data transmission from the terrestrial devices to the HAP is assisted by the low-altitude UAVs; • HAP: The terrestrial devices transmit their data packets to the HAP directly.While the access links mainly focus on RF communication due to the limitation of the mobile devices, the backhaul links can be implemented in FSO by placing the optical transceiver on the BSs.FSO links are strongly depending on the wavelength of the beam and the weather conditions, such as clouds, fog, rain, and snow.The horizontal FSO links have certain distance limitations that prevent them from being the reliable long-distance backhauling solution, while the FSO link above the clouds is less affected by the weather conditions and can accomplish 384 Mbps data rate over 500 km link distance [3].Another challenge for using FSO links is the pointing error.Boresight and jitter are two widely conditioned components for the pointing error.The former one is representing the fixed displacement between the center of the beam and the detector, and the latter one is the random offset of the beam center at the detector plane.In order to successfully build up an FSO communication link, it is necessary to perform accurate pointing for the narrow optical beam.According to [12], the transmission capacity from the transmitter i to the receiver j is given by where L i,j poi (t) and L atm i,j (t) respectively denote the pointing error loss and the atmospheric attenuation loss, P i is the transmitting power of node i, η i and η j are the optical efficiency of FSO transceiver at the node i to the node j, α j and α B i,j are the area of the FSO receiver and the received beam at node j, E p is the photon energy, and N b is receiver sensitivity.
The transmission capacity from the transmitter i to the receiver j over the RF link is where W i,j , g i,j (t), L i,j , and σ 0 are the channel bandwidth, the channel power gain, the path loss, and the power density of additive white Gaussian noise (AWGN), respectively.For the TBS-HAP and LAP-HAP architectures, both HTDs and MTDs select their target BSs based on the maximum received power rule.For the sake of simplicity, round-robin (RR) scheduling protocol is used in the access links.Let f (j, t) denote the number of devices associated with the BS j, 1 ≤ j ≤ M + H, in time-slot t, the achievable uplink data rate for the access link from device k ∈ M ∪ H to BS j is given by We introduce a variable χ i (t) ∈ [0, 1] to indicate the timeresource ratio allocated to the i-th small-cell.Depending on whether RF or FSO is used, according to (1) and ( 2), the transmission rate of the backhaul link from the BS j and the access link from the device k in the j-th small-cell to the HAP can be expressed as and According to (3)-( 5), the achievable end-to-end (E2E) data rate from the device k ∈ M ∪ H to the HAP in time-slot t is where β(k) denotes the index of the BS selected by the device k.
The performance of sensing tasks conducted by MTDs is measured by AoI.Unlike data rate, which measures the amount of data received per unit of time, AoI represents the duration since the creation of the most recent update packet among those received by the monitoring node.A smaller AoI is better, as it indicates more timely and accurate tracking of source status from the monitor.Formally, the AoI of the task associated with MTD m (m ∈ M) in time-slot t is defined as in which where U m (t) is the time-stamp of the most recent update from the MTD m and o m (n) denotes the time-stamp when the n-th update packet was generated at the MTD m.The transmission time of the n-th update packet generated from the MTD k, i.e., T m (n), is given by where D m denotes the data size of the packet generated by the MTD m.

IV. STATIC AND DYNAMIC TIME-RESOURCE ALLOCATION A. STATIC TIME-RESOURCE ALLOCATION
To develop a resource allocation scheme that effectively caters to multiple service demands with the available communication resources, one intuitive solution is to implement a priority-based network scheduler that allocates resources based on the importance and urgency of each service demand.The priority-based network scheduler is a static and low complexity solution that is suitable for implementation in rural areas with limited processing capacity.In practice, it may also be beneficial to consider the spatial and temporal variations in service priorities.For example, certain services may have higher demand during certain hours of the day or in certain geographic locations.By accounting for these variations, a more efficient and effective resource allocation scheme can be developed.The static resource allocation scheme is often used in wireless networks to allocate resources to different types of small cells.In this scheme, a time fraction η M of the backhaul resource pool is reserved for MTC small cells, while the remaining part is allocated to HTC small cells.It is worth noting that η M is fixed and does not change over time.As a result, the time-resource ratios allocated to the m-th MTC small-cell and the h-th HTD small-cell are χ m = η M /M and χ h = (1 − η M )/H, respectively.

B. DYNAMIC TIME-RESOURCE ALLOCATION
For the dynamic scheme, we assume that the time-resource ratio allocated to each small-cell is able to change over time.
In this context, the applications of DRL have shown success in solving long-term dynamic resource allocation problems in wireless communications [33].Rather than focusing solely on short-term performance improvements, DRL naturally takes into consideration the long-term evolution of performance when making decisions, which is essential for time-variant wireless communication systems.Moreover, DRL benefits from the use of deep neural networks (DNNs), allowing it to handle large-scale decision-making problems [34], [35], [36].
Our goal is maximizing the weighted sum of the performance improvements for the AoI and data rate over time.The performance improvements of AoI and data rate are defined as and in which and where A req and R req represent the performance requirements for AoI and data rate, respectively.In particular, we consider the following two dynamic schemes: • Identical Dynamic Scheduling (IDS): The time-resource ratio allocated to the same type of small-cell is the same.For example, the time-resource ratios for all MTC small cells are the same, while time-resource ratios of a HTC small-cell and a MTC small-cell can be different; • Non-identical Dynamic Scheduling (NIDS): The timeresource ratios allocated to different small cells of the same type can be different.Let ω A and ω R denote the weights associated with the performance improvement of AoI and data rate, the IDS and NIDS optimization problems can be formulated as follows and By modelling the resource allocation problems as Markov decision processes (MDPs), we can train intelligent agents to learn the optimal allocation policies.An MDP consists of a set of states S, a set of actions A, and a transition function f (a|s) that describes the probability of taking a particular action a ∈ A given the state s ∈ S. Additionally, it includes a reward function r(s, a) that determines the reward value of the state-action pair (s, a).Recent developments in using neural networks as function approximators have led to significant empirical progress.Specifically, we can use deep neural networks (DNNs) to predict the expected rewards from a particular state and a policy, which are respectively referred to as the value network and policy network.For instance, the state s can be defined as the QoS of multiple services demands along with the channel information in the rural areas, the reward is determined by the QoS of different services, and the action a is constructed by the resource allocation profile across services.As shown in Fig. 2, the policy and value network are updated using the set of tuples stored in the experience buffer with the format {s, a, r(s, a), s next }.
In order to obtain the optimal dynamic allocation polices for aforementioned dynamic schemes, we first need to model the optimization problems to MDPs.The state s is defined as the achievable uplink data rates of HTDs, AoI of the MTDs, and the channel power gains of all the links in the network.For the IDS scheme, the action a is defined as the time-resource ratio allocated to MTC small cells.In the NIDS scheme, since we assume that the time-resource ratio for different small cells can be different, the action a is constructed by the time-resource ratios for different small cells.The reward is defined as the weighted sum of performance improvements We use the proximal policy optimization (PPO) algorithm [37] to solve the dynamic resource allocation problems.PPO is an actor-critic DRL algorithm that consists of a policy network π θ and a value network V φ (s), where θ and φ are the sets of parameters of the DNNs.The policy network π θ and the value network V φ are updated by minimizing the following loss functions via mini-batch stochastic gradient descent (SGD) method [37]: and where N B denotes the mini-batch size, the hyper-parameter is used to prevent θ new dramatically change from θ old , and γ ∈ (0, 1) is the discount factor.The operation clip(x, [x min , x max ]) returns the input value x if it falls within the range from x min to x max , and returns x min if x is less than x min , or x max if x is greater than x max .Âπ θ (s t , a t ) is estimated by a modified generalized advantage estimator (GAE) [34] as follows (19) where 0 ≤ ζ ≤ 1 is the trace-decay parameter.

V. PERFORMANCE EVALUATION
In this section, we will present the numerical results of the HAP communication system which covers a rural area with 20 km radius.The locations of the HTDs and MTDs are modeled as two independent Matérn cluster point processes, the projection of the TBS or UAV on the ground is the center of the corresponding cluster.The carrier frequency in all RF backhaul links is 38 GHz, which is based on  the International Telecommunication Union (ITU) Radio Regulations as defined in F.2439-0.The data size of the sensory data is 1 MB, and is 1 second.We set the values for AoI and data rate as A req = 10 seconds and R req = 1 Mbps, respectively.The remaining simulation parameters are according to [12], which is summarized in Table 1.For the dynamic allocation algorithms, the simulation is built using Python and the deep leaning framework from PyTorch.The list of hyper-parameters for DRL is given in Table 2. Fig. 3 depicts the impact of static time fraction for MTC (η M ) on the average AoI and data rate for different combinations of network architectures and backhaul frameworks.For all considered scenarios, the performance of both AoI and data rate improve as the amount of resources allocated to MTDs or HTDs increases.We can observe from Fig. 3 that the average data rate of HTDs increases linearly with η M , while Fig. 3 indicates that the average AoI of MTDs is a non-linear function of η M .In this context, the optimization of the resource allocation across the considered performance metrics is expected to take into account the different behaviors of the metrics with respect to η M .For the TBS and LAP architecture, FSO backhaul-based networks significantly outperform RF backhaul-based networks.This is reasonable since the FSO links can achieve much higher transmission data rate when the weather condition and the pointing loss meet the requirement.Compared to the HAP architecture, the TBS or the LAP architecture with the RF backhaul framework can achieve at least 22.5% improvement in AoI and 12.9% improvement in data rate, respectively.Moreover, due to the advantage of providing better coverage, it is shown in Fig. 3 that LAP architecture can achieve better performance than the TBS architecture.In Fig. 4, we plot the average worst AoI of MTDs and the average worst data rate of HTDs.The average worst AoI of MTDs and the average worst data rate of HTDs are defined as ( T t=1 min m∈M A m (t))/T and ( T t=1 min h∈H R h (t))/T, respectively.Similar to the findings in Fig. 3, LAP-FSO achieves the best performance.It is worth noting that the performance gap between the TBS and LAP is more significant in terms of the average worst AoI and data rate than that of the average AoI and data rate.This further implies the advantage of deploying UAVs to improve the performance of HAP networks in rural areas.Fig. 5 shows the average AoI and transmission rate obtained by DRL methods under IDS and NIDS schemes with different weight settings.In specific, we consider three settings: w (1) = {ω M = 1, ω H = 10}, the weight associated with the performance improvement of data rate is larger; w (2) = {ω M = 1, ω H = 1}, the weights of two items are equal; w (3) = {ω M = 10, ω H = 1}: the weight associated with the performance improvement of AoI is larger.We can observe from Fig. 5 that the performance of AoI or data rate improves as the value of corresponding weight increases.This indicates that the proposed methods can be applied into design of networks with customed performance requirements.It is clear that the NIDS schemes outperforms IDS schemes.This due to the fact NIDS can accomplish more precise communication resources allocation over time.In addition, the DRL-based dynamic approach can boost performance ceilings achieved by the static scheme and strike a better balance between different metrics.For instance, as shown in Fig. 3, the average AoI of the static scheduling scheme for LAP-FSO can reach nearly 3 seconds at η M = 0.4, while the corresponding data rate is below 9 Mbps.According to the result of LAP-FSO from Fig. 5, the DRL framework can simultaneously achieve the AoI of 2.6 seconds and the data rate of 11 Mbps under the weight w (1) setting.As for the worst AoI and data rate, Fig. 6 demonstrates that the formulated DRL framework can improve the performance in most cases.However, recall that our reward function focuses on the average AoI and data rate, the better-worse performance relationship depicted in Fig. 6 may not always align with that of Fig. 5.For example, the average worst AoI of the TBS-FSO is larger than that of LAP in Fig. 6, while the opposite relationship is observed in Fig. 5.For this reason, a proper reward function design is necessary to fulfill the specific objectives of network design.
To evaluate the effectiveness of our proposed DRL framework, we compare it with the following algorithms under the weight w (1) setting: • Random: The time-resource allocation ratios allocated to all small cells are randomly generated; • Ant colony optimization (ACO) [38]: ACO is a heuristic algorithm inspired by the behavior of ants searching for food.We refer to ACO based algorithms for the IDS and NIDS optimization problems discussed in Section IV-B as ACO-IDS and ACO-NIDS, respectively.In Fig. 7, we compare the performance of our proposed algorithm with the aforementioned methods in terms of the weighted sum of performance improvements, average AoI, and average data rate, under different network architectures.Fig. 7(a) demonstrates that the DRL framework can achieve performance improvements of 20% to 251% and 67% to 244% compared to ACO and Random, respectively.This verifies the advantage of DRL in solving the dynamic resource allocation problems under the time-variant network.As shown in Fig. 7(b) and Fig. 7(c), although the data rate of ACO is slightly better than that of the DRL method, the DRL method can dramatically improve the performance in terms of the average AoI.In this context, for the given requirements A req and R req , the performance balance between the AoI and data rate of the DRL framework is superior to that of the baseline ACO algorithm and the Random scheme.

VI. FUTURE RESEARCH DIRECTIONS
While the previous sections discussed the issues regarding promising diverse communication services in rural areas with limited infrastructure, there are still some challenges and opportunities that deserve further investigation in rural wireless networks.

A. DEPLOYMENT DESIGN
In comparison to the terrestrial communication infrastructures, HAP can be deployed in a more flexible way, since it does not rely on the geographical condition.For the rural communications systems which include both MTCs and HTCs, the deployment design should consider the heterogeneity of the different performance metrics.The optimal placement of the available aerial and terrestrial BSs in multilayer HAP networks needs further investigation.Apart from the communication infrastructures, the placement design of the service caching and storage servers should also be taken into consideration to provide efficient computing and storage resources for the rural areas.

B. TRADE-OFF BETWEEN AOI AND OTHER METRICS
In spite of the transmission rate, some other well-known performance metrics, such as coverage probability and energy efficiency, are worth being investigated along with the AoI.With the developments of the techniques and the increase in the number of devices, the wireless network will convey multiple and diverse services.To cope with the diverse QoS requirements, the design of the networks should pay extra attention to the trade-off among multiple performance indicators.For example, due to the energy limitations, the design of UAV networks should also consider the energy efficiency at the drone BSs.The optimal design to deal with the trade-off between AoI and other performance metrics is a critical demand for wireless systems including but not limited to MTDs.

C. LIGHTWEIGHT AND DISTRIBUTED SCHEDULING
One of the key challenges for wireless networks in rural areas is the effective management of network resources in a distributed manner.This may include the use of distributed multi-agent reinforcement learning algorithms for network scheduling, and the development of new protocols for efficient communication and coordination between network nodes.Lightweight distributed control methods can help rural wireless networks to ensure higher performance and reliability, which are crucial to cope with the increasing demand for wireless services in these areas.

D. INTELLIGENT CONTROL AND MANAGEMENT
HAP networks are more dynamic than terrestrial networks, since aerial BSs may intermittently leave pre-designed locations to recharge their energy resources.The dynamic network topology inevitably affects the system performances, especially for the freshness-aware metrics of the MTCs in the rural areas.In this regard, the intelligent hand-off management solution is essential to ensure robust performance.Utilizing machine learning-based approaches is able to predict the mobility of network entities and design an intelligent network controller.

VII. CONCLUSION
This article studies the envisioned services in rural areas with insufficient communication resources supported by HAP networks.We provide a brief summary of the advances in HAP-enabled networks and highlight the promising services in rural areas.Efficient resource allocation across envisioned services is essential in rural areas due to the lack of communication infrastructure.We discuss the potential of both static and DRL-based dynamic resource allocation strategies with the demand for diverse services in rural areas.Specifically, we present a case study on the resource allocation between the AoI of MTDs and the data rate of HTDs under different combinations of backhaul frameworks and network architectures.In our considered system setup, the terrestrial devices consist of both MTDs and HTDs.The MTDs devices have tasks to monitor several physical processes and generate the sensory data which is requested by the HAP, while the HTDs have the request for Internet access services that are not demanding on freshness.Our results concretely demonstrate that the best network architecture is using UAVs to relay the information packets from terrestrial devices to HAP under FSO based backhaul links.The numerical results prove that the optimization of the resource allocation policy has the potential to meet the heterogeneous service demand.Finally, we conclude by presenting several feasible future directions.

FIGURE 1 .
FIGURE 1.The considered scenarios and a toy example of time-resource allocation decision: one HTC small-cell with three HTDs and one MTC small-cell has two MTDs.

FIGURE 2 .
FIGURE 2. An illustration of DRL for resource allocation across multiple services.

FIGURE 3 .
FIGURE 3. Impact of ηM on average AoI and data rate.

FIGURE 4 .
FIGURE 4. Impact of ηM on average worst AoI and data rate.

FIGURE 5 .
FIGURE 5. Comparison of AoI and data rate for different weight settings.

FIGURE 6 .
FIGURE 6.Comparison of the worst AoI and data rate for different weight settings.

FIGURE 7 .
FIGURE 7. Performance comparison of different algorithms.