Aerial Reconfigurable Intelligent Surface-enabled URLLC UAV Systems

In this paper, we propose an aerial reconfigurable intelligent surface (RIS) system to support the stringent constraints of ultra-reliable low latency communication (URLLC). Specifically, unmanned aerial vehicles (UAVs) employed onboard RIS panels can act as repeaters to reflect the signal from macro base station (MBS) to all users in the networks. To overcome the dense networks’ interference, we propose to use zero-forcing beamforming and time division multiplexing access (TDMA) scheme where each UAV can serve a number of users in its own cluster. We formulate a optimisation framework in terms of UAVs’ deployment, power allocation at MBS, phase-shift of RIS, and blocklength of URLLC. Due to highly nonconvex and complex optimisation problem, we first consider to use a deep neural network (DNN) to solve the optimal UAVs’ deployment. Then, the optimal resource allocation is proposed to provide the maximal reliability of the considered system with respect to the users’ fairness. From the representative numerical results, our proposed scheme is shown to superior to other benchmarks which exhibits the positive impact of aerial RIS in supporting stringent demands of URLLC.


I. INTRODUCTION
Unmanned aerial vehicles (UAVs) have been widely applied in wireless communication systems as an efficient means to extend wireless coverage due to their flexible configuration and mobility nature [1], [2]. Specifically, UAVs play an important role when blockages exist between the base station (BS) and the user equipment (UE). UAVs are a brand-new technology used for surveillance, military, and even remote sensing [3]. According to previous studies, UAV-based communication networks possess outstanding ability to expand the transmit signal's coverage and improve transmission efficiency.
Reconfigurable intelligent surface (RIS) has been considered as a new technique for 6G networks. This technology is now under the spotlight because it can support energyeffective and cost-saving high data rate communication by using passive reflections [4]. RIS is an integrated panel that consists of passive reflecting elements [5] under the control of intelligent controllers and reflecting received signals toward the expected destinations. The intelligent controllers can help RIS to adapt the environment propagation, therefore, the received signals are enhanced, and the interference can be mitigated.
As a result, RIS-UAV systems have attracted much attention because of their abilities to enhance system performance [4], [6]. In [4], RIS is employed to support simultaneous wireless information and power transmission. RIS can also be employed at the cell boundary, creating a controlled propagation environment that improves user performance at the cell edge [6]. The authors formulate the weighted sum rate (WSR) maximisation problem, which is subject to the phase shift with unit modulus constraints while meeting the energy harvesting requirements. In [7], the authors analyse the performance of an integrated RIS-UAV relaying system, where the RIS is used to provide additional degrees of freedom, combined with UAVs, to improve the transmission quality between ground nodes. RIS carried by UAV approach has been proven to improve the performance of the system significantly in nonorthogonal multiple access (NOMA) [8], orthogonal frequency division multiple access (OFDMA) [9] been recognised as a completely new solution for future wireless networks. Although there is a huge potential of applying RIS with UAV in URLLC to cope with the significant increase in demand of stringent QoS services in 5G and beyond networks, the research in this direction is still in the early stage. To the best of our knowledge, only one published research utilises all advantages of RIS, UAV and URLLC [26]. The authors optimised the RIS passive beamforming antenna elements, the UAV's position and block-length to minimise the total decoding error rate. However, the RIS is deployed on buildings in [26] and the blocking effect still exists. To fill this gap, in our study, we propose a system with RIS carried by UAV, thus improving the flexibility of RIS. With the high demand of today's wireless communications, URLLC is crucial for industrial automation, the healthcare industry, vehicle safety and such scenarios. However, due to the complicated propagation environment, it is very challenging to meet the QoS requirements. The combination of UAV-URLLC can be considered as a good solution to this shortcoming. With the coverage extension provided by UAV, the whole system is more flexible and perform better in wireless communication scenarios. Moreover, UAVs are generally more cost-effective and more likely to establish line-of-sight (LoS) transmission links, which enhance communication performance. By cleverly turning the signal reflection low-cost passive reflective elements, the involvement of RIS enables the RIS-UAV-assisted URLLC system with even superior system performance. The main contributions of our paper include the following • We propose an aerial RIS-assisted URLLC system where UAVs carry RIS panels to reflect the signal from main base station to far-end users. With this strategy, the stringent QoS requirements with very high reliability and very low latency of URLLC can be satisfied. • We formulate a joint optimisation framework with respect to deployment of UAVs, the power allocation at base station, the phase shift of RIS, and the blocklength of URLLC to minimise the worst-case decoding error probability by taking into account the fairness of largescale networks. • To efficient solve the highly complex optimisation problem, we first propose a three-step deep neural network (DNN) framework including: i) training and testing, ii) building model, and iii) evaluating the solution. We demonstrate that the DNN model can predict the optimal locations of UAVs precisely and more rapidly compared to the conventional optimisation approach. • We then solve the optimal resource allocation in terms of the phase-shift, power, and blocklength to optimise the minimum error decoding probability with respect to the ultra-reliability requirement. Our proposed optimisation algorithm outperforms the conventional approaches which highlight the advantage of employing RIS onboard UAV to support URLLC.
The remainder of the paper is organised as follows. After the Introduction section, the system and channel models of RIS-UAV-assisted URLLC is presented in Section II. Specifically, in this section, we discuss the background of URLLC communications with less-than Shannon's achievable rate. Then, the UAV channel model, the transmission and beamforming design are also presented. The optimisation problem of the considered networks has been formulated in Section III. The optimal UAVs' deployment with DNN is introduced in Section IV. Then, the detailed solutions to solve the decoding error probability optimisation are provided in Section V. To illustrate the effectiveness of our optimisation framework, the simulation results are shown in Section VI. Finally, the conclusions are highlighted in Section VII.

A. SYSTEM MODEL
We consider a URLLC system where a macro base station (MBS) serves a large number of users (UEs) with stringent QoS constrains, e.g., very low error probability and nearzero latency. Due to the severe shadowing and high pathloss effect, we employ a squadron of drones, a.k.a. UAVs, with RIS on-board to extend the network coverage. In the considered system, the MBS is equipped with a MIMO array with N transmit antennas to communicate with multiple UAVs for extending network coverage. The UAVs are used as small-cell flying base stations with RIS supporting very high URLLC communications. All UAVs can be connected to the cloud via a fog computing node aiming at expanding the coverage of reliable wireless networks.
Because of large-scale heterogeneous networks, the number of UEs may be very high, even greater than the number of transmit antenna at MBS, which leads to the loss of MIMO benefits. In addition, with high chances of lineof-sight (LOS) propagation of air-to-ground channel from UAVs, the inter-cell interferences can become a dominant factor to the degradation of network performance. To overcome these problems, we apply the TDMA transmission scheme by using M timeslots for M RIS-UAV clustering.
There are M RIS-UAVs working as relay nodes, denoted as a set of M = {1, ..., M } to serve K set of UEs K = {1, ..., K}. The mth RIS-UAV will serve K m UEs in its cluster by using UAV integrated RIS platform as shown in Fig. 1. All UEs are equipped with a single antenna and randomly distributed in the entire network. In this subsection, we provide the overview of URLLC concept with the achievable rate of URLLC system (finite blocklength) under TDMA transmission scheme. Different from Shannon's capacity, for short blocklength communications, the coding rate R of the UE at a finite blocklength (b m ) in cluster m can be approximated as [27]: where p m is the transmit power from the MBS for the mth clustter, γ(p) is the signal-to-noise ratio (SNR) at the receiver, ε(p) is the decoding error probability, Q −1 is the dt, called the Q-function, and V given by From (1), the decoding error probability can be presented as follows: where and D is the size of the transmitted packet.

C. AERIAL RIS-ASSISTED COMMUNICATION MODELS
Without loss of generality, we define the locations of the MBS, the RIS-UAVs, and of all UEs as q 0 = (x 0 , y 0 , Z 0 ), q m = (x m , y m , Z m ), m ∈ M and q k = (x k , y k , 0), k ∈ K U , respectively. The antenna height of the BS and the UAV altitude are respectively denoted as Z 0 and Z m . With high probability of the LoS propagation in air-toair (ATA) links, the path loss of the link between the MBS and m-th RIS-UAV follows the free-space path loss model as [28], [29] where β 0 is the channel's power gain at reference distance d 0 and d 0,m is the distance between the MBS and the mth RIS-UAV given by The air-to-ground (ATG) channels between UAVs and UEs are also dominant by LoS propagation but they are more complex due to the effects of propagation attenuation by shadowing and blockage geometry [30]. As such, the path loss of the link between the m-th RIS-UAV and the (m, k)-th UE can be written as where η LoS and η N LoS are the average additional losses for LoS and NLoS, respectively. A = η LoS − η N LoS and B = 10α log 4πfcR m,k c +η N LoS . The distance path loss is given by where f c is carrier frequency (Hz), c is the speed of light (m/s), α ≥ 2 is the path loss exponent. The probability of LoS and NLoS is shown as where the constants a and b depend on the specific arrangement of the environment. Moreover, for the small-scale fading, the channel coefficients of the channels from MBS to m-th UAV-RIS and from the m-th UAV-RIS to the (m, k)-th UE are denoted by g 0,m ∈ C R×N and h H m,k ∈ C 1×R , respectively. Here, these small-scale fading coefficients are assumed to be independent and identically distributed (i.i.d.) random variables with zero mean and unit variance, where the (·) H represents the conjugate transpose operation, and R is the number of elements of the RIS. Next, we denote G 0,m ∈ C R×N and H H m,k ∈ C 1×R as the channel matrix from the MBS to m-th RIS-UAV and from the m-th RIS-UAV to the (m, k)-th UE in the m-th cluster, respectively. Hence, the cascaded channel matrix of the link from the MBS to the (m, k)-th UE via the m-th UAV-RIS, named F m,k ∈ C 1×N , can be expressed as where H H m,k = β m,k h H m,k ; G 0,m = β 0,m g 0,m , and Φ m = diag η 1m e jθ1m , η 2m e jθ2m , ..., η Rm e jθ Rm is the diagonal matrix of size R × R at the m-th RIS-UAV with η rm ∈ [0, 1]; θ r ∈ [0, 2π] represent reflection coefficient and the phase shift value of the r-th element at the RIS.

D. TRANSMISSION AND BEAMFORMING SCHEMES
As shown in Fig. 1, the RIS panel deployed on UAV reflects the signal from the MBS to M groups of UEs in the UAVs's coverage. The received signal at the (m, k)th UE of the m-th cluster (m-th RIS-UAV) are written as 1 where P 0 is the transmit power at the MBS, w m,k ∈ C N ×1 are transmit beamforming vector of the MBS, and s m,k is information transmitted by the MBS with ||s m,k || 2 ≤ 1,

1) MRT approach
In this subsection, we apply MRT for MIMO beamforming design with the beamforming vector as where 0 < p m,k < 1 is the power coefficient control of the MBS for k-th UE in m-th RIS-UAV. Then, the decoding error probability of the (m, k)th UE for downlink transmission with TDMA scheme can be presented as where and 2) ZF approach In this subsection, we first seek the beamforming vector w m = [w m,1 , ...,w m,Km ] ∈ C N ×Km in the class of zeroforcing (ZF) as follows. Let us denotē where We normalize ZF beamforming vector asw m,k / w m,k , and seek w m,k in the class of where 0 < p m,k < 1 is the power coefficient control of the MBS for k-th UE in m-th UAV. Then, the decoding error probability of the (m, k)-th UE with ZF can be presented as where and is the SNR at the UE with ZF beamforming 2 .
Since the decoding error probability with Q function is highly complex, it is profoundly difficult to solve the reliability optimisation directly with the original form. Therefore, we apply the linear approximation ε ≈ E[E(γ)] [19], [20], [31], [32] as where Then, by applying the above approximation, the decoding error probability can be evaluated as From [32], the cumulative distribution function is defined as Finally, combining (22) and (23) together, the point-to-point decoding error probability is written as

III. PROBLEM FORMULATION
Our target is to minimise decoding error probability under blocklength and power allocation optimisation for URLLCenabled UAV system with RIS. Due to a highly complex structure of interference in the SNR formula of MRT beamforming, for the sake of simplicity, we only consider the case 2 Compared with the MRT approach in (10), the intra-cell interference can be removed by using ZF beamformer. of ZF beamforming in this paper. As such, the corresponding decoding error probability minimisation problem for ZF approach is formulated as min p,b,q max k=1,...,Km ε m,k , ∀m ∈ M, ∀k ∈ K, (25a) where BL max is maximum blocklength of the MBS. The constraints (25b) represent the latency requirement. The constraints (25c), and (25d) are the power requirement at the MBS. The constraint (25e) means the blockength is integer number. The constraints (25f), (25g) are the UAVs-UEs clustering problem.
Since the decoding error probability with Q function is highly complex, it is profoundly difficult to solve the reliability optimisation directly with the original form. In this paper, we divide the optimisation problem into the two subproblems. We first deal with the optimal UAVs deployment and then the optimal resource allocation for minimising the error probability.

IV. PROPOSED SOLUTION FOR UAV DEPLOYMENT
In this subsection, we address the problem of optimal UAV deployment before solving the decoding error probability optimisation problem. The UAV deployment procedure actually is a clustering problem to form a network of RIS-UAVs and UEs that satisfies the constraints (25f) and (25g). The objective of this clustering problem is to optimise the position of RIS-UAVs so that RIS-UAVs can serve as many UEs as possible in their coverage range. Based on this concept, we formulate an mixed (binary) integer programming problem for the RIS-UAVs deployment.
We consider the maximum coverage area of an RIS-UAV by defining a circular disc of radius D m,cov . This value is calculated based on the altitude of the m-th RIS-UAV as follows where θ is set to 42.44 • [33]. Then the (m, k)-th UE belongs to the m-th cluster served by the m-th RIS-UAV if the Euclidean distance between the k-th UE and the m-th RIS-UAV (d m,k ) is lower or equal to the maximum coverage distance D max m,cov , which can be expressed as

VOLUME xxxx, xxxx
where D max m,cov = H max m / tan(θ). We then introduce a binary variable φ m,k = {0, 1} representing that the k-th UE and the m-th RIS-UAV belong to the same cluster m-th or not.
Finally, the clustering problem can be expressed as follows: where λ m is chosen as a specific corresponding value to maximize network coverage area of the m-th RIS-UAV (e.g., λ m > (D max m,cov ) 2 ), [q min m , q max m ] are the lower and upper bounds of the horizontal and vertical range of RIS-UAVs, respectively.
As we can observe that the problem (28) is a mixed-integer quadratic programming, which belongs to combinatorial (or discrete) optimisation problem. This kind of problem is often very difficult to solve in real-time. Although there are some well-known packages such as CVX, CVXPY can solve the problem by using appropriate solvers, solving this conventional problem takes time, especially in large-scale scenarios. Therefore, we follow the same approach as in [19], [34]- [36] by proposing a learning-aid solution using DNN to solve this problem rapidly. Our proposed DNN-based solution for RIS-UAVs deployment is displayed as Fig. 2. In particular, this method includes three main steps: (i) formulating the optimisation problem, (ii) generating data for training and testing, (iii) building the DNN model and evaluating the solution. The problem formulation in the first step is fully discussed as above in this section and problem is finally presented as (28).
For the second step, we repeatedly solve the problem (28) to generate sufficiently large data for the training phase. Firstly, we create around 200 system model scenarios with random locations of RIS-UAVs and UEs. Then we implement a CVXPY function in Python for solving the problem (28) in order to obtain the optimal values of RIS-UAVs' location for all generated scenarios. After knowing these optimal location of RIS-UAVs, we save these value as the output of the DNN model that is designed as follows in the third step.
The last step is designing a DNN model for finding the optimal locations of RIS-UAVs rapidly. Our DNN model consists of three hidden layers with the size of 200, 80, 80 neurons, respectively. The neural network is fully connected, and uses "sigmoid" function for hidden layer, while the output layer is "linear" activation function to provide the predicted value of the RIS-UAVs' locations. The model is compiled by a gradient-based optimisation technique named RMSprop with the mean absolute error (MAE) metric. After that, we use the dataset created in the second step for training and creating the DNN model, which can predict the optimal locations exactly and rapidly compared to the conventional optimisation solution. The effectiveness of our DNN-based method is presented in Table 1 in the simulation results section.

V. PROPOSED SOLUTION FOR DECODING ERROR PROBABILITY OPTIMISATION
To solve the optimisation problem (25), we proposed an iterative algorithm, which is fully presented in Algorithm 1. The basic concept is that at the i-th iteration, we solve the problem (25) with fixed blocklength b Obtain the current optimal power (p * (i+1) mk ) 5: Find optimal blocklength (b * (i+1) m m are chosen randomly that satisfy the constraints (25b), (25e).

VI. SIMULATION RESULTS
In this section, the performance of the considered network is evaluated to demonstrate the effectiveness of the proposed URLLC RIS-UAV communications. For the optimal UAVs deployment, we use the CVXPY version 1.0.21 in Python [37] to implement the DNN. The system parameters of simulations results are set up as follows. The MBS area is a circle coverage with a radius of 500m whereas the extended area of the MBS is covered with a radius up to 2000m. We assume that the MBS is located at (0, 0, 30) while UEs are randomly distributed in the network. The limited altitude of the UAVs (H min , H max ) is (5, 20)m. The AWGN spectral density is σ 2 = −130 dBm/Hz. Unless otherwise stated, other parameters of the channel model are defined as in [29], [33], [38]. The maximum blocklength of the MBS is set at BL max = 200 (symbols) [39], [40].

A. EFFECTIVENESS OF DNN-BASED RIS-UAVS DEPLOYMENT
To demonstrate the performance of the DNN-based approach in RIS-UAVs deployment over the conventional optimisation approach, we compare the execution time of the DNN-based approach (DNN) and the conventional optimisation solution (Conv. OPT). Table 1 provides the execution time in RIS-UAVs positioning of two methods among different scenarios. As we can observe from the table, the DNN-based approach significantly outperforms the Conv. OPT in term of execution time, especially in large-scale scenarios. Interestingly, when the number of RIS-UAVs and UEs increases, the execution time of DNN-based method stays at the same level (0.04s), while that steadily rises in the Conv. OPT (from 0.13s to 1.6s).  Table 2 gives information about the effectiveness of RIS-UAVs in increasing reliability of URLLC systems. In particular, the table compares the decoding error probability among different number of RIS's elements with the transmit power range of P 0 = [40,50] dBm. Obviously, the more elements the RIS-UAVs have, the more reliability system can achieve. Particularly, without the support of RIS-UAV (No IRS), the decoding error probability is up to 10 −3 at P 0 = 50 dBm, which does not satisfy the URLLC requirements. It is important to note that for many importance applications such as smart grid, e-health, intelligent transport system (ITS), V2V, AR, the minimum required reliability is 99.999% [21]. Clearly from this table, we can see that without RIS, we cannot support the aforementioned URLLC applications. Differently, in RIS-aided scenarios, the reliability reaches to 1−10 −6 that meet stringent demand for many URLLC-based applications.

C. DECODING ERROR PROBABILITY WITH OPTIMAL POWER ALLOCATION AND BLOCKLENGTH
To illustrate the effectiveness of the proposed iterative optimisation algorithm, we compare the optimal power allocation solution with the equal power allocation in reducing decoding error probability. Fig. 4 displays the performance of the proposed power allocation solution in improving the reliability. As we can observe from the figure, the optimal blocklength and power allocation is always better than the equal power allocation, especially when the power budget of the system is small. As expected, the smaller packet size is, the higher reliability the URLLC system can achieve.

F. DECODING ERROR PROBABILITY VERSUS NUMBER OF RIS-UAV
To illustrate the impact of RIS on the performance of URLLS system, we compare the reliability of scenarios that have the same number of UEs and different number of RIS-UAVs.

VII. CONCLUSIONS
In this paper, to support various QoS's requirements of URLLC, we have proposed an aerial RIS-UAV system to assist the direct communications between a MBS employed with ZF beamforming and UEs. To overcome the severe shadowing and pathloss, each UAV carrying an RIS panel onboard serves as many as UEs in its cluster. We have formulated the optimal resource allocations in terms of UAVs, MBS's power, RIS's phase-shift, and blocklength. Due to the extreme complex optimisation problem, we have separated the optimisation problem into two sub-problems. Firstly, the optimal UAVs' deployment can be solved using DNN frameworks with fasted execution time compared with conventional optimisation approach. Secondly, the optimisation algorithm to solve the minimum decoding error probability of the worst-case user has been proposed. We have demonstrated through the extensive numerical results that our proposed optimisation scheme outperforms other existing approach which reveals the importance of using RIS onboard UAV to support URLLC for the next generation of wireless networks.