Packet-Drop Design in URLLC for Real-Time Wireless Control Systems

In real-time wireless control systems, ultra-reliable and low-latency communication (URLLC) is critical for the connection between the remote controller and its control objective. Since both transmission delay and packet loss can lead to control performance loss, our goal is to optimize control performance by jointly considering control and URLLC constraints in this paper. To achieve this goal, we formulate an optimal problem to minimize control cost by optimizing the packet drop and wireless resource allocation. To solve the problem, we analyze the relationship between communication and control. Then, based on the relationship, we decompose the original problem into two subproblems: 1) an optimal packet-drop problem to minimize control cost and 2) an optimal resource allocation problem to minimize communication packet error. Finally, the corresponding solutions for each subproblem can be obtained. Compared with the traditional method only considering the communication aspect, the proposed packet-drop and resource allocation method shows remarkable performance gain in terms of control cost.


I. INTRODUCTION
In real-time wireless control systems, ultra-reliable and lowlatency communication (URLLC) is critical for the connection between remote controller embedded in the base station (BS) and its control objective (i.e., the plant) [1].In such a system, the inevitable transmission delay and packet loss in URLLC lead to control performance loss.Furthermore, when the controller handles plenty of plants, guaranteeing both ultra-reliable and low latency is extremely challenging, which may lead to significant control performance loss.It is expected that both communication and control aspects can be jointly considered to maintain good overall system performance.
Recently, some works have been done on the impact of communication on control performance [2]- [10].For example, by modeling transmission time delay and packet loss into control systems, the authors in [3] analyzed the consequence of the imperfect transmission on the control performance, The associate editor coordinating the review of this manuscript and approving it for publication was Cunhua Pan .where imperfect transmission coefficients are introduced by communication protocols, e.g., transmission control protocol (TCP) or user datagram protocol (UDP).In [4]- [9], the authors further investigated the control performance of different control categories with imperfect communications.In [10], the authors provided a tutorial and reviewed the existing advances of wireless network design and optimization for wireless networked control systems.However, the aforementioned works are based on the existing wireless networks and cannot be used in control scenarios with ultra-reliable and low-latency requirements.
To deal with the issue, URLLC is proposed in the coming fifth generation (5G) cellular networks to support realtime wireless control systems [11].Some research have been done to maintain the extremely high QoS requirements in URLLC [12]- [19].For instance, Chen et al. [12] discussed resource reservation strategy to maintain extreme high QoS.In [13], Ji et al. investigated resource allocation schedules in physical layer for URLLC downlink.In [14], Esswie and Pedersen analyzed resource allocation method when URLLC coexists with another important scenario, i.e., enhance mobile broadband (eMBB).In [15], Ren et al. derived the closed-form solution for resource allocation based on the concept of effective capacity by considering the queueing delay violation probability.In [16], Pan et al. considered the joint blocklength and location optimization to minimize the decoding error probability while adopting the channel capacity expression for finite blocklength.In addition, Liu and Yu [18] developed a protocol of deviceto-device (D2D) communications in URLLC.However, the packet drop introduced by limited computing capacity is not discussed in the aforementioned works, which may lead to significant queueing packet drop.To deal with this problem, She et al. [19] proposed a queueing policy and a random packet drop policy to maintain the QoS requirement, where only communication aspect is taken into account.However, the interaction between control and URLLC is not obtained, which is very important since the communication design in URLLC is actually determined by the control requirement.
In this paper, we consider the design of the uplinks from sensors to the BS, where we formulate an optimal problem to minimize control cost by optimizing packet drop and resource allocation method.To solve the problem, the key is to decouple the binary packet drop and continuous resource allocation variables.The main contributions of this paper are summarized as follows.
• We formulate an optimal problem to minimize the control cost, where the limited wireless resource and extremely high QoS requirements are taken into account.
The formulated problem allows us to use optimal packet drop and wireless resource allocation method to support real-time wireless control with minimum control cost.
• We analyze the relationship between optimal control law and communication parameter design, and then we find that the binary packet drop and continuous bandwidth allocation can be decoupled based on their contributions to the control and communication.
• We decompose the formulated original problem into two subproblems based on the variable decouple: (1) an optimal packet drop problem to minimize control cost and (2) an optimal resource allocation problem to minimize communication packet error.By solving the subproblems, we obtain the packet drop and resource allocation method.The rest of this paper is organized as follows.In Section II, the system model is presented.In Section III, the optimal problem is formulated.In Section IV, we obtain the resource allocation method and packet drop method for the formulated problem.In Section V, simulation results are provided to show the performance of our method.Finally, Section VI concludes the paper.

II. SYSTEM MODEL
As shown in Fig. 1, we consider a centralized wireless control system, where a base station (BS) embedded M remote controllers conducts the control process for M plants.In the control process, each plant has one sensor sampling the state of the plant.Once the BS receives the sampling signal from each sensor, the corresponding remote controller embedded in the BS calculates the control command.Then, the BS transmits the command to the corresponding the plant to update its current state.With the control process performing, the plant state turns to the target state.In this section, the system model considering both communication latency and reliability is presented for the performance evaluation in real-time wireless control systems.

A. COMMUNICATION
In this subsection, we focus on the uplink from the sensors to the BS, where we assume that only the uplink experiences time delay and packet loss.We consider orthogonal frequency division multiple access (OFDMA), where we assume that each sensor is allocated with independent continuous bandwidth B m .In addition, we consider flat fading channel, where the channel gains for each sensor are approximately identical and perfectly known for the sensor.Furthermore, we assume that transmission duration for the m-th sensor is T m .

1) CHANNEL MODEL
We consider that the channel model consists of the smallscale fading and large-scale attenuation coefficients between transceiver, which are represented as h m and g m for the uplink from the m-th sensor to the BS, respectively.According to [20], the large-scale attenuation coefficient can be expressed as where l m ≥ 0.035 km is the distance between transceiver [20].
The small-scale fading h m follows Rayleigh distribution with mean zero and variance σ 2 0 = 1 [21].However, since the end-to-end (E2E) latency is no more than 1 ms in URLLC [19], the transmission time delay from the sensors to the BS or that from the BS to the plants is less than the channel coherence time, which means that the small-scale fading h m is constant during the transmission period of the uplink and the downlink [22].

2) CHANNEL CAPACITY
According to [19] and [23], we can obtain the uplink channel capacity expression for the m-th sensor in URLLC as where the first term on the right hand of (2) is the achievable Shannon capacity without transmission error, the second term is the minus error bits introduced by channel dispersion V m , the third term is the approximation of the reminder terms of order log( and ε e m is the transmission error probability.Furthermore, we assume that the single-sided noise spectral density is represented by N 0 , then according to [19], we have Shannon capacity C m and channel dispersion V m as follows, respectively, and where γ m is the received signal-to-noise-ratio (SNR) at the BS and can be expressed as where P m is the transmission power spectral density of the m-th sensor.

B. CONTROL
In this subsection, we provide the control model for each plant m with communication time delay and reliability.As shown in Fig. 2, the control process is conducted as following.First, a sensor takes the sample of the corresponding plant's current state and transmits it to the BS.Then, the controller in the BS estimates the state by Kalman Filter [3], calculates the control command, and sends it to the plant.Finally, the plant state updates by the received control command.Based on the above control process, the linear differential equation of the m-th plant can be expressed as [3]  where x m (t) is the plant state, u m (t) is the control input, and n m (t) is the disturbance caused by additive white gaussian noise (AWGN) with zero mean and variance R. In addition, we assume that each plant m has the same A and B, which represent the physical system parameter matrices (more details can be obtained in [24]).We assume that s m,n represents the sample period at time index n, which consists of the wireless transmission time delay T m,n and an idle period sm,n .Their relationship can be expressed as where n = 1, 2, • • • , N represents the sampling time index in the control process.Then, the discrete time control model with time delay d m,n can be obtained as [3] x where m,n = e As m,n , m,n 0 = sm,n 0 e At dt • B, and m,n T is the generalized state, then the control function in (8) can be rewritten as where Considering the packet loss, we have the close-loop system in (9) can be rewritten as (10), as shown at the bottom of this page.
where we use α m,n to indicate if a packet should be discarded or not (α m,n = 0 is to discard the packet while α m,n = 1 is to keep the packer for transmission) and use β m,n to indicate if a packet is successfully transmitted or not (β m,n = 0 is failed transmission while α m,n = 1 is successful transmission).
In the above discussion, we have obtained the wireless control model1 where both communication time delay and packet loss have been taken into account.In the following of this paper, we will formulate the optimal problem and propose corresponding method to obtain packet drop and resource allocation.

III. PROBLEM FORMULATION
Our goal is to find the optimal packet drop method and wireless resource allocation method by minimizing control cost.To achieve this goal, we first provide the objective function, i.e., the control cost.Then, we obtain the packet drop constraint in wireless communications.Finally, we formulate the optimal problem.

A. OBJECTIVE
The quadratic control cost is one the most important criterions to evaluate the control performance [25] [26], which are composed by the sum of the deviations of the plant state from its desired setpoint and the magnitude of the control input.Then, the quadratic control cost can be expressed as [10] where W and U are the weight of the state and that of the control input, respectively, and they can be adjusted according to the emphasis of the control system.The control variables in (11) can be obtained by Appendix A. From (10), we can obtain that the generalized plant state ξ m,n is a function of transmission time delay and packet loss, i.e., ξ m,n (T m,n , α m,n ).In addition, u m,n is also a function of T m,n and α m,n .Furthermore, the total number of packet loss is determined by constraints on the packet loss probability.Thereby, the control cost J m,N is a function of communication parameters.

B. CONSTRAINT
In Section II.A, we have introduced transmission error probability ε e m .However, when we consider the packets from M sensors, the queueing delay violation probability cannot be ignored [18], which results in some part of the packet loss at the BS to maintain the extreme high QoS in URLLC.
We assume that ε q m represents the queueing delay violation probability at the BS.Considering both queueing delay violation probability and transmission error probability, we have the total packet loss probability of the m-th sensor as where ε th is the upper bound of the total packet loss probability.From (2), we have where λ = T m B m R m is payload information for each sensor, and f Q (•) is the Q function.Next, we discuss the the queueing delay violation probability ε q m in details.As shown in Fig. 3, each E2E communication pair, i.e., sensor-BS-plant, has the corresponding buffer at the BS, where i m represents the packets uploaded to the BS from the m-th sensor and Q m represents the queue length for the m-th plant.Furthermore, we assume that j m represents the packets departed from the m-th queue.Then, according to [19], we have the following Lemma 1.
Lemma 1: The queueing delay violation probability ε q m can be expressed as where φ m is the QoS exponent for the m-th plant, D q max is the queueing delay bound, and E B m (φ m ) is the effective bandwidth and can be expressed as Proof 1: The details of the proof for Lemma 1 can be obtained in [19].

C. OPTIMAL PROBLEM
In this subsection, we formulate the optimal problem, which is described in Problem 0, i.e., P 0 , and can be expressed as where the objective of this optimal problem is to minimize the total control cost J sum constrained by communication where the parameters can be obtained by Appendix A. Observing the expression in (17), the optimal control cost J * m,N is related with α m,n for given transmission time delay.parameters, B th is the upper bound of the allocated bandwidth for each sensor at any time index, T th is the upper bound of the communication time delay.In addition, ε th is the upper bound of the packet loss probability in URLLC, which is in the region that guarantees the convergence of the control systems [1].Furthermore, α m,n is used to indicate if a packet should be discarded or not.Then, we obtain that the successful packet transmission probability can be expressed as Pr{β m,n = 1|α m,n = 1} = 1 − ε e m and the failed packet transmission probability can be expressed as Pr{β m,n = 0|α m,n = 1} = ε e m .In (16), the constraint in (16e) maintains the URLLC QoS requirements to guarantee successful transmission when wireless transmission is triggered by control process.Constraints in (16b) and (16c) are related to both communication and control.On the one hand, (16c) maintains the total packet loss probability to guarantee the URLLC QoS requirements for successful transmission, which determines packet drop strategy α m .On the other hand, the total control cost of M plants is determined by how to arrange α m in (16b), where α m is related to communication packet loss probability.Furthermore, the constraint on transmission time delay in (16d) is also related to communication performance and control performance.Thus, P 0 in ( 16) is extremely challenging to be solved under the constraints in (16b), (16c), and (16d).In the next section, we will discuss the solution for the problem P 0 in details.

IV. PACKET DROP AND WIRELESS RESOURCE ALLOCATION METHOD
In this section, we first analyze the relationship between communication and control.Based on this, we decompose (16) into two subproblems: (1) an optimal wireless resource allocation problem to minimize transmission error probability and (2) an optimal packet drop problem to minimize control cost.Then, the solution for the two subproblems can be obtained.

A. RELATIONSHIP BETWEEN CONTROL AND COMMUNICATION
As shown in Fig. 2, the linear feedback control law is used in this paper (more details about this law can be obtained in [2]).Then, the optimal expression for the control cost in (11) can be rewritten as (17), as shown at the top of this page, [2].Then, we can obtain the relationship between control and communication by the following Theorem 1.
Theorem 1: Once the communication time delay and packet loss probability is determined, the optimal control cost in ( 17) is related with α m,n from communication aspect.Thus, the minimization of the objective function in (16a) is relatively independent with communication constraints on communication time delay and packet loss probability in (16).
Based on Theorem 1, α m,n is the connection between communication and control.Then, P 0 can be divided into two subproblems.The first subproblem is to optimize the communication time delay and packet loss probability by wireless resource allocation.Once they are obtained, the second subproblem is to minimize control cost by packet drop design.

B. PACKET DROP AND RESOURCE ALLOCATION
Compared with transmission time delay in URLLC, packet loss introduces more control performance loss, i.e., larger control cost.This is because packet loss can be treated as larger time delay than the required time delay in URLLC.Then, minimizing packet loss probability is critical for better control performance.Thus, our goal is to minimize packet loss probability by optimizing transmission time delay and bandwidth allocation in the first subproblem.By the obtained transmission time delay and packet loss probability, the second subproblem is to design the packet drop α m,n to minimize control cost.

1) RESOURCE ALLOCATION
Since the queueing delay violation probability is independent with wireless communications, the first sub-problem is to minimize transmission error probability by optimal wireless resource allocation, which can be expressed as P 1 , i.e., By minimizing error probability, the transmission requirement for the control process can be guaranteed.To obtain the optimal resource allocation for P 1 , we assume that the resource block consists of time resource and bandwidth resource, i.e., T m,n × B m,n .Then, to solve the problem (18), we need the following property about ε e m,n .
By Property 1, we can apply the exact linear search method to find the optimal (B m,n T m,n ) * to minimize ε e m,n [28].From the above discussion, we can obtain the optimal resource block allocation to minimize transmission error probability.To reduce control cost, the transmission time delay is desired to be short enough.Then, the time delay T m,n in resource block can be calculated by By the obtained minimum transmission error probability ε e m and time delay T m,n in (19), we discuss the packet drop design in the following.

2) PACKET DROP
The second subproblem minimizing the overall control cost can be expressed as P 2 , i.e., where (20b) is the overall communication packet loss probability for each E2E (i.e., sensor-BS-plant pair).To deal with P 2 in (20), we assume that M plants have the same control parameters.In addition, we assume that the weight on the plant state W is much larger than that on the control input U.
This assumption holds in this paper, since the plant state is more important than the control input in mission-critical realtime wireless control systems [1].In addition, we have proved that J * m,N increases strictly with the overall communication packet loss probability in [1].
Since control process is a sequential process, packet drop strategy leads to different control cost.In addition, it is extremely difficult to predict the plant state since the state update in (6) has disturbance term.Then, it is challenging to obtain global optimal packet drop method to minimize total control cost.Instead, we propose a suboptimal packet drop method, where we obtain the point-wise minimum control cost by the suboptimal packet drop.
In the proposed method, we assume that E m,n = x T m,n Wx m,n represents the instantaneous control cost of the m-th plant at time index n.When packet drop occurs at time index n, the BS will drop the packet that contributes to minimum plant state, which leads to point-wise minimum control cost.The detailed method is summarized in Algorithm 1.

V. SIMULATION RESULTS
In this section, we provide simulation results to demonstrate the performance of our analysis in this paper.In communication sub-systems, we assume that the payload information is 100 bits.The maximum time delay of URLLC is 1 ms and the maximum packet loss ε is 10 −5 .The control parameters Furthermore, we assume that the initial state is (100, 100).Each curve is obtained by 10000 Monte Carlo trails if there is no extra declaration.Moreover, the random packet drop method and channel gain based packet drop method are considered as comparison.In addition, the exhaustive search method solving the formulated problem is considered to justify the benefits of the proposed algorithm.Fig. 4 demonstrates the total control cost when the available bandwidth B th is different, where the queueing delay bound is 0.1 ms.From the figure, all the curves decrease monotonously with B th .This is reasonable since larger B th can guarantee less packet loss, which maintains the timely control input for control systems to reduce the control cost.In addition, the decreasing rate of all the curves is smooth and low when B th is more than 2 × 10 6 Hz, which is because that B th is saturated.Furthermore, the total control cost is similar for both random method and channel gain based method, since they have equal contribution to the control cost.On the one hand, all the three curves are similar when B th is small.This is reasonable since small B th leads to large packet error probability, which results in that the control system is not very sensitive to the packet drop method.From the figure, compared with random method and channel gain based method, the proposed method decreases the total control cost by almost 40% when B th ≥ 2 × 10 6 Hz, which indicates that the proposed method in this paper has large advantage compared with only considering the communication aspect.In addition, compared with exhaustive search method, the control cost of the proposed method is raised by at most 6.5%, which indicates that the solution of the proposed method is close to the global optimal solution.Fig. 5 shows the total control cost when the number of sensor-plant pairs M is different, where the total available bandwidth is B th = 2 × 10 6 Hz, the queueing delay bound is 0.1 ms.From the figure, all the curves increase monotonously with the number of sensor-plant pairs M .This is reasonable since the supported number of sensor-plant pairs is fixed with given B th , which further leads to large control cost when M increases.In addition, the curves of the random method and the channel gain based method are similar, which is because that both of them have the same effect on the control performance.Furthermore, the advantage of the proposed method is approximative when the number M is very large, i.e., M ≥ 300.This is reasonable since the dropped packets to minimize control cost has minor effect on the total cost when M is too large compared with the traditional methods.In addition, the control cost of the proposed method is is similar to the global optimal solution obtained by exhaustive search method.FIGURE 6.Total control cost with different arriving rate of sampling Fig. 6 demonstrates the total control cost when the arriving rate of the sampling packets is different, where the total available bandwidth is B th = 2 × 10 6 Hz, the total number of sensor-plant pairs is M = 250, and the queueing delay bound is 0.1 ms.From the figure, all the curves increase monotonously with the arriving rate until 500 packets/s.In addition, after 500 packets/s of the arriving rate, the curves of the total control cost are approximative horizontal.This can be explained by the following two aspects.On the one hand, when the arriving rate is less than 500 packets/s, larger arriving rate means smaller sampling period h k , which leads to smaller d k /h k .Then, the control cost increases as the arriving rate increasing before 500 packets/s [1].On the other hand, when the arriving rate is larger than 500 packets/s, the number of arriving packets tends to saturated, which leads to a balance state and the curves of the control cost have little changes.Furthermore, from the figure, we can obtain that the proposed method decreases the control cost by at most 60% compared with only considering the communication aspect.In addition, the control cost of the proposed method is approximated to the global optimal solution obtained by exhaustive search method.Fig. 7 demonstrates the total control cost when the queueing time delay constraint is different, where the total available bandwidth is B th = 2 × 10 6 Hz.From the figure, all the curves decrease monotonously with the queueing time delay constraint.This is reasonable since larger queueing time delay constraint allows more packets in the queue, which leads to less packet drop probability and larger transmission successful probability.Then, the control cost can be reduced.However, the control cost changes when the queueing time delay constraint is larger than 5 × 10 −7 s.This is because the allowed number of arriving packets tends to saturated, and a balance state is maintained.Then, the curves of the control cost have little changes.Furthermore, from the figure, we can obtain that the proposed decreases the control cost by at most 62% compared with only considering the communication aspect.In addition, the performance gap between the proposed method and the exhaustive search method is minor.

VI. CONCLUSIONS
In this paper, we proposed a packet drop and wireless resource allocation method in URLLC for real-time wireless control systems.To obtain good control performance, we formulated an optimal problem to minimize the control cost with communication constraints.To solve the problem, we discussed the relationship between control and communication.Based on that, we decomposed the original problem into two relatively independent sub-problems.By solving the two subproblems, we obtained the transmission time allocation, bandwidth allocation and packet drop method.The proposed approach established a theoretic foundation for the URLLC enabled real-time wireless control system performance analysis and algorithm design.

APPENDIX A
This appendix provides the detailed calculation of the parameters in (17).
According to [2], S k is calculated by The generalized state can be estimated by a modified Kalman filter, which can be obtained as follows.
• Step 1: prior generalized state estimation.The prior estimation for the generalized state can be expressed as ξm,n+1|n = m,d ξm,n|n + m,d u m,n , where ξm,n|n is the generalized state estimation based on the current generalized state, and ξm,n+1|n is the generalized state estimation at time n + 1 based on the last generalized state at n.
• Step 2: prior error variance estimation.The prior estimation for the error variance can be expressed as where P m,n|n = E[(ξ m,n − ξm,n )(ξ m,n − ξm,n ) T ] is the estimation error variance, and P m,n+1|n is the prior estimation error variance at time k + 1.
• Step 4: optimal control gain estimation.The optimal control gain estimation K m,n+1 can be expressed as Finally, substituting the above parameters into (11), we can obtain (17).Furthermore, to minimize the control cost in (11), the control input needs to satisfy the following expression ), and G 2 = (log e), then ε e m,n can be rewritten as Let Taking derivative with respect to x, we can obtain Then, the second derivative with respect to x can be expressed as

FIGURE 4 .FIGURE 5 .
FIGURE 4. Total control cost with different available bandwidth B th .

FIGURE 7 .
FIGURE 7. Total control cost with different constraints on queueing delay.