Real-time scheduling of massive data in time sensitive networks with a limited number of schedule entries

Time sensitive networks support deterministic schedules over Ethernet networks. Due to their high determinism, high reliability and high bandwidth, they have been considered as a good choice for the backbone network of industrial internet of things. In industrial applications, the backbone network connects multiple industrial ﬁeld networks together and has to carry massive real-time packets. However, the off-the-shelf time-sensitive network (TSN) switches can deterministically schedule no more than 1024 real-time ﬂows due to the limited number of schedule table entries. The excess real-time ﬂows have to be delivered by best-effort services because the switch only supports the two scheduling services. The best-effort services can reduce average delay, but cannot guarantee the hard real-time constraints of industrial applications. To make the limited number of schedule table entries support more real-time ﬂows, ﬁrst, we relax scheduling rules to reduce the requirement for schedule table entries and formulate the process of transmitting packets as a satisﬁability modulo theories (SMT) speciﬁcation. Then, we divide the SMT speciﬁcation into multiple optimization modulo theories (OMT) speciﬁcations so that the execution time of solvers can be reduced to an acceptable range. Second, we propose fast heuristic algorithms that combine schedule tables and packet injection control to eliminate scheduling conﬂicts. Finally, we conduct extensive evaluations. The evaluation results indicate that, compared to existing algorithms, our proposed algorithm requires only one-twentieth the number of schedule entries to schedule the same ﬂow set.


I. INTRODUCTION
Industrial internet of things (IIoT) has changed the way that manufacturing is carried out [1].For example, IIoT supplies connectivity between customers and production lines so that customers can guide the production process directly [2], and the IIoT connectivity allows data exchange among all industrial devices so that the whole production process can change rapidly to adapt to new products [3].Industrial networks are The associate editor coordinating the review of this manuscript and approving it for publication was Chunsheng Zhu .
the most fundamental and important part of IIoT.To satisfy the strict requirements of industrial applications, industrial networks have to transmit data deterministically, such as precisely transmitting control data, supplying adequate bandwidth for video streams, and managing each packet in massive-machine-type communications.
Many industrial wired and wireless networks have been proposed to handle these high-requirement scenarios, e.g., WIA-FA [4], WIA-PA [5], RT-WiFi [6], WirelessHP [7], WirelessHART [8], TTEthernet [9], and time sensitive networks (TSNs) [10].However, there is still no single network framework that can guarantee all industrial requirements.Heterogeneous networks constituted by a wired backbone network and multiple wireless field networks are considered as the future solutions of industrial networks, for example, the wired fronthaul/backhaul and cell sites in 5G [11].The wireless field networks, which are flexible and easy to deploy, connect massive devices to the upper management; the wired backbone network, which has high bandwidth and high reliability, carries all data transmitted by the wireless field networks.Thus, the wired backbone network should support all performance-optimization technologies that wireless field networks have done.If not, the optimization effect achieved in wireless field networks will be reduced, and even totally disappear, in wired backbone networks.For example, when the backbone network supports only best-effort services, regardless of which field network is used, the end-to-end delay of a packet is still uncontrollable and nondeterministic.
TSNs are a good choice for the backbone network because they support deterministic schedules over Ethernet networks, and provide high-bandwidth, high-reliability and high-determinism connectivity.Almost all performance indicators of TSNs are better than those of wireless field networks.However, a performance-optimization problem that wireless field networks do not need to consider but that backbone networks have to handle is how to transmit massive, real-time data deterministically.Wireless field networks support real-time communications, but do not need to transmit massive data because a single field network will not cover a large-scale region.However, when multiple field networks submit their data to the backbone TSN, the backbone TSN must consider how to handle massive data.There is no related work that has considered how to schedule massive data in TSNs under real-time constraints (as described in Section II).Therefore, in this paper, we will propose real-time scheduling algorithms that can transmit massive data deterministically in TSNs.
There are two challenges to carrying out real-time scheduling of massive data.First, in off-the-shelf TSN switches, e.g., [12]- [14], the limited capacity of schedule tables makes it difficult for them to handle massive data.TSN switches support both deterministic and nondeterministic communications.For deterministic communications, the schedule tables record the precise times to control egress ports so that the transmitting times of forwarded flows can be precisely controlled.However, due to the limited number of schedule table entries in switch chips, the off-the-shelf TSN switches support no more than 1024 deterministic data flows [12]- [14].The excess deterministic data flows have to be delivered by best-effort services.Although a new large-capacity chip can be created to support more deterministic data flows, the relevant time costs and price costs cannot be ignored.Second, massive data with different requirements cannot be aggregated to reduce the difficulty of scheduling.TSN switch chips do not support aggregating functions.Only micro control units (MCUs) that are connected to switch chips can aggregate data packets.However, if data packets are sent to MCUs, the exchange of data packets between the switch chips and MCUs will require plenty of time and thus reduce the system performance.Therefore, data can be aggregated only before they are injected into the TSNs and disaggregated only when they leave the TSNs, i.e., the aggregated data packets must have the same path, the same release time and the same real-time constraint.This severe aggregation is not suitable for flexible and diversified machine-to-machine communications.Therefore, our proposed scheduling algorithms should be constrained by the capacity of schedule tables and nonaggregated.
To schedule massive real-time data packets using a limited number of schedule entries, we make the following contributions.
(1) In some cases, TSN switches do not need to record port controls for each packet.For example, when a switch forwards packets immediately after it receives them, the precise transmitting time is determined by its previous switch.Thus, we relax the constraint that switches have to record the port controls for all packets and formulate the process of transmitting packets as a satisfiability modulo theories (SMT) specification [15].However, the execution time of SMT solvers increase exponentially with the problem size.To obtain a feasible solution in an acceptable time, we divide the SMT specification into multiple optimization modulo theories (OMT) specifications [16].The dividing strategy makes the execution times of solvers increase linearly with the problem size.Our evaluations indicate that in 2 days, the OMT-based algorithm can solve 40000 packets, while the SMT-based algorithm can solve only approximately 400 packets.
(2) The problem we focus on is NP-complete.There is no polynomial time algorithm for finding a solution for our problem.Hence, we adopt heuristic algorithms to improve the scalability of our work.First, the NGC algorithm is presented to eliminate all conflicts by adjusting the injection times.In this algorithm, switches do not need to record any time to transmit packets, and once a packet is injected into the network, it can be transmitted to its destination without interruption.Then, we analyze the upper bound of end-to-end delays in this algorithm and find two factors that make data flows difficult to schedule.Based on the analysis, we propose the MF algorithm that uses schedule entries to support postscheduling adjustments.The evaluation results indicate that, compared to existing algorithms, the MF algorithm requires only one-twentieth the number of schedule entries to schedule the same 32000 packets.
The rest of the paper is as follows: Section II first reviews two categories of related work, including specification-based algorithms and heuristic algorithms, and then explains why the existing work cannot be used in our problem.Section III details our system model and problem.Section IV and Section V propose specification-based algorithms and heuristic algorithms to solve our problem, respectively.Section VI evaluates our proposed algorithms based on massive test cases.Section VII discusses the underlying meaning of this paper.Section VIII concludes the paper.

II. RELATED WORK
Real-time scheduling has been widely studied in uniprocessor/multiprocessor systems [17], [18] and networked systems [19], [20].However, the algorithms designed for other systems cannot obtain the expected results in TSNs due to their special transmitting mechanisms.TSNs represent a set of IEEE standards, in which the IEEE standard 802.1Qbvdefines the architecture of TSN schedulers, but does not specify scheduling algorithms.In some recent papers, the TSN scheduling problem has been considered.The work in [21] formulates the scheduling problem as SMT and OMT problems, and then calculates static schedules using an off-theshelf solver.In addition to SMT/OMT solvers, integer linear programming (ILP) solvers can be used to generate schedules.The work in [22] proposes various ILP formulations to generate incremental schedules, and its results are close to the optimal solutions of the corresponding static scheduling problem.The work in [23] formulates the joint routing and scheduling problem as an ILP and explores the real limitations by evaluating extensive test cases with widely varying parameters.The work in [24] uses an ILP approach to maximize the resources available for audio-video-bridging and best-effort traffic in TSNs.Although the results obtained by off-the-shelf solvers are optimal or close to optimal, their execution times become unacceptable as the system size increases.
Heuristic algorithms are faster than off-the-shelf solvers and have been used in TSNs.The work in [25] proposes a K-shortest path heuristic and a meta-heuristic randomized adaptive search procedure to solve the joint routing and scheduling problem for time-triggered traffic, and at the same time reduce the end-to-end delays of audio-video-bridging traffic.The work in [26] develops two algorithms to schedule multicast time-triggered flows: the first algorithm is based on genetic algorithm and generates schedules under real-time constraints; the second one is based on fixed routings and implemented as a list scheduler.The work in [27] presents a heuristic incremental scheduling algorithm to minimize the impact of reconfiguring schedules on existing flows.The work in [28] proposes a configuration agent architecture and a simple scheduling heuristic such that the schedules of dynamic flows can be quickly generated and configured.However, these algorithms do not consider the capacity limitation of network switches.
The work in [29], [30] introduces capacity limitations into their problem models to make the scheduling problem more similar to real systems.However, the method proposed in [29] generates schedules using solvers, and its formulation is to optimize the individual jitter of each flow; the objective optimized in [30] is to minimize the maximal end-to-end delay, i.e., the problem focused on by [30] is not a typical real-time scheduling problem with deadlines.The work [31] splits a problem into several subproblems so that capacity limitations can be ignored.Although the similar method can be used in TSNs to distribute data flows to multiple switches, the cost of the extra switches is unaffordable to users when massive flows need to be distributed.In contrast, our proposed algorithms are for a normal real-time scheduling model with a limited number of schedule entries and can quickly generate schedules for massive data packets.

III. SYSTEM MODEL AND PROBLEM STATEMENT
In this section, we will detail our system model and problem.The symbols used in this paper is summarized in Appendix.

A. NETWORK
A time-sensitive network is characterized by a three-tuple < N , W , L >.The node set N = {n 1 , n 2 , ...} includes all switches and gateways.The switch set W is a subset of N .Gateways connect other networks, e.g., RT-WiFi, WIA-PA, and WirelessHP, to TSNs.We use n i,j to denote the j-th port of switch n i .The element l i,g in the link set L denotes that there is a cable between nodes n i and n g .Our algorithms are suitable for all topologies.To describe the model more clearly, we specify the network topology is mesh since mesh topologies can reduce to the other topologies.
The architecture of a TSN switch with 4 ports is shown in FIGURE 1. Ports and cables are both full-duplex.After a packet is injected into a switch from an ingress port, it is switched to its destination egress port according to routing tables, and then stored into a queue of the egress port.Each egress port has multiple queues, and the queue with a smaller ID has lower priority.The packet is stored in the queue that has the same ID as that of the priority code point (PCP) segment in the packet header.To avoid conflicts among packets, one queue is used by at most one packet at a time [21].Each queue connects to a gate.If the gate is opened, the packet stored in the queue is allowed to use the egress port.Otherwise, the packet waits until the gate is opened.When multiple gates are opened, the nonempty queue with the highest priority uses the egress port.VOLUME 8, 2020 Gates are controlled by schedule tables.Each entry in the table contains time, port, and gate states [13], [14].For example, in FIGURE 1, at Time 1, the first entry is applied to Port A, and gate states 1000 0000 mean that only the gate of the first queue, i.e., Queue 0, is opened, and the others are closed.Then, at Time 2, the gate states change to 0100 0000 that mean the gate of Queue 0 is closed, and the gate of Queue 1 is opened.A switch contains at most T entries.
If not all entries are used, the time of unavailable entries is marked as 0. Available entries are stored in the order of time, i.e., if Time i and Time j are not zero and i > j, then Time i > Time j.After all available entries are executed, the first entry is executed again.Note that the last available entry has the same control information as that of the first entry because the last entry is added to mark the running time of the penultimate entry.The limitation of a table capacity T is the main reason why it is difficult for TSN switches to support massive data communications.Some related papers adopt other kinds of schedule tables.For example, in the paper [29], a switch has multiple schedule tables (also called gate control lists), and a schedule table corresponds to only one port.Thus, schedule entries do not need to contain port information.Regardless of which kind of schedule tables is adopted, our proposed algorithms can still be used as long as one entry corresponds to only one port.

B. DATA FLOW
Industrial data flows can be divided into two categories: critical flows and noncritical flows.Critical flows are hard real-time, while noncritical flows are best-effort.Our proposed algorithms schedule only critical flows.The remaining resources can be used by noncritical flows.In the following, if no specific description is given, flows refer to critical flows.A flow f a in the flow set F is characterized by a five-tuple < p a , d a , q a , γ a , a >, which denote the period, relative deadline, queue ID, end-to-end delay, and routing path, respectively.Each flow generates a packet periodically with a period p a .In addition to the backbone TSN, packets are transmitted in field networks.Hence, two time intervals in a period are occupied by field networks as shown in FIGURE 2(a).Only the middle interval belongs to the backbone TSN, i.e., our proposed algorithms only consider the middle intervals.The scheduling problem between field networks and the backbone network exceeds the scope of our paper and is not considered here.The lengths of these time intervals can be zero when sources or destinations are gateways.This does not impact our proposed algorithms.
If flows are allowed to start at any time, the schedules cannot be generated due to the unknown conflicts among packets.Thus, we adjust flows to make them start at the same time.The adjustment is easy to implement because all nodes are time synchronized, and the gateways can control the injection times of flows.Then, relative deadlines are equal to the available time intervals in which packets can be transmitted in TSNs as shown in FIGURE 2(b), i.e., d a ≤ p a .Flow f a releases its k-th packet at time p a × k + 1, and its absolute  deadline is p a × k + d a .We consider only how to schedule in the first hyperperiod H , which is defined as the least common multiple of their periods, i.e., H = LCM {p 1 , p 2 , ...}, because after the first hyperperiod, the subsequent schedules are periodically repeated.
The queue ID q a is written into the PCP segment in the packet header before packets are injected into the TSNs.The routing path a is a node set, i.e., a = {π a1 , π a2 , ...} and a ⊆ N .Since routing is already well-studied [32], [33], we do not propose any routing algorithm and consider that the routes are already generated based on some existing algorithms.
The delay of a packet is composed of four delays, including the receiving delay, processing delay, queuing delay, and transmission delay γ + a , as shown in FIGURE 3. The receiving delay is from the time that the packet enters the switch to the time that the packet is completely received.The processing delay is needed by the switch engine.
Since the processing delay is less than 1µs, we ignore it in our problem.Then, the packet might be blocked in the queue.Thus, the queuing delay depends on schedules, and can be zero.The transmission delay is the amount of time needed to serialize and propagate the packet on the cable.Serializing delays and propagation delays should be listed separately because queues are still occupied during serializing delays.However, propagating a packet in a single cable requires at most 556 ns [34], and in TSNs, the synchronization error is approximately 1µs.Therefore, we merge propagation delays and serializing delays together.Queues are occupied in all of the delays.The time durations spent in different nodes overlap each other since different parts of a serialized packet are processed in two adjacent nodes simultaneously.We use delay γ a to denote the time it takes for a packet to transmit from its source to its destination without interruption.
TSNs adopt the IEEE 802.1AS standard to synchronize nodes.The only effect of the synchronization standard on our problem is that time synchronization introduces time errors to transmission delays.To simplify the description, we consider that the time synchronization errors are contained in transmission delays and will not be mentioned in the following.

C. SCHEDULING PROBLEM
To deterministically schedule massive data packets, the following four rules have to be followed.
1) A queue cannot be shared between critical packets and noncritical packets even at different times.Noncritical packets are uncontrollable.We cannot predict when noncritical packets will occupy queues and how noncritical packets will affect critical packets.Therefore, to make critical packets controllable, we assign several dedicated queues to them, and then the remaining queues can be used by noncritical queues.
2) Although critical packets and noncritical packets share the same network resources, scheduling critical packets does not require considering the noncritical packets.To maximize the bandwidth of noncritical data flows, their gates are always opened.However, noncritical transmissions cannot preempt critical transmissions because the queues used by critical packets have higher priorities than those used by noncritical packets.Although, critical packets can preempt noncritical packets, we still do not need to consider the preemption process.This is because guard bands, which are inserted before critical packets to keep them from being affected by unfinished noncritical packets, can be integrated into noncritical packets.
3) When multiple queues have critical packets, they cannot be opened at the same time.For example, there are two packets in two queues, and their gates are opened.In a regular case, the high-priority packet can use the egress port, and the low-priority packet is blocked.However, if the high-priority packet loses, the low-priority packet will be sent to the cable before its expected time.Then, the subsequent transmissions will be affected, and their determinism will be difficult to guarantee.Therefore, we do not permit the queues that are storing critical packets to be opened at the same time.

4) If a critical queue has no packets, its gate can remain open.
Note that even though a high-priority gate is opened, the low-priority queue with opened gate can use the egress port as long as there is no packet in the higher-priority queue [35].Based on this rule, we can reduce the number of closings, i.e., the number of table entries.
Therefore, based on the four scheduling rules, given the network < N , W , L >, the critical flow set F and the number of queues used by critical packets, our objective is to generate schedules for all critical flows to satisfy the following requirements.
• Real-time requirement: All of the critical packets are delivered to destinations before their deadlines.
• No resource conflict: Every queue and every link serves at most one packet at any time.
• Resource limitation: For each switch, the number of schedule entries used is not greater than the limitation T .Since the schedules must be deterministic, we adopt fine-grained scheduling algorithms.Comparing with coarsegrained algorithms that specify a long time duration to transmit all of the critical packets, our fine-grained algorithms will assign a link and a time duration to each transmission so that end-to-end delays are controllable and deterministic.
A flow set is schedulable, if it has a feasible schedule that meets all the requirements.A scheduling algorithm is called optimal, if it can find a feasible schedule whenever one exists.If there are enough table entries to close the gates after each packet is sent, our problem is the same as the problem in [29].Since the problem in [29] is NP-complete, our problem is also NP-complete.
To solve our NP-complete problem, we propose multiple algorithms that are briefly introduced in FIGURE 4. First, since SMT algorithms are optimal for NP-complete problems, we use SMT specifications to state our problem, and use SMT solvers to generate schedules (in Section IV-A).However, the execution time of SMT solvers increases exponentially with the problem size.For simple scheduling problems, SMT solvers have to spend tens of minuets to generate the schedules of tens of flows [21], [29].Thus, SMT solvers cannot solve our SMT specification in acceptable time because our problem model is more complex than that of any existing work.Therefore, based on the SMT specification we propose an OMT-based algorithm that can significantly reduce the execution time (in Section IV-B).
Second, since the OMT-based algorithm is still restricted by third-party solvers, to further improve the scalability of our proposed algorithms, we propose heuristic algorithms to make a tradeoff between performance and efficiency.Our first heuristic algorithm adopts the minimum number of schedule entries to transmit packets (in Section V-A), and then we analyze why the first heuristic algorithm cannot schedule more packets (in Section V-B).Based on the analysis, we propose a second heuristic algorithm that  adopts the unused schedule entries to transmit more packets (in Section V-C).

IV. ALGORITHMS BASED ON SMT/OMT
SMT specifications are based on first-order logic, and a SMT problem is to determine whether the specification is satisfiable or not.OMT is an extension of SMT and combines SMT with optimization procedures.To describe our problem in detail, we formulate the problem as an SMT specification.Then, we divide the problem into multiple OMT specifications to improve its scalability.

A. SMT SPECIFICATION
Our objective is to assign queues to flows and compute a schedule table E i for each switch n i (∀n i ∈ W ). Each entry e i,y in table E i is denoted as < t i,y , v i,y,1 , . . ., v i,y,V , c i,y,1 , . . ., c i,y,Q >.The entry starts to run at time t i,y .Not all entries in the schedule table E i are used.If the entry is not used, its t i,y is set to 0. A switch has V ports, and each egress port has 1) Range constraint: Based on our model and problem, the ranges of all variables are as follows.
2) Path sequence constraint: In a routing path, the (b+1)-th hop starts after the b-th hop finishes.
) 3) Real-time constraint: All packets have to be delivered to their destinations before their absolute deadlines.
This equation has been included in Constraint 1).4) Link conflict constraint: If two packets use the same link, the times during which they pass through the link cannot overlap.
). ( 8) 5) Queue conflict constraint: If two packets use the same queue, the times at which they are stored in the queue cannot overlap.
) 6) Time validity constraint: Only schedule table E i is recorded in the TSN switch.There must be a relationship between α k a,b , β k a,b and E i so that the schedules in switches can guarantee the above constraints.We know that β k a,b is equal to α k a,b plus the transmission delay γ + a .Therefore, we consider only α k a,b in this constraint.α k a,b has to satisfy the condition that it is the first opening time after the end of its previous hop.We use A(x) to check the condition.Thus, (10) where port(f a , π a,b ) denotes the port that in node π a,b , f a is sent from, and b starts from 2 because the first node in the routing path is not a TSN switch.In A(x), if the value x satisfies the condition, and if, before the value x, there is no value that satisfies the condition, then A(x) is true.B(x) is used to check if the gate is opened at time x.
If there exists an entry e i,y that is the last entry to control the queue of flow f a before time x, and if the entry is to open the queue of flow f a , then B(x) is true.
If an entry is in the time duration, i.e., α k a,b ≤ t i,y ≤ β k a,b , the corresponding gate must be opened.

B. AN ALGORITHM BASED ON OMT
As described in FIGURE 4, to improve the scalability of our proposed algorithms, we break down an SMT problem into multiple OMT problems.These OMT problems are the subproblems of the original SMT problem and independent of each other.For each subproblem, an OMT solver needs to be invoked once to schedule packets, where is a parameter given by users or selected based on the execution time of the OMT solver to make a trade-off between execution time and efficiency.As the problem size increases, the number of subproblems increases, while the subproblem size is not changed.Thus, the OMT solver is invoked more times, but the execution time of each invocation is basically unchanged.Therefore, the execution time of scheduling all the packets increases linearly with the problem size.
Before dividing the original problem, we must assign queues to flows because if the queue assignment is divided into multiple subproblems, the packets that belong in the same flow may be assigned to different queues.Algorithm 1 shows our assignment method.The heavier the workload is, the greater the number of conflicts that occur, resulting in more gate controls being needed to isolate transmissions.Our objective of assigning queues is to balance their workloads.
In Algorithm 1, we first calculate the utilization ρ a that a flow introduces to a node (line 1).Then, each flow selects the queue with the minimal workload to use (lines 3-5), and the utilization of the selected queue is updated (line 6).
ρt,i,j,g denotes the utilization of the g-th queue in n i,j at Algorithm 1 QA (Queue Assignment) Input: a network < N , W , L >, a flow set F Output: ∀f a ∈ F, q a 1: ∀f a ∈ F, ρ a = γ a d a ; 2: sort flows in decreasing order of their utilization, where f 1 has the largest utilization; 3: for a = 1 to |F| do 4: q a is assigned the h-th queue; 6: ∀n i,j ∈ a , ρ1,i,j,h = ρ1,i,j,h + ρ a ; 7: return all q a ; time t.To find the maximum ρ, all possible t should be checked.However, we know that all flows release packets at time 1, i.e., the maximum utilization must occur at time 1.Therefore, we consider only ρ1,−,−,− in the algorithm.The time complexity of Algorithm 1 is Based on the result of Algorithm 1, we divide the original problem into multiple subproblems.There are packets in a hyperperiod.We divide them into ∀fa∈F H pa subsets.First, we sort all packets in the order of their earliest deadline first (EDF) priorities.EDF priorities mean that a packet with an earlier deadline has a higher priority.In the sorted packet set {s 1 , s 2 , ...}, the elements s u× +1 , . . ., s (u+1)× are assigned to the subset S u .This assignment method schedules conflicting packets in the same subset so that the OMT solver can resolve conflicts more efficiently.
Since different packet subsets require different numbers of gate control entries, we cannot determine how many entries should be used in each subset.Therefore, the objective of each OMT subproblem is to minimize the maximum number of entries used, i.e., obj.min ē, subject to ∀n i ∈ W , where ē denotes the maximum number of entries used among all switches, and λ i,y denotes if the corresponding entries are used.In addition to the two constraints, Constraints 1) to 7) also need to be guaranteed.Algorithm 2 shows the process of invoking the OMT solver.For each subset S u , the OMT solver is invoked to find an optimal solution (lines 1-2).Then, all the variables in the solution are set to constants and cannot be changed in subsequent invocations (lines 3-7).This process is repeated until all subsets finish.Except when invoking the OMT solver, the other codes have O(n) time complexity.n is approximately the number of packets.for each e i,y do 6: the entry e i,y can no longer be changed;

V. HEURISTIC SCHEDULING ALGORITHMS
To handle massive data, scheduling algorithms should be simple and fast.In this section, we first present a fast scheduling algorithm (in Section V-A), and analyze the end-to-end delays (in Section V-B).Then, based on the analytic result, we modify the fast scheduling algorithm to improve its efficiency (in Section V-C).

A. NO-GATE-CLOSING SCHEDULING
There is a scheduling algorithm that can use one entry and one queue to transmit all packets.We call it the no-gate-closing (NGC) scheduling algorithm.In the NGC algorithm, all gates are always opened.After a switch receives a whole packet, it sends the packet to the next hop immediately.Source nodes control the injection times of all packets to avoid conflicts, and no queue is needed to block packets.Therefore, for each egress port, only one queue is used to buffer packets.This transmission rule is similar to that in no-wait packet scheduling [30].However, we focus on a different problem model, and the no-wait scheduling algorithm is not suitable for massive data due to its execution time.for each f a ∈ F do return all η k a ; The NGC algorithm (as shown in Algorithm 3) uses the classical EDF policy to schedule massive data since it has high real-time performance [18] in many real-time systems.The schedule is nonpreemptive.Once packets start to be transmitted, they cannot be blocked.In a hyperperiod, the network runs from time 1 to time H .At each time t, the newly released packets are added to the set S, which contains all released but unscheduled packets (lines 2-5).Then, the highest priority packet in set S is selected (lines 7-8) and checked by the function NoConflict( ) to determine whether it can be scheduled without conflicts at time t (line 11).As shown the example in FIGURE 5(a), the paths of flow f 1 and f 2 are {A, C, D, E, F} and {B, C, D, E, G}, respectively, and the corresponding transmissions are shown in FIGURE 5(b) and (c).Up arrows denote release times, and down arrows are deadlines.There is no time gap between successive transmissions since transmissions are not blocked.The two flows generate their first packets at time 1.The packet of f 1 is first transmitted since it has a higher EDF priority than f 2 .If the packet of f 2 also starts to be transmitted at time 1, the two packets will conflict in node C as shown in FIGURE 5(b).Hence, the packet of f 2 cannot start at time 1.The packets that have conflicts with the scheduled packets are removed to the set S to be checked again at the next time (lines 13-15).After all packets in S are checked (line 6), the same process is repeated at the next time.Until the packet does not conflict with the scheduled packets, it can be injected into the network (line 12), such as the packet of f 2 is injected at time 2 as shown in FIGURE 5(c).η k a denotes the injection time of packet s k a .The second packet of f 1 is released at time 33.However,

B. ANALYSIS
We analyze the worst-case end-to-end delay in Algorithm 3 so that we can find the key factors affecting the real-time performance.In the following, we focus on the packet s k a , the worst-case end-to-end delay of which is denoted as x.The delay x includes the end-to-end delay γ a and the conflict delay D(s k a , x) introduced by other packets in the time interval [σ k a , σ k a + x], where σ k a is the release time of s k a and is equal to p a × k + 1.Hence, First, we calculate D(s k a , x).We use four packet sets HB(s k a ), HA(s k a ), LB(s k a ), and LA(s k a ) to denote the four kinds of packets that will introduce conflicts to packet s k a .The letter H (or L) indicates that the packets the corresponding sets have higher (or lower) EDF priorities than s k a , and the letter B (or A) indicates that the packets in the corresponding sets are released no later than (or after) the release time σ k a of packet s k a .For example, the set LB includes the packets that have lower priorities and are released no later than the time σ k a .A lower-priority packet can delay a higher-priority packet when the higher-priority packet has been blocked by other packets, with the lower-priority packet beginning the transmission process before the higher-priority packet.Since critical packets cannot be preempted, the higher-priority packet has to wait until the lower-priority packet releases the shared resources.For the packet s k a , we use the time interval [µ1 a,g , µ2 a,g ] to denote its conflict interval, where the earliest conflict time µ1 a,g of flow f a is the relative start time of the first transmission that uses the same node as that of f g , and similarly, the last conflict time µ2 a,g is the relative end time of the last transmission conflicting with f g (as shown in FIGURE 6).For each packet s m g in the four sets, there must be g does not affect the transmissions of s k a .Once we know the release times and deadlines of all the packets, we can then find the packets of the four sets.Thus, The calculations of the four delays are as follows.

1) D HB
First, we consider all the transmissions of s m g as being in the time interval [σ k a , σ k a + x].Thus, the delay introduced by conflicts with s m g when it starts to be transmitted t time units after s m g ; otherwise, C(s k a , s m g , t) = 0.In some cases, not all the transmissions of s m g are in the time interval [σ k a , σ k a + x].As shown in FIGURE 7(a), if the front part of s k a overlaps with the rear part of s m g , the length of the overlapping time is σ m g +d g −σ k a .Thus, when part of s m g is in the conflict interval, i.e., (σ m g + d g − σ k a ) < γ g , only the last (σ m g + d g − σ k a ) transmissions can introduce delay.Therefore, the delay is . We obtain that where

2) D HA
All packets in HA(s k a ) have higher EDF priorities and are released after σ k a .Thus, compared to D HB , there is no case where only some of the transmissions affect s k a .Therefore, 3

) D LB
All packets in LB(s k a ) have lower EDF priorities and are released no later than σ k a .In the worst case the latter γ g − 1 transmissions will conflict with s k a .This is the only difference between D HB and D LB .Therefore, the first transmission is excluded, i.e., All packets in LA(s k a ) have lower EDF priorities and are released after s k a .In the worst case, the latter γ g − 1 transmissions will affect s k a .As shown in FIGURE 7(b), the overlapping time is σ k a + x − σ m g .When γ g > σ k a + x − σ m g , i.e., only some of the transmissions will affect s k a , and thus where Based on the above analysis and Equation ( 17), x can be solved by the following recurrent relation: x i+1 is calculated repeatedly until x i+1 and x i have the same value.Then, x is assigned the final x i+1 .
From the analysis, we know that there are two key factors that delay flows.The first one is priority inversion, where lower-priority packets delay higher-priority packets.This weakens the efficiency of priorities.The second one is that a packet may suffer all conflicts in series.To cover the worst case, the analysis is pessimistic.However, in an actual schedule, the delay can be reduced by parallelism.

C. MOVE-FORWARD SCHEDULING
In this subsection, we propose a scheduling algorithm to reduce the effect of the two factors on end-to-end delays.First, we schedule packets in the strict order of their priorities.The classical EDF policy is designed for multitask systems, where the deadlines of unreleased tasks may be unknown during algorithm running.Thus, the classical nonpreemptive EDF policy selects tasks to be scheduled from the currently released tasks.However, in our model, all the deadlines are known.Thus, we can schedule packets in the strict order of their EDF priorities (as shown in Algorithm 4) so that lower-priority packets do not delay higher-priority packets.In Algorithm 4, all packets are sorted in decreasing order of their EDF priorities (line 1).Then, for each packet, the algorithm checks whether it can start to be scheduled without conflicting with the other scheduled packets at time t (lines 2-4), where t is from its release time to its deadline.The first no-conflict time is its injection time, and then, the packet can be transmitted without blocking (line 5).If there is no time t that satisfies NoConflict(s k a , t), then the packet is moved to the set Ŝ to await subsequent processing (lines 6-7).

The time complexity of Algorithm 5 is O(|S||H ||H |).
Algorithm 4 SPS (Strict Priority Scheduling) Input: network < N , W , L >, flow set F Output: η k a and Ŝ 1: add all packets into a set S in increasing order of their deadlines; 2: for s k a = the first packet to the last one in S do 3: if there is no t that makes NoConflict(s k a , t) true then 7: Ŝ+ = {s k a }; 8: return all η k a and Ŝ; Second, we propose a strategy to improve the parallelism of packets.Recall that in Algorithm 2 and 4, transmissions cannot be blocked.Thus, if two packets conflict with each other, all transmissions of one of them have to be delayed.This leads to a decrease in parallelism.Closing gates is a good way to improve parallelism because the transmissions of a packet can be transmitted discontinuously to use more resource fragmentation.Therefore, our proposed scheduling algorithm closes and opens gates based on the following three properties.
Property 1: If a packet can be transmitted continuously before its deadline, it can also be transmitted discontinuously before its deadline.
Discontinuous scheduling is a special case of continuous scheduling.The resources used by continuous scheduling can also be used by discontinuous scheduling.Therefore, in the worst case, discontinuous scheduling is the same as continuous scheduling.Resource(s a ) denotes the network resources required to transmit packet s a .FIGURE 8 illustrates Property 2. The unit of the horizontal axis is time, and the letters of the vertical axis correspond to nodes.The red (blue) blocks represent when the corresponding nodes are used to transmit s a (s b ).The gray blocks represent the resources that have been occupied by other higher-priority packets.s a has a higher priority than s b .One-hop transmissions of s a and s b require 8 unit times and 5 unit times, respectively.The overlap time between two adjacent hops is 1 unit time.When s a is scheduled continuously, it has to be injected into the network at time 25 because there is no continue resource at the front.Thus, s b can use the idle resources between time 3 and 15 (as shown in FIGURE 8(a)).In this case, the two packets can be transmitted before their deadlines.However, when s a is discontinuous, it will occupy the resources between time 1 and 25 (as shown in FIGURE 8(b)).The transmissions of s a are broken between nodes B and C. Since node D is occupied by other packets, the last hop of s b has to be transmitted between time 46 and 51, i.e., s b misses its deadline time 50.Therefore, discontinuous scheduling may make packets that are schedulable in other algorithms unschedulable.
Property 3: Packet s a , s b ∈ S, and a < b.For packet s a+1 ,..., s b−1 , if there exists a subset {s 1 , . . ., s j }, and then packet s a may delay packet s b .
Based on Property 2, we know that the discontinuous s a can affect the transmissions of s 1 , and then s 1 affects s 2 .Repeat this process, and then s j affects s b .Thus, the discontinuous s a will affect s b .
Our proposed algorithm is shown in Algorithm 5. We first invoke Algorithm 4 SPS() to generate schedules sch[ ][ ][ ] (lines 1-2), and create a set Ŝ to mark all unscheduled packets and a set S to mark the packets that affect the unscheduled packets based on Properties 2 and 3 (line 5).If SPS() can schedule all packets, the following algorithm does not need to run (lines 3-4).If not, we reschedule the packets in Ŝ and S under the discontinuous scheduling rule (lines 6-23).There are two reasons why our rescheduling algorithm can improve schedulability: first, based on Property 1, rescheduling a scheduled packet does not make it unschedulable; second, discontinuous transmissions will use resource fragmentation at the front and leave more available resources for unscheduled packets.
NoConflict(s k a , h, t) = 1 denotes that the h-th hop of s k a can be scheduled without conflicting with scheduled packets at time t (line 10).For a packet in Ŝ and S, we schedule its hops separately before its deadline (lines 8-9 and 16-18).The new schedules are written in the schedule array sch (line 11).
If two successive hops cannot be scheduled continuously, the entries need to be set to control the corresponding gate (lines 12 and 15).However, if there is no unused entry, the flow set cannot be scheduled (lines [13][14].Repeat this process to schedule all packets in Ŝ and S until a packet misses its deadline (lines [21][22] or all schedules are generated and returned (line 23).We compare our OMT (Algorithm 2), NGC (Algorithm 3), and MF (Algorithm 5) algorithms with three fundamental methods ORG, EDFT and UP.ORG is the method used in the current TSN switches, in which gates must be closed immediately after they finish a transmission.EDFT schedules each hop based on their EDF priorities.UP is a conservative upper bound of optimal results.Since, to verify the performance of our algorithms, our evaluation results should be compared with optimal results.However, our problem is NPcomplete, and SMT/OMT solvers cannot solve large-scale test cases.There is no method to obtain accurate optimal results.Thus, we have to compare our proposed algorithms with the conservative upper bound UP.In UP, if test cases satisfy the following necessary condition, they are considered schedulable.The necessary condition for schedulability is that for each port in a switch, its unidirectional utilization cannot be greater than 100%.Since UP is better than optimal results, if our results are close to UP, they must be close to optimal results.Note that UP is a baseline for comparisons and does not generate schedule tables.OMT is solved by the Microsoft solver Z3 [36], [37].All algorithms are written in C and run on a Windows machine with a 3.4 GHz CPU and 16GB of memory.
Our test cases are generated based on the given parameters, which are summarized in TABLE 1.For each test case, n switches with Q queues are randomly deployed in a square area.Each switch connects a gateway and three adjacent switches.f flows randomly select their source nodes and destination nodes.Periods and packet sizes are random numbers in the period range p and the packet size range s, respectively.The deadline of a flow is also a random number between its transmission delay and its period.We use a mass of random test cases to verify our algorithms so that their universality can be demonstrated.Before running algorithms, to avoid unnecessary computations, we use UP to pre-filter the unschedulable test cases.If the test cases cannot satisfy UP, they are discarded.The pre-filtering operation is based on the necessary condition and does not affect our evaluation results. is used to control the execution time of OMT.
In the following evaluation, we first compare all the algorithms, and the comparison is shown in FIGURE 9 of Section VI-A.To verify the universality of the algorithms, for each parameter setting, 10000 test cases are generated and solved.Due to the long execution time of Z3, the scale of test cases is limited.Therefore, in Section VI-B, we compare the other algorithms except OMT (Z3) using large-scale test cases.In Section VI-B, we set the parameters to different values so that the algorithms can be tested fully.FIGURE 10 shows the comparisons under varying f and n.FIGURE 11,12 and 13 show the comparisons under varying Q, p and s, respectively.Thus, our proposed Algorithm 2, 3, 4 and 5 are evaluated completely.Finally, we evaluate Algorithm 1 that assigns queues in OMT, and show the evaluation results in FIGURE 14.

A. EVALUATIONS WITH Z3
To make test cases solvable by Z3, we set n = 6, Q = 2, p = [32, 64], s = [100, 400], f ∈ [10,50] and ∈ [2,14].The average results of 10000 test cases are shown in the subsequent figures.FIGURE 9 shows the comparison of schedulable ratios, the number of schedule entries, and the execution times under varying f and .Since does not affect the other algorithms except OMT, our proposed algorithms are not shown in FIGURE 9(b) and (d).In FIGURE 9(a) and (b), the schedulable ratios are normalized, with UP as the baseline.As f increases, NGC decrease rapidly, and the others decrease slowly.For NGC, the greater the number of flows, the greater amount of resources that are fragmented, and the  more conflicts that are introduced to high-priority packets.Thus, NGC has the worst schedulable ratio.OMT calls the solver Z3 to minimize the number of schedule entries.In each call, an attempt is made to satisfy the schedulability, while OMT cannot optimize the schedulability between multiple calls.Thus, our proposed MF algorithm has higher schedulable ratios than those of OMT.EDFT provides a flexible strategy to schedule flows.However, it still cannot avoid the conflicts introduced by lower-priority packets.Thus, it is better than NGC but worse than the other algorithms.In FIGURE 9(b), as increases, the schedulable ratio of OMT   almost does not change.This is because the setting of test cases does not change.
FIGURE 9(c) and (d) show a comparison of the number of schedule entries.As f increases, the other algorithms except NGC increase.NGC is allowed to use only one schedule entry.If one entry is not enough to schedule all flows, NGC cannot generate schedules.For the same network settings, the greater the number of flows, the greater the number of entries that are needed to isolate packets.Hence, the other algorithms exhibit an increase.ORG closes the corresponding gates after finishing transmissions, while only when transmissions need to be blocked do the other algorithms close gates.Therefore, ORG consumes the most entries.In EDFT, if two transmissions are continuous, no schedule entry is needed.Thus, EDFT requires fewer schedule entries than ORG.The objective of OMT is to minimize the number of required schedule entries.Hence, it requires the least number of schedule entries.Although our proposed MF algorithm requires more entries than does OMT, MF has the higher schedulable ratio and a shorter execution time.For all the test cases in this subsection, MF can finish within 1 ms, while the execution time of OMT is approximately a few minutes.As the number of flows increases, the execution time of OMT increases linearly (as shown in FIGURE 9(e)).Based on this trend, OMT will require less than two days to solve a test case with 10000 flows and 40000 packets.However, for the original SMT specification shown in Section IV-A, it is impossible to obtain the result of any large-scale test case.We tried to solve a test case with 100 flows based on the SMT specification but did not obtain the result within two days.Therefore, when a flow set needs to be scheduled, MF is used first because it is fast and efficient.Then, if MF requires more schedule entries than those supported by network devices, OMT can be used to find a more optimized solution.

B. EVALUATIONS WITHOUT Z3
To fully evaluate our proposed algorithms, we vary every parameter to compare the results.The basic settings are n = 20, Q = 4, f = 6000, p = [4096, 32768], and s = [100, 1500].In the following evaluation, if these parameters are not specified, the basic settings are used.In FIGURE 10, the parameters f and n are changed.FIGURE 10(a) and (b) show schedulable ratios that are normalized with UP as the baseline.NGC is still the worst one.When f > 3000, NGC cannot schedule any test case.Thus, in the following comparisons, only FIGURE 10(e) shows three points corresponding to NGC.As f increases, the schedulable ratios decrease.This is because the greater the number of flows, the more difficult the scheduling becomes.MF can avoid priority inversion, while in EDFT, priority inversions still occurs between transmissions.Thus, MF is better than EDFT.As shown in FIGURE 10(b), the schedulable ratios increase as n increases because the greater the number of switches, the fewer the number of conflicts that occur.
FIGURE 10(c) and (d) show the number of required schedule entries.Since the flow set that is difficult to schedule requires more schedule entries, the trend of the lines in the two figures is opposite to that in FIGURE 10(a

) and (b).
There are approximately 40000 packets in a test case with 10000 flows.ORG has to spend many schedule entries to provide complete isolation for these packets.Our proposed MF algorithm requires the fewest schedule entries because the strict priority scheduling requires only one entry in each switch, and the selection of moving packets limits the number of operations on gates.Compared to ORG, in the best case, MF requires only one-twentieth the schedule entries to schedule the same flow set.
The related work in [29] considers the number of schedule entries in its problem model.However, the work uses only 64 entries to schedule small networks.Due to the unacceptable execution time of SMT solvers, the work cannot use more entries.Although, from the evaluation shown in [29], we can deduce that maybe a switch with 1024 entries can schedule more than 1024 flows, we do not know the exact number of flows and cannot compare it with our results.
The datasheet [14] of the NXP switch chip mentions that the chip with 1024 entries can support 1024 deterministic flows.The datasheet does not explain why the number of deterministic flows is 1024.However, this is the only exact number we found.Therefore, we compare it with our results.In our algorithms, as shown in FIGURE 10(c), the switch with 1024 entries can deterministically schedule approximately 8000 flows, which include more than 30000 packets.Thus, our proposed algorithm offers approximately 8 times more schedulability than does the suggestion in the datasheet [14].
FIGURE 10(e) and (f) show the execution time.MF is based on fixed priorities and does not need to find the highest priority packet for each schedule.However, EDFT not only needs to find the highest-priority one at every scheduling time but also finishes only one transmission after each discovery.This is the reason why MF can finish within 1 minute, while EDFT spends up to 1 hour.As f increases, the execution time spent to search the solution spaces increases.As n increases, the execution time slightly fluctuates.The reason is that the scheduling algorithms traverse flows rather than switches.
FIGURE 11 shows the comparison a varying number of queues.To make the difference obvious, we set n = 8.As Q increases, the schedulable ratios slightly increase.This is because having more queues helps to eliminate some of the queue conflicts, though the link conflicts still exist.Comparing FIGURE 10(b) and FIGURE 11(a), we find that link conflicts have more impact on schedulable ratios than do queue conflicts.The greater the number of queues, the more entries are needed to control gates.Thus, the number of schedule entries also increases as Q increases.
FIGURE 12 shows the comparison under different p.All the results are solved by our MF algorithm.When the range of periods decreases, the high real-time packets are eliminated.Thus, the test cases are easy to schedule.FIGURE 13 shows the comparison under different packet sizes.When the range of packet sizes becomes narrow, there are no small packets to make use of resource fragmentation.Therefore, the schedulable ratio reduces, and the number of schedule entries increases.
FIGURE 14 shows the comparison between our proposed queue assignment algorithm (QA) and a method (PR) currently used.In current applications, high-priority flows are assigned to high-priority queues.In the compared PR method, flows are evenly assigned to queues based on their priorities.This method does not consider the workload assigned to a switch.If a switch has a greater workload, it needs more entries to isolate flows.Therefore, the methods with PR have lower schedulable ratios and spend more schedule entries than those with our proposed QA method.

VII. DISCUSSION
For scheduling problems of TSNs, there is no benchmark.Thus, we have to vary all the parameters related to networks and flows to generate extensive test cases such that our evaluations can cover as many situations as possible.From the evaluation results, we can conclude that our proposed algorithms OMT and MF are better than the others.This is because our algorithms use the following two strategies to reconcile the contradiction between the limited number of schedule entries and massive data.The first one is to control the injection times of packets to reduce conflicts.The second one is to use one schedule entry to isolate as many packets as possible.These are also the reasons why our proposed algorithms can tackle the two challenges mentioned in Section I.
Our algorithms are based on software and can be applied to all of the networks following the standard IEEE 802.1Qbv.For example, our algorithms can be used in a 5G system to solve practical issues, since TSNs are defined as the fronthaul networks of 5G systems in the IEEE standard 802.1CM, and transmitting massive data is one of the three use cases supported by 5G.Even if in the future a switch chip may include a large schedule table, our algorithms can still break through the limitation of the chips and make a switch support more communications.
In industrial applications, almost all communications are fixed and known, while bursty communications still exist, such as emergency alarms.Our algorithms are designed to handle known communications.When a bursty packet needs to be transmitted, schedule tables have to be reconfigured.This reconfiguration process will introduce a long delay that may reduce the real-time performance of industrial networks.Therefore, in the future, we will propose distributed scheduling algorithms to handle bursty communications.

VIII. CONCLUSION
TSNs, as the backbone network of industrial internet of things, have to schedule massive real-time data packets.However, the off-the-shelf TSN switches only contain limited schedule entries.This causes that they cannot support massive real-time packets.Therefore, in this paper, we propose the algorithms that can schedule massive real-time data with a limited number of schedule entries.First, to find the optimal results of the NP-complete problem, we formulate it as an SMT specification.The SMT/OMT solver Z3 can find optimal solutions for SMT specifications.However, due to the complexity of the problem, Z3 cannot even find any feasible results for a small test case in two days.Hence, we divide the SMT specification into multiple OMT specifications and still use Z3 to solve them.The evaluation results indicate that the execution time of OMT increases linearly as the number of flow increases.In two days, using the OMT-based algorithm, we can obtain feasible solutions for test cases with 10000 flows.Second, to obtain solutions quickly, we propose heuristic algorithms.Based on an initial algorithm NGC, we find two factors that make flows difficult to schedule.Then, we propose the MF algorithm to reduce the impact of the two factors on the schedulability.Compared with existing heuristic algorithms, our proposed MF algorithm requires only at least one-twentieth the number of schedule tables to schedule the same number of packets.Therefore, if users want to obtain feasible solutions quickly, MF is a good choice; if the usage of entries is the most important, the OMT-based algorithm can be used.

FIGURE 1 .
FIGURE 1. Architecture of a TSN switch with 4 ports.

FIGURE 3 .
FIGURE 3. Delay of a packet in a switch.
H p a )) to distinguish the packets that are generated by flow f a .For the k-th packet generated by flow f a , its b-th hop transmission is called transmission τ k a,b .We use α k a,b and β k a,b to denote the starting time and the ending time of transmission τ k a,b , i.e., β k a,b − α k a,b = γ + a .Thus, the SMT problem has to respect the following constraints.

7 )
Time duration validity constraint: During the time duration [α k a,b , β k a,b ], the corresponding gate must always be opened to send τ k a,b continuously.

VOLUME 8, 2020 Algorithm 2
OMT-Based Scheduling Algorithm Input: network < N , W , L >, flow set F Output: all schedule tables 1: for each S u do 2: if OMT (S u ) found a solution then 3: for each packet in S u do 4: the variables α k a,b , β k a,b (∀b ∈ [1, | a | − 1)) are assumed to be constants in the following calculation; 5:

10 :
return all E i ;

FIGURE 7 .
FIGURE 7. Some of the transmissions in a conflict interval.

Property 2 :
Packet s a , s b ∈ S, and a < b.If Resource(s a ) ∩ Resource(s b ) = ∅, and s a is transmitted discontinuously, then s a may introduce more delays to the lower-priority packet s b .
CHI XU received the Ph.D. degree from the University of Chinese Academy of Sciences, China, in 2017.He has been with the Shenyang Institute of Automation, Chinese Academy of Sciences, China, since 2013, where he currently is doing his postdoctoral research and serving as an Assistant Professor.His research interests include cognitive radio networks, industrial wireless networks, tactile internet, and 5G URLLC.He is a Voting Member of the IEEE 1918.1 Working Group for Tactile Internet as well as a member of the IEEE 1918.1.1Task Group for Haptic Codec.He is also a member of the IEEE 1932.1 Working Group for Licensed/Unlicensed Spectrum Interoperability in Wireless Mobile Networks.He serves as a 3GPP standardization delegate in the Technical Specification Group (TSG) for Radio Access Network (RAN).DONG LI received the Ph.D. degree in mechatronic engineering from the Shenyang Institute of Automation, Chinese Academy of Sciences.He is currently involved in research on industrial networks and software-defined networking for intelligent manufacturing.YUE YIN received the master's degree in technical economy and management from the Business School of Liaoning University, in 2013.He is currently pursuing the Ph.D. degree in technical economics with the School of Economics, Liaoning University.His research interests include network management and optimization algorithms.PENG ZENG received the Ph.D. degree from the Shenyang Institute of Automation, Chinese Academy of Sciences.He is currently a Professor with the Shenyang Institute of Automation, Chinese Academy of Sciences.His research interests include industrial communication and wireless sensor networks.VOLUME 8, 2020