Adaptive Group Routing and Scheduling in Multicast Time-Sensitive Networks

Traditionally, industrial networks rely on field bus to support real-time applications. In the trend of industrial internet of things (IIoT), traffic load has increased dramatically, and industrial networks are moving towards time-sensitive networking (TSN), which is based on the widely used Ethernet. In TSN, the time-triggered mechanism is a promising technology for real-time control due to its ability to provide deterministic latency and jitter guarantees. However, this mechanism is mainly based on offline computed schedules, making it difficult to satisfy dynamic requirements in data transmission of IIoT. In this paper, we propose an adaptive group routing and scheduling (AGRS) approach in multicast time-sensitive networks. The framework of this approach contains three phases including pre-processing, schedule synthesis, and post-processing phases. While the pre-processing phase first prunes unnecessary links and adaptively groups qualified complete subgraphs of the network to obtain a simplified topology, the schedule synthesis phase then jointly considers routing and scheduling processes based on the simplified topology to improve schedulability. Finally, the post-processing phase handles the scheduling results of previous phases to generate a feasible schedule. Experiments show that our approach improves schedulability by 68% and reduces execution time by 74% for multicast time-sensitive networks compared with existing approaches.


I. INTRODUCTION
With the proliferation of distributed sensors, surveillance cameras and actuators, cyber-physical systems (CPS) emerge as the mechanisms monitoring and controlling physical processes, which transmit data and commands through computer networks. Typically, CPS have stringent timing requirements, and the performance of networks has a significant impact on the quality of CPS control. For instance, a motion control application in industrial automation may require that the time of control cycles is measured in a few microseconds [1]. In addition, a majority of industrial applications, such as trains and ships, use multicast networks for system control. Therefore, multicast time-sensitive networks that provide deterministic latency and latency variance (jitter) are urgently desired in order to ensure real-time control for CPS.
The associate editor coordinating the review of this manuscript and approving it for publication was Guangjie Han .
Generally, there are two types of application data in industrial networks: real-time data for system control and best-effort data for ancillary service. The real-time data, with fixed period and size, must arrive at all the destinations within its deadline and satisfy the jitter constraints. In contrast, the best-effort data requires no timing guarantees. Traditionally, industry networks achieve deterministic guarantees for real-time data by dedicated field bus [2]. However, traffic load has increased dramatically in the trend of industrial internet of things (IIoT). Therefore, the low bandwidth, typically 1Mbps, and limited maximum number of supported devices number in the obsolete field bus cannot address this new tendency.
In the last decade, Ethernet standardized in IEEE 802.3 [3] has been widely used in the local area networks (LAN) for offices and families. Compared to the networks based on field bus, Ethernet brings higher bandwidth up to 100 Gbps and better compatibility with existing network devices. Although the IEEE 802.3 Ethernet is initially designed for best-effort VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/ data and thus cannot provide real-time communication services [4], various real-time Ethernet technologies, such as ProfiNET [5], EtherCAT [6], FTT [7], TT-Ethernet [8], have been proposed to achieve hard real-time guarantees for Ethernet. In addition, the Deterministic Networking (DetNet) Working group and IEEE 802.1 Time-Sensitive Networking (TSN) Task Group strive to promote the standardization of real-time networks to provide deterministic end-to-end latency and jitter for time-sensitive applications. Moreover, the recently released IEEE 802.1 Qbv [9] standard schedules real-time data in a time-triggered mechanism. The time-triggered mechanism provides latency and jitter guarantees by following two steps: (1) Before network deployment, a schedule synthesis procedure receives the timing parameters of real-time flows as input and generates a valid scheduling table for these flows. (2) While the network is operating, switches schedule the real-time flows according to the pre-computed scheduling  [12], the trend of IIoT brings new challenges from dynamic applications [13]. These dynamic applications occasionally introduce new requirements of data transmission in time sensitive networks, for instance, adding a plug-and-play device in the production line [14]. Therefore, a novel scheduling approach is required to schedule the new requirements without disturbing existing schedule, as schedule modifications may lead to a violation of latency and jitter bounds for the existing flows, resulting in unpredictable consequences [15]. Moreover, this scheduling approach is able to take full advantage of available bandwidth to accept new transfer requests as much as possible, and achieve low execution time to provide a rapid response to these new requests.
In this paper, an adaptive group routing and scheduling approach is explored to maintain the quality of service (QoS) of dynamic applications in multicast time-sensitive networks. In this approach, the network model and the routing and scheduling problem are first formalized respectively. As computational complexity is closely related to the topology complexity, a pre-processing phase is devised to simplify the network topology, which prunes unnecessary links and adaptively groups qualified complete subgraphs. While existing approaches [10], [11], [16] use the Steiner Tree as default routes, a routing and scheduling (RS) procedure is devised in the schedule synthesis phase, which jointly considers routing and scheduling processes to improve schedulability. The RS procedure is based on the simplified topology processed by the former phase, and thus the execution time of schedule synthesis is sharply reduced. Furthermore, a post-processing phase is proposed to handle the results of the schedule synthesis phase, which uses a counter example strategy to generate a feasible schedule. Finally, the experiments show that our approach can achieve better schedulability and lower execution time for multicast time-sensitive networks compared with existing approaches. In summary, the main contributions are as follows: • An adaptive link grouping strategy is proposed to reduce the execution time of the schedule synthesis process.
• A routing and scheduling (RS) procedure that jointly considers routing and scheduling is developed to improve the schedulability for multicast time-sensitive networks.
• A 24-port TSN switch supporting on-the-fly schedule reconfiguration is implemented, and the approach is validated on the hardware platform.
The paper is organized as follows: In Section II, the related work is discussed. Then, the network model is illustrated and the routing and scheduling problem is formalized for the multicast time-sensitive networks in Section III. After the AGRS framework is discussed in Section IV, the pre-processing phase, schedule synthesis phase and post-processing phase are presented respectively in Section V, VI, and VII. Finally, experimental results show the performance in Section VIII, and the paper is concluded in Section IX.

II. RELATED WORK
Message scheduling in real-time networks is known to be an NP-complete problem [12], and various researchers have explored methods of improving and optimizing schedule synthesis for decades. Traditionally, real-time networks are analyzed in regards to worst-case end-to-end delay through network calculus [17], [18]. The major drawback of this approach is that the delay analysis is based on the pre-assigned priorities and the arrival patterns of competing flows, which makes it difficult for the compositional system design and the communication isolation.
Due to its deterministic and composability [19], [20], time-triggered networks, such as FTT-CAN [21] and FlexRay [22], have been investigated in detail. However, these approaches are based on the traditional field bus, and thus are not suitable for large-scale and high-bandwidth networks based on the widely used Ethernet. For multihop time-sensitive networks based on the Ethernet, the approaches, such as TT-Ethernet [10] and ProfiNet [12], have been proposed to synthesize offline schedules. These approaches first generate a sufficient constraint set of transfer requirements as the input, and then use satisfiability modulo theory (SMT) or integer linear programming (ILP) to synthesize a schedule satisfying these constraints. For large-scale networks, these approaches have been extended to improve capability and performance [23], [24]. Furthermore, these approaches have also been extended to generate gate control list for IEEE 802.1 Qbv networks [16], [25]. Heuristic approaches [12], [26] to scheduling the flows based on the constraint set have been discussed. F. Dürr, et al. [27] formalize the schedule synthesis problem as a no-wait packet 37856 VOLUME 8, 2020 scheduling problem (NWPSP), and use Tabu search to generate near-optimal solutions.
All of the aforementioned approaches separate routing and scheduling processes, and assume that the routes of flows are given as a prior. Generally, the Steiner Tree is used as the default routes. However, the separation of routing and scheduling processes reduces the solution space, and critical paths may become the bottlenecks of scheduling. Schweissguth et al. [28] and Smirnov et al. [29] propose approaches that jointly consider routing and scheduling for unicast time-triggered networks. However, due to lack of optimization, the greatly increased solution space makes it unsuitable for large-scale networks. Nayak et al. [30] propose an approach that schedules the transmission of time-triggered flows only on the hosts. Although online scheduling for dynamic applications can be achieved by this approach, the network diameter of this approach is restricted to seven hops, which also fails to satisfy the requirements of large-scale networks.
As a contrast, our approach not only jointly considers routing and scheduling processes to improve schedulability, but also optimizes the solving process to reduce the execution time for large-scale networks.

III. SYSTEM MODEL AND PROBLEM DEFINITION
In this section, the system model of multicast time-sensitive networks are first formalized, and then the routing and scheduling problems considered in this paper are defined.

A. SYSTEM MODEL
The time-sensitive network is modeled as an unweighted directed graph G =< V , E >, where V is the set of network nodes, consisting of the set of switches SW and the set of end devices ED, and and v i , v j are connected by a link} is the set of directional network links.
| V | and | E | are the number of nodes and links respectively.
The whole network is assumed to be precisely synchronized.
A set of multicast flows F with latency and jitter constraints is considered in the time sensitive networks. Each flow f a ∈ F is denoted by a tuple < s a , D a , p a , l a , ddl a >, where s a ∈ V is the source, D a ⊆ V is the set of destinations, p a is the transmission period, l a is the maximum length of the flow and ddl a is the transmission deadline. Without loss of generality, all these parameters are assumed as positive integers. To ensure the real-time features of the networks, the flow f a is required to transmit from the source s a to all the destinations D a within its deadline ddl a .

B. PROBLEM DEFINITION
Two types of variables are introduced to illustrate the routing and scheduling problems respectively. To address the routing problems, a binary routing variable lo [ To address the scheduling problems, an integer scheduling In addition, a schedule S is uniquely determined by the tuples below: Due to transmission collision and lack of latency planning, an arbitrary schedule cannot provide real-time guarantees. To ensure this feature, the tuples of a schedule S should satisfy certain constraints, which are called time-sensitive constraints. The routing and scheduling problems in multicast time-sensitive networks are defined as synthesizing a valid schedule S v , whose tuples satisfy all the time sensitive constraints. The time sensitive constraints are discussed in detail in Section VI.

IV. AGRS FRAMEWORK
As described above, the aim of routing and scheduling is to generate a valid schedule whose tuples satisfy all the time-sensitive constraints. In this section, we illustrate the framework of the adaptive group routing and scheduling (AGRS) approach to generate a valid schedule. As shown in Fig. 1, the framework contains the pre-processing, schedule synthesis and post-processing phases, which are described in detail below.

1) PRE-PROCESSING PHASE
As computational complexity is closely related to the complexity of networks, the pre-processing phase simplifies the network topologies to reduce computational complexity for future scheduling process. This phase contains two steps including the pruning step and link grouping step. While the pruning step removes the unnecessary links that flows can never be transmitted through, the link grouping step merges the qualified complete subgraphs of the network to further simplify the topology.

2) SCHEDULE SYNTHESIS PHASE
The schedule synthesis phase jointly considers the routing and scheduling problems. That is, the routes of the flows in this phase are not determined as a prior, but are dynamically generated during the scheduling procedure. Therefore, the solution space is sharply enlarged. If a valid schedule exists, the approach is able to find the schedule and its corresponding routes, which may be pruned by the approaches with fixed routes. Through this strategy, the AGRS approach takes full advantages of available bandwidth, and improves schedulability. Furthermore, this phase synthesizes schedules based on the simplified topologies generated by the pre-processing phase, and thus the scheduling procedure is accelerated.

3) POST-PROCESSING PHASE
The post-processing phase handles the results of the schedule synthesis phase. This phase contains two steps including the splitting step and the schedule mapping step. If a valid schedule of the simplified topology is able to be synthesized, the schedule mapping step converts the valid schedule to an equivalent schedule of the original topology. Otherwise, the splitting step refines the simplified topology for future scheduling, during which the link failing to satisfy the constraints is split out of the group.
It should be noted that the schedule synthesis phase itself is a fully featured scheduling approach, taking the original topology as input. However, this phase considers all the possible combinations of routing and scheduling variables, and thus enlarges the problem space dramatically. In the meantime, the number of constraints increases in a quadratic order with respect to the fixed routing methods, which makes it unsuitable for large-scale networks. Furthermore, the large problem space and huge number of constraints sharply increase the execution time of schedule synthesis procedure, resulting in a failure to provide a fast response for the transfer requests of dynamic applications. Therefore, the pre-processing and post-processing phases are introduced to prune and merge the network links without affecting the solution space. On the one hand, the reduced number of links decreases the constraint number of routing and scheduling variables enormously, and thus accelerates the schedule synthesis for the dynamic applications. On the other hand, the maintained solution space preserves the schedulability. Therefore, the AGRS approach can achieve better schedulability and lower execution time compared with existing approaches.

V. PRE-PROCESSING PHASE
In this section, the two steps, pruning and link grouping, in the pre-processing phase are illustrated, which simplify the network topologies for future scheduling process.

A. PRUNING
While the routing and scheduling variables of a flow are allocated for each link in the system model, some of them may be unnecessary in the schedule synthesis procedure. For example, there is always one routing variable lo [ In the pruning step of the pre-processing phase, all the links that can never be part of the routes are pruned, as well as their corresponding routing and scheduling variables. The links connected to the three types of nodes shown below should be removed:

1) SOURCE
The incoming links to the source s a of a flow f a should be pruned. Moreover, the predecessor link of the pruned one should also be removed iteratively, if the link has only two predecessors.

2) DESTINATION
The outgoing links from all the destinations D a of a flow f a should be pruned. Moreover, the successor link of the pruned one should also be removed iteratively, if the link has only two successors.

3) OTHER END DEVICES
The links that connect to the other end devices should be pruned. Moreover, the adjacent link should also be removed iteratively, if the link has only two predecessors or two successors.
After pruning the unnecessary links described above, the system model can be simplified by about 20%, which has a quadratic impact on the constraint number. Furthermore, the following phases are based on the simplified topology instead of the original one.

B. LINK GROUPING
Although the unnecessary links are removed in the pruning step, the topology is still complicated for jointly considering the routing and scheduling processes. In order to further reduce the amount of all possible combinations of routing and scheduling, we adaptively group the qualified complete subgraphs and merge the relative links in the link grouping step. A bandwidth threshold B T is introduced to avoid overmerging, and only links with bandwidth utilization below the threshold are merged. The pseudo-code of the procedure is given as follows. Compute the subset R m ⊆ R with the most edges 6: while R m = ∅ do 7: Choose r ∈ R m with the least average bandwidth 8: if r ∩ X = ∅ then 9: Merge the links in r into a group 10: Add the links of r into X 11: end if 12: end while 13: end while 14: Merge the links between the link groups 15: return G The link grouping procedure merges the qualified complete subgraphs within the bandwidth threshold B T into link groups, where the link set X stores the already merged links. The links that fail to satisfy the bandwidth requirement are first removed (line 2), and the famous Bron-Kerbosch algorithm [31] is used to compute the complete subgraph set R (line 3). It should be noted that the complete subgraph set of G can be computed offline and filtered out the undesirable subgraphs online to get R in order to reduce the calculation time. Then, the complete subgraphs with the most edges and least average bandwidth containing no merged links are iteratively grouped into link groups (line 6-13). Finally, the links between the link groups are merged (line 14), and a further simplified network G is generated, which is used as input for the schedule synthesis phase. As the maximum number of complete subgraphs is (|V | − d) · 3

VI. SCHEDULING SYNTHESIS PHASE
In this section, the routing and scheduling (RS) procedure in the schedule synthesis phase are illustrated to generate a valid schedule, whose tuples satisfy all the time-sensitive constraints. The procedure is based on the simplified topology generated by the pre-processing phase, and jointly considers routing and scheduling processes. The constraints and optimization objective of multicast time-sensitive networks are first discussed, and the RS procedure is then presented to synthesize a valid schedule.

A. TIME-SENSITIVE CONSTRAINTS
The time-sensitive constraints consist of the routing constraints, path-dependent constraints, contention-free constraints, resource constraints and latency constraints. The five constraints for flow f a are defined as follows:

1) ROUTING CONSTRAINTS
The routing constraints describe the transmission continuity for a flow over the network. Flow f a should be transmitted from the source s a to all the destinations D a through the relay nodes. The routing constraints are given as follows: Equation (3) describes that flow f a is generated from the source s a . To ensure s a is the source node, there is at least one outgoing link of s a is occupied by flow f a , and no incoming link is occupied.
Equation (4) describes the forwarding behavior of all the possible relay nodes of flow f a . As only relay nodes are involved in this equation, these constraints are not applied to the source and destinations of flow f a . Since a multicast model is adopted, a relay node requires that if flow f a is transmitted through one incoming link of the node, there is at least one outgoing link that is occupied by the flow, and vice versa. It should be noted that no incoming and outgoing link mean that flow f a is not transmitted through the node. As shown in Equation 4, the first inequation ensures that there is at most one incoming link of a relay node, maintaining a tree structure. The last two inequations formulate the contrapositive constraints of the forwarding behavior of a relay node, where no outgoing link is occupied means that flow f a is not transmitted through any of the incoming links of the node, and vice versa. As a relay node may have multiple outgoing links, M is used to constrain the routing variables. M is a sufficiently large integer, which is commonly used to formulate the conditional constraints in integer linear programming (ILP).
∀d a ∈ D a : Equation (5) describes that flow f a finally arrives at the Destinations D a . To ensure d a ∈ D a is a destination node, VOLUME 8, 2020 there is exactly one incoming link of d a is occupied by flow f a , and no outgoing link is occupied.
Equation (6) describes the routing constraints for scheduling variables. As shown in this equation, the scheduling variable Equation (7)

2) PATH-DEPENDENT CONSTRAINTS
The path-dependent constraints describe the transmission timing constraints for a flow. As a flow can only be forwarded to the next switch after it has arrived at the current switch, the path-dependent constraints for the flow f a are given as follows: where [v i , r a ], [r a , v j ] are two adjacent links in the network, t a is the transmission delay of flow f a over the link, and t p is the maximum process delay in each switch, which is closely related to hardware implementation. In addition, the third component of the right part of the inequality indicates that the constraints only work when flow f a is transmitted through the link [v i , r a ] and [r a , v j ].

3) CONTENTION-FREE CONSTRAINTS
The contention-free constraints describe the exclusive link constraints for all the links. As a link can only transmit one data flow at the same time, the resource constraints are given below: where gcd(p a , p b ) function calculates the greatest common divisor (GCD) of flow period p a and p b . The constraints for each flow pair imply that the transmission of the two flows does not conflict in the greatest common divisor of their periods. To simplify presentation and maintain constraints linear, two auxiliary variables q, r are introduced, where q and r are the quotient and remainder of ( (p a , p b ). It should be noted that r is always a non-negative integer and q may be To ensure that the transmission of the two flows does not conflict, the minimal transmission difference r must be larger than the transmission time of each flow. In addition, the introduction of M ensures that the constraints only work when the link [v i , v j ] is occupied by both of the flows.

4) RESOURCE CONSTRAINTS
The resource constraints describe the storage usage constraints for switches. The number of available queues of each port for real-time flows in switch v i is denoted by N q (v i ), and the queue occupied by flow f a in switch v i is denoted by Q f a ,v i . As the number of queues is limited by N q (v i ), the first resource constraint is defined as follows: Furthermore, in order to avoid the impact on delay and jitter caused by undetermined order of arriving flows in the queue, the flows are isolated in the queue. That is, the storage times of flows in the same queue do not overlap. To guarantee this feature, the second resource constraint of switch v m is defined as follows: where f a transmits from The left side of the inequality is the minimal time that f b arrives before f a in the queue over the macro period, and the right side is the storage time of f b in the queue. If f a and f b are allocated to the same queue, f b is supposed to leave the queue before f a arrives.

5) LATENCY CONSTRAINTS
The latency constraints describe the application constraints that each flow should arrive at all its destinations within the deadline. The latency constraints are formalized as follows: where ddl a is the deadline, and t a is the transmission delay of flow f a over the link. As constrained by equation (6), there is only one nonzero value in ,f a , which is the actual sending time point of flow f a to the destination d a . As defined in equation (7), φ s a is the sending time point of the source node. In order to provide latency guarantee, the difference in the sending time points of the source and destinations should be within the deadline.

B. OPTIMIZATION OBJECTIVE
Finally, the optimization objective of the RS procedure is defined as follows: The primary objective is B max , which is the maximum bandwidth utilization over all the links after scheduling flow f a . This objective balances traffic load in the network, and thus increases the number of links that meet threshold B T for link grouping. The secondary objective is the number of occupied links multiplied by a weighted factor, which normalizes the number and ensures the secondary objective is always less than 1. Since a multicast model is adopted, the routing constraints (3)(4)(5) only ensure that there are routes from the source to all the destinations without the consideration of loops. The secondary objective eliminates potential loops, ensuring that routes are a multicast tree structure. It should be noted that scheduling variables are not introduced in the optimization objective, as a real-time flow is only required to arrive at all its destinations within the deadline (not the sooner, the better). Moreover, the additional constraints of B max are shown below: (14) where B [v i ,v j ] is the total bandwidth utilization of the link [v i , v j ] before scheduling the flow, and B f a is the bandwidth utilization of flow f a . These constraints ensure that B max is the maximum bandwidth utilization over the links.

C. RS PROCEDURE
As the time-sensitive constraints and the optimization objective are described above, the RS procedure is devised to generate a valid schedule in this subsection. The RS procedure is based on the simplified topology generated by the pre-processing phase. For each link group or merged link, the RS procedure regards it as one single link. Specifically, given a grouped Network G , the existing flow set f e and the new flow set f n from dynamic applications, the RS procedure generates a valid schedule S v for the new flow set f n without disturbing the schedule of existing flow set f e . The pseudo-code of the procedure is shown in Procedure 2.
As the schedule synthesis problems are known to be NP-complete [12], the ILP solver is used in the RS procedure. The procedure iteratively selects a flow f a from the new flow set F n (line 2), and generates the constraint set C for this flow (line 3). Then, the ILP solver takes the constraint set C as the input, and tries to synthesize a schedule S f a for the flow f a (line 4). If a valid schedule is synthesized, the schedule S f a and the flow f a are added into the schedule S v and the existing flow set f e respectively (line 6). Otherwise, the procedure fails and returns an empty schedule (line 8). As the schedule is generated based on the simplified topology taking advantage of the link groups, the schedule synthesis procedure is sharply accelerated.

VII. POST-PROCESSING PHASE
In this section, the two steps, schedule mapping and splitting, in the post-processing phase are illustrated to handle the results of schedule synthesis.

A. SCHEDULE MAPPING
If a valid schedule S v is able to be synthesized for the grouped network G , S v needs to be transferred to a valid schedule S v for the original network G. To transfer from S v to S v , Lemma 1 is introduced below:

Lemma 1: If a schedule S v is a valid schedule for G , a valid schedule S v can be constructed for G.
Proof: Without loss of generality, a flow f a is chosen in the schedule S v . The routes of f a in G from the source s a are first constructed by the routes in schedule S v . If a node v i is connected to a link group r in S v , the link [v i , v j ] in G that connects v i and node v j ∈ r is added into the route. If a link group r 1 is connected to a link group r 2 in S v , the link [v i , v j ] in G is added into the route where node v i ∈ r i is connected to node v j ∈ r 2 . If a link group r is connected to v j in S v , the link [v i , v j ] in G that connects node v i ∈ r and v j is added into the route. Then, the routes are complemented in the link groups to form a complete multicast tree. Finally, the scheduling results in S v are assigned to the corresponding links in S v . Since there is no transmission conflict in S v for G , there is no conflict in S v for G. Moreover, the latency constraints are satisfied for S v according to S v . Therefore, S v is a valid schedule for G.
From the above, the schedule S v can be simply transferred to a valid schedule S v for the original network G using the method in the proof of the Lemma 1.

B. SPLITTING
Although the grouped network G accelerates the schedule synthesis procedure, it is an abstract of the original network G. That is, there is not necessarily a valid schedule in the grouped network G , even though a valid schedule exists in the original G. The reason is that the constraints are tightened where the scheduling variables are forced to not overlap in a link group or merged link. For example, if the total bandwidth utilization of a link group or merged link exceeds 1, the RS VOLUME 8, 2020 procedure is certain to fail to generate a valid schedule for the grouped network G .
When the RS procedure fails, the grouped network G needs to be refined to maintain schedulability. The splitting step analyzes the scheduling results, and locates the link group or merged link whose time-sensitive constraints are violated. Then, the link with the highest bandwidth utilization is split out of the link group or merged link, and is marked as not being considered for future link grouping or merging. Finally, the schedule synthesis step tries to generate a valid schedule based on the refined network G .

VIII. EVALUATION
In this section, the performance of the AGRS approach is presented. The experiment setup is first discussed, and schedulability, bandwidth bottleneck, impact of B T and execution time are then evaluated. The experimental results are compared to the SMT method [10] with fixed routing (FR), which is generated by the traditional Steiner Tree [32]. Moreover, the results of the RS procedure without the pre-processing and post-processing phase are also compared.

A. EXPERIMENT SETUP
The experiments are performed on a hardware testbed as shown in Fig. 2. The testbed consists of TSN switches, end devices and a controller. A TSN switch is a fully featured 24-port real-time switch implemented by FPGA with a Xilinx Chip xc7vx690t of the Virtex-7 family, which supports on-the-fly reconfiguration of schedules by the controller through a southbound interface (SBI) similar to the Open-Flow protocol [33]. The testbed works in 100 Mbps.
In addition, a schedule synthesis tool is implemented for multicast time-sensitive networks, and the underlying ILP solver employed in this tool is IBM CPLEX (version 12.7.1.0). The controller is a Windows 10 (64 bit) desktop equipped with an Intel i7 4790K quad-core 4.00GHz and 32 GB memory. The scheduling approaches are evaluated via various network topologies generated by NetworkX, which is a popular Python package for the creation of complex networks. Four different network topologies are first created using the 5 model, each consisting of 24 switches, 128 core links and 72 end devices. Then, flows are randomly generated from 64 bytes to 1518 bytes with periods of 2, 4, 8, 16 and 32 ms and a deadline of 8 ms, which simulates the industrial control data. The source and destinations of each flow are randomly selected among the end devices. Finally, 1000 scenarios are generated by combining the four topologies and data transfer requirements for each case.

B. SCHEDULABILITY
To evaluate schedulability, flows are gradually added into networks until the scheduling procedure fails. For each number of flows, schedulability is calculated by dividing the number of schedulable scenarios from the total scenarios. As shown in Fig. 3, FR can schedule about 150 flows, while RS and AGRS can schedule about 250 flows. That is, schedulability is improved by about 68% through jointly considering routing and scheduling processes. The schedulability of FR drops dramatically after flow number exceeds 200, and FR can schedule no more than 323 flows. As a contrast, the schedulability of RS and AGRS is above 90%, which is still acceptable. Moreover, RS and AGRS can schedule at most about 450 flows, which is about 39% more than FR. It should be noted that there is a little difference of the schedulability between RS and AGRS. As the scheduling procedure schedules subsequent flows without disturbing previous flows, previous scheduling results may affect subsequent scheduling results. Since AGRS adopts a link grouping strategy, the assignment of scheduling variables over a specific link may be scattered to avoid conflict within link groups or merged links, while RS does not have this restriction. However, neither of the two strategies is significantly better than the other as shown in Fig. 3.

C. BANDWIDTH BOTTLENECK
It is observed that schedulability is closely related to the maximum bandwidth utilization over the links in the network. In addition, schedulability drops sharply after the maximum bandwidth utilization exceeds 60%. Therefore, the maximum bandwidth utilization is used to indicate bandwidth bottleneck. As shown in Fig. 4, FR has the highest maximum bandwidth utilization due to an adoption of fixed routes. Since RS includes load balancing in the optimization objective, the scheduling procedure can adaptively choose routes 37862 VOLUME 8, 2020  according to the traffic load over the links. Therefore, bandwidth bottleneck is dramatically reduced compared with FR. Furthermore, although AGRS also uses a load balancing strategy, the traffic load of links within a link group or merged link is not balanced. Therefore, the bandwidth bottleneck of AGRS is about 30% higher than RS, which is the reason why the schedulability of AGRS drops before RS.

D. IMPACT OF B T
As the link grouping step only merges complete subgraphs within the bandwidth threshold B T , B T may have a large impact on the performance of the AGRS approach. In this subsection, the AGRS approach is evaluated under various values of B T as 0, 0.3, 0.6 and 1. If the value of B T is 0, grouping condition cannot be satisfied, and no complete subgraph is merged. Thus, the AGRS approach degrades to the RS approach. If the value of B T is 1, the complete subgraphs are always merged in the descending order of vertex number, and the adaptivity of the AGRS approach disappears. Fig. 5 shows the impact of B T on the execution time of scheduling procedure. The horizontal axis presents the number of existing flows in the network, and the vertical axis presents the scheduling time per flow. Since the link grouping step merges complete subgraphs as much as possible when B T = 1, the network topology is the most simplified among the four values. Therefore, the running time of B T = 1 is the least when the number of flows is less than 50. As the number of flows in the network grows, the possibility of potential scheduling conflicts within the complete subgraphs increases dramatically. When scheduling conflicts occur, the schedule synthesis phase fails to generate a valid schedule based on the current simplified topology, and the splitting step is invoked to refine the topology. In this scenario, the schedule synthesis phase needs to be re-run based on the new topology, which consumes large execution time. As shown in Fig. 5, due to multiple refinement, the running time of B T = 1 is even worse than the no-merging scenario (B T = 0) when the number of flow exceeds 80. As the AGRS approach uses adaptive merging strategy, the running time performance of B T = 0.3 and 0.6 is satisfying. Moreover, the performance of B T = 0.3 is better than B T = 0.6 when the number of flow exceeds 90. Fig. 6 shows the execution time of scheduling procedure. The horizontal axis presents the number of existing flows in the network, and the vertical axis presents the scheduling time per flow. Due to the adoption of fixed routes, FR has the least number of time-sensitive constraints, and thus has the minimum scheduling time among the three methods. As RS jointly considers routing and scheduling processes based on the original topology, all the combinations of routing and scheduling are considered, and thus the problem space is dramatically enlarged. Therefore, the number of constraints increases sharply as the number of flows grows, resulting in a dramatic increase in the running time of RS. In addition, the execution time of RS is about eight times of FR when there are 100 flows in the networks. In contrast, due to the adoption of the link grouping strategy, the complex of the input topology of AGRS is enormously simplified, which reduces the number of constraints sharply. Therefore, AGRS is able to have a similar running time performance compared to FR, and the execution time of AGRS is also in seconds when there are 100 flows in the network. Moreover, the execution time of AGRS is reduced by 74% though the link grouping strategy compared to RS as shown in Fig. 6. As industrial networks have rich connectivity due to redundancy, our link grouping strategy is general to accelerate the scheduling procedure.

IX. CONCLUSION
The time-sensitive networks are urgently desired in industrial automation to provide real-time control, and the time-triggered mechanism is a promising technology to deploy due to its determinacy. However, time-triggered networks faces the challenges from dynamic applications in the trend of IIOT. The schedule synthesis approach should take fully advantage of the bandwidth without disturbing the existing schedule, and have low execution time to provide a fast response to the data transfer requirements. In this paper, a novel adaptive group routing and scheduling (AGRS) approach is proposed for multicast time-sensitive networks. In this approach, a pre-processing phase is first devised to accelerate the schedule synthesis procedure for dynamic applications, which prunes unnecessary links and adaptively groups qualified complete subgraphs of the topology. Then, a routing and scheduling (RS) procedure that jointly considers routing and scheduling processes is developed to improve schedulability. This procedure is based on the simplified topology, and thus the execution time is sharply reduced. Furthermore, a splitting process is designed to refine the network, which maintains schedulability. Experiments show that the AGRS approach can provide better schedulability and lower execution time compared to existing approaches.