Cooperated Traffic Shaping with Traffic Estimation and Path Reallocation to Mitigate Microbursts in IoT Backhaul Network

An aggregating switch (SW) network offers cost-effective accommodation to Internet-of-Things (IoT) traffic by aggregating the traffic. In such a network, it is crucial to eliminate the discarded traffic caused by the simultaneous transmission of massive IoT devices, namely microburst. Traffic shaping is a technique of storing traffic in one SW to mitigate microbursts. Conventional traffic shaping is limited because only one SW performs shaping with a limited queue length. Thus, we propose cooperated traffic shaping using multiple SWs to accommodate more traffic. We formulated equations to derive the minimum queue length with shaping rates which gradually decrease in geometric progression. To acquire the queue length using general SWs without short-cycle monitoring, we propose a scheme for estimating the instantaneous input rate and data size of microburst traffic required for our equations. If the calculated queue length cannot be prepared in the current path, we propose reallocating the path to another one with more SWs. We experimentally demonstrated the proposed coordinated traffic shaping technique by implementing it in commercial SWs with 125 emulated IoT devices. The results showed that the difference between the experimental and numerical results was below 4.2%, and the queue length can be reduced by 40% when there are three SWs. In addition, a path with two SWs was successfully reallocated to one with three SWs.


I. INTRODUCTION
As 5G mobile services have begun to spread worldwide, the amount of data traffic has been increasing rapidly. These services take advantage of 5G features such as high speed/large capacity, massive connected devices, ultra low latency, and ultra reliability [1]. Internet-of-Things (IoT) devices are deployed for various applications such as camera monitoring and sensing. They use cellular systems as well as non-cellular wireless systems such as Long-Range Wide Area Network (LoRaWAN) [2], depending on the application requirements.
IoT devices are connected to IoT servers via IoT gateways (IoT-GWs), which provide a wireless coverage area via cellular or non-cellular connections. The gateways aggregate the IoT traffic in their coverage areas and send the aggregated data to the server via an optical network known as IoT backhaul. While each device only generates a small amount of traffic, the total amount of traffic is large because of the massive devices. In order to accommodate the traffic, a network consisting of Layer-2 or Layer-3 aggregation switches (SWs) would be suitable for IoT backhaul [3]. The network (NW) can cost-effectively accommodate such traffic by using statistical multiplexing.
Microbursts are sudden increases in traffic caused by the simultaneous transmission of massive devices. This presents a problem when accommodating IoT traffic in an aggregated NW [4]. When microbursts occur, the amount of traffic can exceed the queue length in SWs, resulting in discarded packets. In other words, an increase in delay occurs due to packet loss and retransmission. Retransmission should be avoided for IoT devices because they have limited battery life. A method of controlling the transmission timing for each service can be used to eliminate microbursts. However, because the NW accommodates the traffic of various independently-operated IoT service providers, it is not practical to adjust the timing for all of them.
In this paper, we propose cooperated shaping using SWs to accommodate a greater amount of traffic with a limited queue length. We derive the minimum queue length through theoretical investigations. Then we introduce a method of acquiring the queue length by using general SWs without short-cycle monitoring. We improved the algorithm for cooperated shaping presented in our previous paper [5] to estimate the parameters required for the instantaneous input rate and data size of microburst traffic. If the calculated queue length cannot be prepared in the current path, we propose reallocating the path to one with more SWs in order to reduce the required queue length.
The rest of this paper is organized as follows. In Section II, we discuss related work on techniques for dealing with microbursts. Then in Section III, we describe the details of the proposed cooperated traffic shaping. In Section IV, we present our experimental results. Finally, Section V concludes this paper.

II. RELATED WORK
Several solutions have been proposed to mitigate microbursts, particularly in data center networks [6]- [11]. These solutions include transmission control protocol (TCP) pacing mechanisms to avoid TCP incast in SWs, predictive packet drop mechanisms to slow down the transmission rate of TCP flows, and explicit congestion notification (ECN) marking schemes from switches to servers.
End-host batching schemes can cause microburst traffic in a network. ECNs are widely used in data centers to mitigate microburst traffic by maintaining a low queue occupancy. However, the current ECN marking scheme based on instantaneous queue length may lead to problems such as buffer underflow [6]. Specifically, the current ECN marking scheme used in data centers is prone to spurious congestion signals, which may cause overreaction of the sender and oscillation of the switch queue length.
Adaptive pacing (AP) is a protocol that was proposed to dynamically adjust burstiness on the basis of flow concurrency to reduce the probability of TCP incast [10]. AP can be integrated transparently into different TCP protocols, such as NewReno and data center TCP (DCTCP). The design of AP is based on uniformly distributed traffic from the characteristics of the data center NW. However, IoT devices may access the network in a highly synchronized manner, for example, after a power outage [12]. Thus, the performance of AP may be affected in an aggregated NW.
A tiny packet program (TPP) was proposed in which the end host embeds a small program in a packet to actively query and manipulate the internal state of the network [13]. The TPP interface enables end hosts to have unprecedented visibility into the behavior of the network. However, coordinating with the network controller to control communication timing would make IoT devices more complex.
Traffic shaping is a method in which traffic is stored in SW buffer memories to reduce traffic flow to the IoT server. Compared with other methods, shaping can be applied to any protocol and is suitable for a network that accommodates various service providers. Conventional traffic shaping is limited because only one SW performs shaping [14]. We previously proposed a cooperated traffic shaping scheme for reducing the required queue length of each SW [5]. However, the input data size of microbursts was assumed to be known. A method for estimating the input data size is needed for cooperated shaping using general SWs without short-cycle monitoring. In addition, our previous study only considered linear topology. By considering the control of the path in a ring or mesh NW, it may be possible to further eliminate the discarded frames caused by microbursts. Figure 1 shows the architecture of an aggregated NW using cooperated traffic shaping techniques. Our proposed aggregated NW consists of SWs and a network controller (NWC) which configures the SW settings such as queue Shaper SW length and shaping rate. Each SW aggregates the uplink traffic from the IoT-GW and forwards it to the IoT server by statistical multiplexing. The traffic shaper and queue range of each SW is set on the basis of the buffer memory size. The queue here indicates the control for each VLAN-ID. The upper limit of the buffer memory size is assumed to be the same for all SWs. The IoT server sends a request for data transmission to each device via the IoT-GW, and the application is assumed to send data in response. In this case, the IoT devices transmit data simultaneously, and a microburst occurs with an input data size ( ) and an input rate ( ).

A. Numerical analysis
Here, the queue length of all SWs is calculated to be the minimum so that the buffer memory required for SWs is equalized for efficiency. , the queue length of SW # , should be set as follows (1) For = 1, 0 denotes the input rate of microburst traffic to the first SW.
at the last SW #n is equal to . The right-hand side of (1) indicates the accumulation size of the queue that accumulates until the end of the burst. In other words, it is the multiplication of the accumulation rate in the queue, −1 − , and the burst length input to SW # , / −1 . To shape the IoT traffic, becomes smaller as # becomes larger. In (1), the required increases monotonically as  decreases monotonically as increases, the longest queue length is minimized under when the queue length is constant (i.e., = +1 ). Calculating the shaping rate from this condition, we get = � −1 +1 , which is a geometric progression. Thus, optimizing the shaping rate and the queue length yields where is the minimized length of the longest queue. Figure 3 shows the maximum calculated by cooperated traffic shaping without frame dropping when a microburst occurs to a three-SW series ( : 10 Gbps). We assumed that the buffer memory size of each SW is up to 64 MBytes and that the IoT-GW and all SWs are connected via 10-Gigabit Ethernet (10 GbE). In the cooperated traffic shaping, the NWC orders SW #1, #2, and #3 to store the overload data. In conventional traffic shaping, one SW stores the overload data. Thus, the proposed aggregated NW with the cooperated traffic shaping can forward larger microburst traffic (113 MBytes) than the conventional method (64 MBytes).

B. Implementation of cooperated traffic shaping
The required queue length depends on the measurement accuracy of the observable and , as shown in (3). Figure 4 (a) shows various estimated queue length errors under the condition that is 3 Gbps, is 150 MBytes, and that there are three SWs. In this case, is calculated to be about 46 MBytes. Figure 4 (b) shows the calculated queue length when is misestimated and is correctly estimated as 3 Gbps. The result coincides with the dotted horizontal line in Fig. 4 (a). Figure 4 (c) shows the calculated required queue length when is misestimated and is correctly estimated as 150 Mbytes. The result coincides with the dotted vertical line in Fig. 4 (a). In the case of underestimation, the queue length allocation is reduced, and the data is discarded. In the case of overestimation, the allocation of the queue length becomes large, and the queue becomes excessive. No data is discarded, but the number of queues that can be allocated decreases when considering the entire buffer. Note that the estimated input rate increases and decreases logarithmically, while the estimated amount of data increases and decreases linearly.
To know the values of and , the traffic at all of the ports in each SW should be continuously monitored because it is impossible to predict where microburst traffic will occur. Moreover, the monitoring cycle should be relatively short in order to calculate the instantaneous value accurately. However, short-cycle monitoring increases the SW cost. Our previous study [5] assumed that was obtained accurately. A new estimation method is needed to estimate and without short-cycle monitoring so that it can be performed even by general SWs.
We propose estimating and through two-time estimation. Because the two unknown variables are the input rate and data size, the solution can be obtained with simultaneous equations from two different parameters. In this method, each SW measures the input rate and data size periodically within a specific time interval. A microburst of traffic is assumed to occur under the same conditions of and . First, the NWC configures the shaping rate and queue length to the target rate and the initial queue length , respectively, for all SWs. The initial queue length is determined from the minimum queue length in the SW specifications. In regular operation, the traffic counter in each SW measures discarded data size 1 at a specific time interval. Once a SW observes dropped data (i.e., 1 > 0), the discarded data size 1 and queue length are expressed by The equation includes two unknown values, and , so it cannot be solved. The NWC tentatively defines the SW as SW #0 and corrects the estimated input rate and input data size measured by the SW #1 traffic monitor. We set the estimated input rate as the average traffic rate in a monitoring interval. Then, using these estimates, the shaping rate of SW #k is set as follows, Because the burst length is smaller than the monitoring interval, the estimated input rate becomes smaller than the instantaneous rate of the microburst that is the actual required value. Thus, the calculated queue length is shorter than the length actually required, and the second data discursion 2 will occur at SW #1. From this result, can be obtained. From (4) and (7), we obtain the two unknown values and by the following: Then the NWC calculates the shaping rate and queue length of each SW by using (8) and (9) and sends the values to each SW.

C. Path reallocation
In the IoT backhaul, the transmission path candidates of the flow are based on the topology of the NW. As shown by (3), as the number of SWs increases, the required queue length per SW decreases. The number of SWs is determined by the path that is selected. The NWC generally sets the shortest path to each IoT flow to use the resources of the queues in the NW efficiently. If the required queue length calculated by the shortest path exceeds the upper limit of the SW, we propose changing the path to reduce the queue length required for each SW. This will increase the amount of traffic that can be accommodated.

IV. EXPERIM ENTAL SETUP AND RESULTS
We experimentally demonstrated the coordinated traffic shaping technique by implementing it in commercial SWs [15]. The experimental NW consisted of IoT devices, an IoT server, three SWs, a virtual SW, and a NWC, as shown in Figure 5.
The NWC consists of a flow controller and a SW controller. We emulated massive IoT devices to generate a microburst. This was performed in StarBED, a large-scale emulation platform [16]. The 125 IoT devices were implemented on Ubuntu with Kernel-based Virtual Machine (KVM). Each device periodically sent 5 kB of data (a JPEG file) with 1,500byte user datagram protocol (UDP) packets and 12-byte interframe gaps, assuming periodic data transmission such as sensor data. The IoT server sent requests at a specific interval to control the transmission timing of the IoT devices. The interval of the microburst traffic was set to 10 seconds. Figure  6 shows the amount of traffic in a microburst with a 1millisecond resolution. The SWs periodically monitored 1 within 10 seconds and reported to the SW controller of the NWC. The SW controller was implemented as a software controller on an Ubuntu server. The SW controller has the path information for each flow and SW specifications such as minimum queue length. The SW controller calculated the queue length and shapting rates as shown in the previous chapter and configured them to all of the SWs. and the initial shaping rate of each SW were both set to 100 Mbps.
The queue length of each SW was initially set to 10 kB, and the maximum input data size that could be transferred without frame loss was measured by varying the maximum queue length of the SW. For comparison, a SW with a fixed queue length was prepared as a conventional method by keeping the shaping rate at 100 Mbps.
The experimental NW was constructed by a virtual SW using Lagopus [17], and Ryu [18] was used as a flow controller. The SW controller calculated the queue length required for eliminating discarded frames; if it is greater than     Figure 7 shows the experimental results of cooperated traffic shaping. When the number of SWs was set to one, the shaping rates of SW #1 and #2 were set to for the conventional traffic shaping method, i.e., the baseline. As the number of SWs increases, cooperative shaping improves the performance of required queue length because the required queue length decreases with the number of SWs. The required queue length was 348 kBytes in the proposed method using three SWs while the required queue length was 585 kBytes in the conventional method. That is, our proposed cooperated traffic shaping method reduced the queue length required in the conventional traffic shaping (using one SW) by 40%. As a result, the data size of the microburst which can be accommodated is 1.68 times larger than that of the conventional method with the same queue length. Figure 7 also shows the numerical analysis results when and are known. The results were computed using (2) and (3). The difference between the experimental and numerical results was below 4.2% for up to 3 SWs. Because the difference is negligible, we can conclude that the estimation scheme in (8) and (9) worked accurately.
Next, we demonstrated the path reallocation by adjusting the rest of the queue length of SW #2. The vSW outputs the data from IoT devices to port #1 first. The queue length calculated by the NW controller was larger than the rest of the queue length at SW #2 so the path was reallocated by the flow controller to an additional path through SW #3. Figure 8 shows an example of offloaded traffic, in which 10% of the traffic is offloaded to output port #3. The NWC uses the traffic offload function of Lagopus and Ryum and controls the traffic for each path by setting the weight. When the weight is set to 100% for output port #3, the path can be reallocated to the longer one. The number of SWs in the path increased from two to three, so the required queue length was reduced and the queue length was allocated to eliminate discarded frames.
However, the path reallocation algorithm presents some limitations. For example, IoT traffic fluctuates if the number of IoT devices changes; we assumed the number of IoT devices was constant in this paper. The optimal path should be changed in accordance with such time dependent factors. An adaptive reallocation algorithm will be required to further improve the proposed method.

V. CONCLUSION
Internet-of-Things (IoT) devices will be deployed for various applications such as camera monitoring and sensing in 5G mobile networks and beyond. A network consisting of Layer-2 or Layer-3 aggregation switches (SWs) can accommodate the traffic for IoT backhaul; however, the problem of microbursts arises when accommodating IoT traffic in an aggregated NW. Thus, we proposed cooperated traffic shaping using SWs to accommodate more traffic with a limited queue length. We formulated equations to derive the minimum queue length with shaping rates which gradually decrease in geometric progression. To acquire the queue length using general SWs without short-cycle monitoring, we proposed a scheme for estimating the instantaneous input rate and data size of microburst traffic required for the equations. If the calculated queue length cannot be prepared in the current path, then the path is reallocated to one with more SWs. We experimentally demonstrated the proposed coordinated traffic shaping technique by implementing it in commercial SWs with 125 emulated IoT devices. We found that the difference between the experimental and numerical results was below 4.2%, and the queue length could be reduced by 40% when there were three SWs. In addition, the path was successfully reallocated from a path with two SWs to one with three SWs. In the future, an adaptive reallocation algorithm will be required to address the dynamic microburst traffic in an actual network.