Power Efficiency and Delay Tradeoff of 100G Energy Efficient Ethernet Protocol

This paper investigates the dual-mode power-saving strategy designed for the 100G Energy Efficient Ethernet. The process of this strategy is a sequence of cycles, where each cycle is an interval elapsed between two consecutive instants when the buffer of the Ethernet interface becomes empty. In each cycle, the interface first enters the fast-wake mode to perform the conditional sleep-mode selection of the strategy. If the number of arrivals during the fast-wake mode reaches a threshold, the interface directly wakes up; otherwise, it proceeds to the deep-sleep mode. The sequence of cycles switches between two types: deep-sleep cycles with deep-sleep mode and light-sleep cycles without deep-sleep mode. We analyze the dual-mode strategy based on the condition of the number of arrivals during the fast-wake mode. We first derive the weights of conditions according to the feature of the conditional sleep-mode operation, and then calculate the unconditioned performance measure of the dual-mode strategy based on the weighted average of that of these two kinds of cycles. Finally, we obtain the close-form expressions of the power efficiency and the mean delay, based on which we provide a set of parameter selection rules. We show that the dual-mode strategy with these rules can select suitable sleep modes according to the instantaneous traffic rate, and thus perform well under bursty input traffic.


I. INTRODUCTION
In recent years, the demands for high-speed Ethernet have been driven by ever-increasing Internet use and emerging bandwidth-hungry applications, such as 4K/8K video [1] and high-performance computing [2]. 100G Ethernet is gradually displacing the 10G Ethernet and will become mainstream very soon [3]. However, with high data speed, energy consumption per Ethernet device exponentially increases [4]. Meanwhile, the link utilization of typical Ethernet interfaces is only about 5%-30% [4]- [6]. Therefore, a lot of energy is wasted if the Ethernet interfaces run at full power all the time.
In this context, IEEE 802.3az [7], the first Energy Efficient Ethernet (EEE) standard, was published in 2010. This standard defines a Low Power Idle (LPI) mode for 100M/1G/10G Ethernet interfaces. When the buffer is emptied, the Ethernet interface turns off most of its components, The associate editor coordinating the review of this manuscript and approving it for publication was Ahmed Mohamed Ahmed Almradi . and enters the LPI mode through a Sleep operation. In the LPI mode, the interface consumes only ∼10% of the full power. The interface uses two thresholds, counter N and timer τ , to control the duration of the LPI mode. Once the number of arrived frames accumulated in the buffer reaches N or the waiting time of the first frame exceeds τ , the interface terminates the LPI mode via a Wakeup operation and starts to transmit frames. This sleep strategy is referred to as single-mode strategy in this paper.
However, the single-mode strategy cannot be directly applied to 100G Ethernet interfaces because the duration of the Wakeup operation for the LPI mode is excessively long [8], [9]. It would take a 100G interface 5.5µs to wake up from the LPI mode [10], [11]. In addition, the data rate of a 100G link is ten times faster than that of a 10G link. As a result, up to 45 frames of 1500 bytes can be accumulated in the transmission queue of a 100G interface during the Wakeup operation, which may increase the queue length in the buffer remarkably. As [11] points out, this may require a large buffer to accommodate incoming frames.
Therefore, the IEEE 802.3bj standard published in 2014 [10] introduces another power-saving mode, called fast-wake (FW) mode, to 100G Ethernet, and refers to the LPI mode as the deep-sleep (DS) mode. In the FW mode, the interface powers off only a few components, and thus consumes up to 70% of the full power [8]. Even so, it only requires a Wakeup time as short as 0.34µs, which can avoid the sharp increase of the queue length. To distinguish these two kinds of Wakeup, we refer to the Wakeup of the FW mode as FW Wakeup, and that of the DS mode as DS Wakeup. The simulation results of [11]- [14] showed that the combination of the DS mode and the FW mode can help the 100G EEE interface to save more power than the DS mode only.
To use the complementary advantages of the FW mode and the DS mode, a dual-mode strategy was proposed for 100G EEE [11], [15] recently. The dual-mode strategy introduces two additional FW thresholds, FW counter N f and FW timer T FW . The 100G interface with the dual-mode strategy first enters the FW mode when the buffer becomes empty. The FW mode at most lasts for a duration of T FW . If the traffic rate is high such that the number of arrivals reaches N f before the FW mode ends, the interface directly wakes up. In this way, the interface can avoid a sharp increase of the queue length. On the contrary, if the traffic rate is low such that fewer than N f frames arrive before the FW mode terminates, the interface enters the DS mode to reduce the energy consumption. Similar to that of the LPI mode in the single-mode strategy, the duration of the DS mode is controlled by thresholds N and τ , which are called the DS counter threshold and DS timer threshold, respectively.
The dual-mode strategy is essentially a kind of conditional sleeping strategy. Given the parameters T FW and N f , the usage of the DS mode is conditional on the number of arrivals during the FW mode, i.e., controlled by the traffic rate. When the traffic rate is low, the number of frames that arrive before the FW mode ends will be smaller than N f with high probability, and the system will tend to enter the DS mode; otherwise, the system will almost not enter the DS mode. Therefore, the FW mode with thresholds T FW and N f is actually a kind of conditional sleep-mode operation, which can help the system to adaptively select the sleep mode according to the traffic rate. As a comparison, the single-mode strategy is a special case of the dual-mode strategy, since it lets the interface enter the DS mode with probability 1 once the buffer becomes empty regardless the traffic condition.
Clearly, FW thresholds T FW and N f play a key role in carrying out the conditional sleep-mode operation. If they are not properly selected, the dual-mode strategy may lead to a hostile queue length or a poor power efficiency, which is defined as the percentage of energy that the strategy can save. For instance, if N f is small while T FW is large, the system will almost not enter the DS mode even if the traffic rate is low, thereby resulting in low power efficiency. Thus, it is necessary to find a proper selection rule for these parameters, based on the understanding of system performance.
However, the feature of the dual-mode strategy imposes a big challenge on the performance analysis. Similar to the single-mode strategy considered in [16], the dual-model strategy can be modelled as a queueing system with vacation times governed by the frame arrival process and two sets of thresholds, the FW thresholds T FW and N f , and the DS thresholds τ and N . In addition, the dual-mode strategy has a notable feature that it can use the FW thresholds to determine whether to enter the DS mode at the end of the FW mode according to the instantaneous traffic rate. This conditional sleep-mode operation introduces a complicated dependency between the number of arrivals during the vacation time and that during the FW mode. As a result, it is hard to derive the distribution of the number of arrivals during the vacation time, which however is the key to analyze the queueing system with vacation times governed by the arrival process [16]. Thus, previous analysis techniques for the single-mode strategy, such as the model in [16], cannot be directly applied to the dual-mode strategy.

A. RELATED WORKS
Most of the previous works focused on the analysis of singlemode strategies [16]- [23]. The models in Ref. [17]- [23] failed to obtain a unified closed-form formula for the mean delay in the general case. References [17], [18] investigated the simplest cases, where either counter threshold N [17] or timer threshold τ was used [18] to control the duration of the LPI mode. References [19]- [22] studied the general strategy where these two thresholds were employed at the same time. To simplify the analysis, these works analyzed the system respectively in the low-traffic rate region and the high-traffic rate region, and thus can only obtain the piecewise expressions of the mean delay and the power efficiency. To derive the unified expression over the entire region of the traffic rate, Ref. [23] developed an analytical model, starting with the derivation of vacation time distribution. However, the model in [23] is too complex to characterize the behavior of the single-mode strategy. Reference [16] pointed out the failure was incurred by the fact that the single-mode strategy is a kind of queueing system with vacation times controlled by the frame arrival process. Starting with the distribution of the number of arrivals during the vacation time, Ref. [16] derived a generalized P-K formula for the mean delay. Even so, we show in Appendix A that, if the model of [16] is directly applied to the dual-mode strategy, it will become complicated and intractable. This is because the model in [16] was not designed to deal with the complicated dependency between the number of arrivals during the vacation time and that during the FW mode.
Currently, a few works [15], [24], [25] applied the analytical models developed for single-mode strategies to study the dual-mode strategy. Based on the model in [26] and [27], Ref. [24] and [25] analyzed the dual-mode strategy with N f = N = 1, where the interface wakes up immediately VOLUME 8, 2020 as soon as a frame arrives. However, such a strategy has quite low power efficiency and is not suitable for practical applications [11], [12], since the vacation time is short and a large amount of energy is wasted by Wakeup operations. Reference [15] extended the model presented in [21], [22] to study the case where N > N f > 1, which can only derive the power efficiency. Similarly, Ref. [15], [24], [25] did not notice the feature of the dual-mode strategy, and thus cannot find the closed-form mean delay for the dual-mode strategy in the general case. This indicates that simply extending the models of the single-mode strategy for the dual-mode strategy is not feasible. As a result, Ref. [24] specifically mentioned that it was quite difficult to model the dual-mode strategy.

B. OUR WORK AND CONTRIBUTIONS
In this paper, we develop a new approach to study the dual-mode strategy. Our goal is to derive the closed-form solutions of the mean delay and the power efficiency, such that we can propose proper parameter selection rules for the dual-mode strategy.
We consider the dual-mode strategy as a regenerative process, in which the cycle is defined as the interval between two consecutive instants when the buffer becomes empty. We show that under the conditional sleep-mode operation governed by thresholds T FW and N f , the cycle switches between two types: 1) DS cycle that contains the DS mode; 2) LS (light-sleep) cycle that does not. In particular, such an operation determines the probabilities of the DS cycle and the LS cycle, according to the traffic rate. When the traffic rate is low, the DS cycle occurs with high probability. When the traffic rate is high, the LS cycle appears with high probability.
Motivated by this observation, we propose an analytical model based on the concept of conditional sleep-mode, which treats the performance of the dual-mode strategy as the weighted average of that of the DS cycle and the LS cycle. We first derive the weights, which describe the function of the conditional sleep-mode operation governed by thresholds T FW and N f . We then analyze the conditional expectation of the system performance given that the DS cycle or the LS cycle occurs. Given that the system is almost in the DS cycle when the traffic rate is low, or the LS cycle when the traffic rate is high, the derivation of conditional performance measures can substantially be simplified and then be handled by the technique in [16]. We finally obtain the closed-form formulas for the mean delay and the power efficiency, which are accurate enough to analyze the dual-mode strategy.
Based on these results, we propose four threshold selection rules for the dual-mode strategy, such that it can achieve a high power efficiency under a given delay requirement. We further show how these rules can be applied in practice. Our simulation results demonstrate that our rules are conservative for the dual-mode strategy under the bursty input traffic. This implies that our rules can provide worst-case performance guarantee though the burstiness of Ethernet traffic could change over time.
In summary, our contributions are listed as follows: • We successfully model the conditional sleep-mode operation of the dual-mode strategy.
• We derive the close-form formula for the mean delay, mean queue length and power efficiency, which are quite accurate for the dual-mode strategy.
• We propose a set of threshold selection rules for the dual-mode strategy, which can help the interface to perform well under bursty input traffic. We organize the rest of this paper as follows. In Section II, we introduce the process of the dual-mode strategy, and show that the major function of thresholds T FW and N f is to perform the conditional sleep-mode operation. In Section III, we propose an analytical model to derive the mean delay, mean queue length and power efficiency. In section IV, we explore the impact of thresholds T FW , N f , τ , and N on performance tradeoffs, based on which we provide four threshold selection rules. We further evaluate the performance of the dual-mode strategy with the proposed rules under bursty input traffic. Section V concludes this paper.
Besides, for ease of reading, the main notations used in this paper are listed as follows:

II. PRELIMINARIES
A. DUAL-MODE POWER-SAVING STRATEGY Fig. 1(a) illustrates the process of the dual-mode strategy. Each time the buffer of the interface becomes empty, a vacation period, denoted by V , begins. The interface first enters the FW mode via a 0.9-µs transition period, named busy-to-FW (BtF) and denoted by T BtF , during which a few components are powered off. In the FW mode, the power consumption is ϕ f = 0.7ϕ h , where ϕ h is the power consumption when the interface is busy with frame transmission. The FW mode at most lasts for a duration of T FW , during which the interface counts the number of arrivals. If less than N f frames arrive before the FW mode ends, the interface proceeds to the DS mode after a 1.0-µs transition period [11], [15], called FW-to-DS (FtD) and denoted by T FtD . In the DS mode, the interface turns off most of its components and thus its power consumption, denoted by ϕ d , is only 0.1ϕ h . The length of the DS mode is regulated by two thresholds, DS counter N and DS timer τ . As soon as the queue length reaches N or the waiting time of the first frame reaches τ , the interface terminates the DS mode and wakes up via a 5.5-µs DS Wakeup. We denote the duration of DS Wakeup as T DtB . On the contrary, if N f frames arrive before the FW mode ends, the interface interrupts the FW mode immediately, and wakes up via a 0.34-µs FW Wakeup. We denote the duration of FW Wakeup as T FtB . After the Wakeup, the interface starts a busy period, denoted by B, and transmits frames until the buffer becomes empty.
Clearly, the dual-mode strategy is a regenerative process. In this process, the system renews and starts a new cycle each time the buffer becomes empty. These cycles can be divided into two types. As Fig. 1(a) shows, a cycle is called an LS cycle if the interface wakes up directly from the FW mode, and called a DS cycle if the interface rouses up from the DS mode. The FW mode helps the interface to choose the type of the cycle, according to the number of arrivals before the FW mode ends. If the number reaches N f , the interface wakes up directly from the FW mode (i.e., it selects the LS cycle); otherwise, it enters the DS mode at the end of the FW mode (i.e., it selects the DS cycle). From this viewpoint, the FW mode at the beginning of each cycle can be regarded as a sleep-mode selector, which determines the type of cycle to be adopted. With such an operation, the cycle switches back and forth between the DS cycle and the LS cycle, as Fig. 2(a) illustrates.
Benefiting from the conditional sleep-mode operation, the best advantage of the dual-mode strategy is that it can adaptively select the DS cycle and the LS cycle, according to the traffic rate. In particular, such an operation with parameters T FW and N f determines the probabilities of the DS cycle and the LS cycle, when the traffic rate is given. For example, let's consider the case where the frame arrival process is a Poisson process with rate λ. The probability that the interface enters the DS cycle, denoted by p, is the probability that less than N f frames arrive before the FW mode ends, which can be given as follows: Accordingly, the probability that the interface enters the LS cycle, denoted by q, is given by: (2) Fig. 2(b) plots p and q changing with the traffic rate λ, where N f = 4 and T FW = 1.1µs. When λ is small and the link is almost idle, the interface enters the DS cycle with probability p → 1 such that it can save relatively more power. With the increase of λ, p decreases while q increases. When λ is large, say λ > 4.0 frames/µs, and the link is busy, the interface chooses the LS cycle with probability q → 1, such that it can avoid an excessively long queue length.
Hence, the selection of T FW and N f should be done with care; otherwise, the FW mode cannot fulfill the function of the conditional sleep-mode operation, and may even have a negative impact on system performance. For example, if N f → ∞, the interface always enters the DS cycle irrespective of the number of arrivals during the FW mode. In this case, the dual-mode strategy reduces to the DS single-mode strategy in Fig. 1(b). The queue length increases rapidly when the traffic rate is high, due to the long DS Wakeup. On the other hand, if T FW → ∞, the interface never enters the DS cycle, and the dual-mode strategy reduces to the FW single-mode strategy shown in Fig. 1(c). In this case, the power efficiency is poor when the traffic rate is low.

B. ISSUES IN SYSTEM MODELING
In this paper, our goal is to develop an accurate and tractable model to analyze the dual-mode strategy, which will provide the threshold selection rules to optimize system performance.
In the modeling, we use the following assumptions: A1. The frame arrival process is a Poisson process with arrival rate λ. A2. Frames are transmitted in the first-in/first-out (FIFO) manner. A3. The frame transmission times X 1 , X 2 , · · · , are i.i.d.
random variables with the first and second moments X and X 2 , respectively.
Assumption A4 is justified since it ensures a relatively long duration of the DS mode such that the interface can obtain a high power efficiency when the input traffic rate is low.
Under these assumptions, the dual-mode strategy can be modeled as an M/G/1 queue with vacations governed by two sets of thresholds and the frame arrival process, which is like the 10G EEE studied in [16]. Starting with the derivation of the distribution of the number of arrivals during the vacation, Ref. [16] obtained the unified formulas of the power efficiency and the mean delay of 10G EEE over the entire offered load. However, different from 10G EEE, the vacation of the dual-mode strategy is a two-stage process, and the duration of each stage is controlled by a pair of thresholds, which is quite difficult for analysis. As Appendix A shows, the difficulty is mainly incurred by the fact that thresholds T FW and N f make the distribution of the number of arrivals during the whole vacation time depend on that of the number of arrivals during the FW mode. Therefore, though the method in [16] can be applied to analyze the dual-mode strategy, the derived formulas are too complex to provide clear physical interpretation.

III. ANALYSIS OF CONDITIONAL SLEEP-MODES
As mentioned before, the analysis of the dual-mode strategy is a significant challenge due to the impact of two sets of thresholds on the vacation process. In this section, we are going to circumvent such difficulty and derive closed-form formulas for the mean delay and the power efficiency, based on the characteristics of the dual-mode strategy.
Recall from Section II that the probabilities of the DS cycle and the LS cycle are controlled by the traffic arrival rate λ, once parameters N f and T FW are given. In other words, given λ, N f and T FW , it is possible to obtain the probability that a frame arrives at the DS cycle or the LS cycle, and the fraction of the time that the interface stays in the DS cycle or the LS cycle. In this case, if we can further derive the performances of the DS cycle and the LS cycle separately, we can solve the performance of the dual-mode strategy.
Motivated by this observation, we propose an analytical model based on the concept of conditional sleep-mode. Our idea is to treat the performance of the dual-mode strategy as the weighted average of that of the DS cycle and the LS cycle. We first derive the weights, which describe the function of the conditional sleep-mode operation governed by the two thresholds T FW and N f . We then analyze the system performance given that it is in the DS cycle or the LS cycle.
A brief profile of our approach is given as follows. Define an indicator ξ = 0, if a frame arrives within a DS cycle, 1, if a frame arrives within an LS cycle.
and let d i be the delay of the ith frame. The mean delay of the dual-mode strategy can be expressed as follows: where Pr{ξ = 0} and Pr{ξ = 1} are respectively the number fractions of the frames that arrive in DS cycles and LS cycles. Similarly, conditioning on ζ , we have the power efficiency (4) where Pr{ζ = 0} and Pr{ζ = 1} are respectively the time fractions of DS cycles and LS cycles. We derive the weights in (3) and (4) in Section III-A, and solve the conditional expectations of the delay performance and the power efficiency in Section III-B and III-C. We show in Section III-B and III-C that the performance of DS cycle and LS cycle can be described by that of the DS single-mode strategy and the FW single-mode strategy, respectively. Finally, we combine these results according to (3) and (4) in Section III-D, and obtain the mean delay and power efficiency of the dual-mode strategy. Fig. 3 depicts a graphical overview of our model, to facilitate the understanding.

A. CONDITIONAL SLEEP-MODE OPERATION ANALYSIS
Consider a time period [0, t], during which the traffic rate is λ(t), and the interface experiences k d (t) DS cycles and k f (t) LS cycles. Let C d (t) and n d (t) be the mean cycle time of the DS cycle and the mean number of arrivals during a DS cycle, respectively. Similarly, let C f (t) and n f (t) be the mean cycle time of the LS cycles and the mean number of arrivals during an LS cycle, respectively. Under condition ρ = λX < 1, the system will eventually reach the steady state. It follows that the probabilities of the DS cycle and the LS cycle are respectively and where p and q are given by (1) and (2). Let n d , n f , C d , and C f be the limits of n d (t), n f (t), C d (t), and C f (t) when t → ∞. We thus have the number fractions of the frames that arrive in DS cycles and LS cycles as follows: and the time fractions of DS cycles and LS cycles as follows:

B. DS CYCLE ANALYSIS
The interface enters the DS cycle if the number of arrivals is less than N f before the FW mode ends. In a DS cycle, the interface will experience the FW mode and the DS mode in turn. Fig. 1 shows that the DS cycle is quite similar to the cycle of the DS single-mode strategy, except that the number of arrivals during the interval T BtF + T FW in the DS cycles must be less than N f while the cycle of the DS single-mode strategy has no such restriction.
When λ is small, the number of arrivals during T BtF + T FW is less than N f with high probability, and thus the difference between the DS cycle and the cycle of the DS single-mode strategy is negligible. With the increase of λ, the probability that more than N f frames arrive during T BtF + T FW in the DS single-mode strategy increases, and thus the distinction between these two kinds of cycles gradually becomes remarkable.
On the other hand, as Fig. 2(b) shows, when λ is small, the DS cycles predominate in the process of the dual-mode strategy, and thus determine system performance; when λ is large, the DS cycle rarely happens, and thus has little influence on system performance.
Thus, if using the performance of the DS single-mode strategy to describe that of the DS cycle, we can obtain analytical results with reliable accuracy. In the region of small traffic rate, the similarity between the DS cycle and the cycle of the DS single-mode strategy is very high. With the increase of the input traffic rate, though the error increases, the impact of such error on the overall performance decreases since the weight of the DS cycle in the overall performance decreases. That is, the weights can adaptively amend the error. Motivated by this observation, we adopt this method to simplify the analysis of DS cycle.
As Fig. 1(b) illustrates, each time the buffer becomes empty, the interface with the DS single-mode strategy enters the DS mode through a transition of fixed duration T s = T BtF + T FW + T FtD . As soon as N frames arrive at the buffer or the waiting time of the first frame reaches τ , the interface wakes up via the DS Wakeup. This process is similar to that of the 10G EEE studied in [16], and thus we apply the method in [16] to analyze the DS single-mode strategy.
According to [16], the key is to derive the number distribution of the arrivals during a vacation period, which is VOLUME 8, 2020 defined as h d (n) = Pr{n arrivals during a vacation period of DS single-mode strategy}.
In Appendix B, we derive h d (n) based on the event tree that includes all the mutually exclusive events which may occur during the vacation period of the DS single-mode strategy. From the result,we obtain the generating function of h d (n) and then the mean number of arrivals during the vacation period According to Little's Law, we have the mean vacation time Let ρ = λX . The mean cycle time is given by As Fig. 1(b) shows, the interface first enters the FW mode and then the DS mode in each cycle. The duration of the FW mode is T FW , and the mean duration of the DS mode is V d − T s − T DtB . Thus, we have the mean power consumption from which we derive the power efficiency of the DS single-mode strategy given by (14), as shown at the bottom of this page.
The mean delay can be derived via the P-K formula in [16]: where H d (1) is the second moment of H d (z) and given by (16), as shown at the bottom of this page.
We thus obtain the mean delay of the DS cycle and the power efficiency

C. LS CYCLE ANALYSIS
The interface selects the LS cycle when more than N f arrivals come before the FW mode ends. Once the LS cycle occurs, the interface will only experience the FW mode. As Fig. 1 shows, the LS cycle is similar to the cycle of the FW single-mode strategy, except that the N f th frame must arrive within interval T BtF + T FW in the LS cycle while the cycle of the FW single-mode strategy has no such constraint. When λ is large, the difference between the LS cycle and the cycle of the FW single-mode strategy is negligible, because the N f th frame arrives within interval T BtF + T FW of the FW single-mode strategy with high probability. With the decrease of λ, the probability that the N f th frame arrive during T BtF + T FW in the LS single-mode strategy decreases, and thus the distinction between these two kinds of cycles gradually becomes pronounced.
On the other hand, as Fig. 2(b) shows, when λ is large, the LS cycles predominate in the process of the dual-mode strategy, and thus determine the system performance; when λ is small, the LS cycle rarely happens, and thus has little influence on system performance. Thus, following the argument similar to that in Section III-B, we use the performance of the FW single-mode strategy to describe that of the LS cycle.
As Fig. 1(c) illustrates, each time the buffer becomes empty, the interface with the FW single-mode strategy goes into the FW mode through a fixed transition time T BtF . As soon as N f frames arrive at the buffer, the interface wakes up via the FW Wakeup. Again, we apply the method in [16] to analyze the FW single-mode strategy.
Similarly, based on the event tree that includes all the arrival events which may occur during the vacation period of the FW single-mode strategy, we first derive in Appendix C the distribution of the number of arrivals during a vacation period h f (n) = Pr{n arrivals during a vacation period of FW single-mode strategy}.
Accordingly, we have the generating function from which we obtain the mean number of arrivals during a vacation period We then have the mean vacation time and the mean cycle time As Fig. 1(c) shows, in each cycle of the FW single-mode strategy, the mean duration of the FW mode is V f − T BtF − T FtB . Thus, the mean power consumption is given by Accordingly, we obtain the power efficiency of the FW single-mode strategy as follows: In addition, applying the P-K Formula in [16] yields the mean delay where H f (1) is the second moment of H f (z) and is given by Thus, we have the results for the performance of the LS cycle as and

D. PERFORMANCE OF DUAL-MODE STRATEGY
We are now ready to derive the mean delay and the power efficiency of the dual-mode strategy by the unconditional equations (3) and (4). Approximating the mean cycle time of the DS cycle C d by C d and that of the LS cycle C f by C f , we have Similarly, we approximate n d (n f ), the mean number of arrivals during a DS cycle (an LS cycle), by λ C d (λ C f ) according to Little's Law. Then, we have Substituting (17), (18), (25), (26), and (27) through (30) into (3) and (4), we obtain the mean delay and power efficiency of the dual-mode strategy:  As Section I mentions, the mean queue length, denoted by L, is also an important performance metric to study the dual-mode strategy. Following Little's Law, we have the mean queue length: To verify the effectiveness of our modeling methodology, we compare the performances D d and η d in (17) and (18) and the weighted performances D d Pr{ξ = 0} and η d Pr{ζ = 0} to the simulation results of the DS cycle in Fig. 4, where N f = 4, T FW = 1.1µs, N = 41, and τ = 20µs. Fig. 4(a) displays that D d and η d agree well with the simulated mean delay and the simulated power efficiency of the DS cycle when λ is small, and the difference between η d and the simulated power efficiency becomes remarkable when λ becomes large. However, D d Pr{ξ = 0} and η d Pr{ζ = 0} fit the simulated weighted performance very well for all the values of λ, as Fig. 4(b) shows. This clearly indicates that the errors of D d and η d in (17) and (18) when λ is large can be amended by the weights Pr{ξ = 0} and Pr{ζ = 0}, respectively. Similarly, Fig. 5 confirms that the errors of D f and η f in (25) and (26) when λ is small can be suppressed by the weights Pr{ξ = 1} and Pr{ζ = 1}. Therefore, as we will verify by simulations in Section IV, the analytical results of (31), (32), and (33) are quite accurate.

IV. THRESHOLD SELECTION RULES
The essence of the EEE protocols is to save power at the expense of delay performance. Similarly, in the dual-mode strategy, such a performance tradeoff can be achieved by tuning thresholds T FW , N f , N , and τ . In this section, based on the results in Section III, we study the impacts of T FW , N f , N , and τ on performance tradeoff and seek a way to select these thresholds. We conduct simulations by a discrete-event simulation model [28] coded in C. We consider a 100G EEE interface in the simulation. We assume that the frame size is exponentially distributed with an average of 1250 bytes. Considering the fact that the utilization of a typical Ethernet link ranges from 5% to 30%, we suppose that the steady-state traffic rate is 2.0 frames/µs, which corresponds to a link utilization of 20%.

A. FW MODE VS. DS MODE
The impact of T FW and N f on the tradeoff between the DS mode and the FW mode is two-fold. As we mention in Section II, for a fixed traffic rate λ, T FW and N f can make a tradeoff between the mean delay (or the queue length) and the power efficiency by controlling the probabilities of the DS cycle and the LS cycle. On the other hand, the duration of the FW mode increases with T FW and N f . Intuitively, if the time fraction of the FW mode in the DS cycle is larger, the interface can save less power via the DS cycle when λ is low, but more power via the LS cycle when λ is large. This is another kind of tradeoff between these two modes. In the following, we start our discussion from the first aspect.
The dual-mode strategy is devised to accommodate the high-speed feature of 100G Ethernet [11], [15]. For such high-speed Ethernet, the power saving is mainly carried out when λ is low. The major function of the FW mode is to implement the conditional sleep-mode operation to decide whether the DS mode should be implemented or not.
According to Little's Law, the mean number of arrivals during the BtF and the FW mode is λ(T BtF + T FW ). Given that N f = λ(T BtF + T FW ), the probability that fewer than N f arrivals are accumulated at the end of the FW mode is relatively large if the traffic rate is smaller than In this case, the interface implements the DS cycle with large probability, and thus achieves a high power efficiency. On the contrary, when the traffic rate is larger than N f T BtF +T FW , the number of arrivals during T BtF + T FW will be larger than N f with high probability. In this case, the interface almost always enters the LS cycle, and thus keeps its queue length at a low level. We demonstrate this point in Fig. 6, where we set λ = 2.0 frames/µs, N f = 2, T FW = 0.1µs, N = 41, and τ = 20µs such that N f = λ(T BtF + T FW ). We see from Fig. 6 that (32) and (33) agree with the simulation results very well. Also, Fig. 6 clearly shows that the dual-mode strategy is able to attain a high power efficiency when λ < N f T BtF +T FW , and a bounded queue length when λ > N f T BtF +T FW . As a comparison, Fig. 6 also depicts the performance of the DS single-mode strategy and the FW single-mode strategy, of which the analytical results are plotted according to (17), (18), (25), and (26). We can see from Fig. 6 that the queue length of the DS single-mode strategy grows fast when λ > N f T BtF +T FW , while the power efficiency of the FW single-mode strategy is quite low even when λ < N f T BtF +T FW . This is attributed to the fact that the FW single-mode strategy and the DS single-mode strategy are both unable to select proper sleep modes according to the traffic rate.
Therefore, the interface with the dual-mode strategy can attain an optimal tradeoff between the DS cycle and the LS cycle in different traffic rate regions if we configure T FW and N f in accordance with the following rule.
On the other hand, though relation (34) is given, the values of N f and T FW will also lead to another kind of performance tradeoff between the FW mode and the DS mode in different traffic rate regions.
When λ is remarkably smaller than λ * , the number of arrivals before the FW mode ends can hardly reach N f . Thus, the interface will enter the DS cycle with probability p → 1, which means Pr{ξ = 0} → 1 and Pr{ζ = 0} → 1. In this case, the system performance is almost determined by that of the DS cycle, that is, (32) and (33) change VOLUME 8, 2020 to (35), as shown at the bottom of the previous page, and Recall that we assume τ > T BtF + T FW + T FtD and N > N f in Assumption A4, and the transition time T FtD = 0.9µs is very short. Since λ is remarkably smaller than λ * , the probability that the number of arrivals during time interval T BtF + T FW + T FtD reaches N is negligible, i.e., for n ≥ N . In other words, the trigger condition of the DS Wakeup can be satisfied only after the FW mode ends. In this case, the queue length builds up during the vacation time is almost independent of N f and T FW . This point can be confirmed by substituting condition (37) into (36), which yields (38), as shown at the bottom of the this page. On the other hand, the power efficiency η in this case depends on the ratio of the FW mode to the vacation time. Intuitively, if T FW becomes large, such ratio will be large and thus η will decline. We demonstrate this point by substituting (37) into (35), as shown in (39), as shown at the bottom of the this page. The above shows that η in this case decreases with the increase of T FW . In summary, when λ is remarkably smaller than λ * , large N f and T FW impair the power efficiency, but have no benefit to the reduction of the mean queue length.
When λ is remarkably larger than λ * = N f T BtF +T FW , the interface enters the LS cycle with probability q → 1, which leads to Pr{ξ = 1} → 1 and Pr{ζ = 1} → 1. In this case, the system performance is dominated by that of the LS cycle, and the power efficiency changes to: Equation (40) shows that the mean vacation time of the LS cycle (i.e., H f (1)/λ in (40)) and the power efficiency η increase with N f and T FW . However, the increment of η is quite limited, which can be demonstrated by (40) as follows.
Since the mean vacation time of the LS cycle is larger than T BtF + T FtB , the second term in (40) satisfies Therefore, the power efficiency of the LS cycle given by (40) is upper bounded by: where 1 − ρ is much smaller than 1 when the traffic rate λ is high. On the other hand, when N f and T FW increase, the number of frames accumulated in the buffer at the end of the FW mode grows, which in turn increases the queue length. Briefly, when λ is remarkably larger than λ * , increasing N f and T FW incurs a large queue length, while only leading to a slight improvement in power efficiency. Fig. 7 plots the power efficiency η and the mean queue length L changing with N f , where λ * = 2.0 frames/µs, and N f and T FW satisfy Rule 1. In addition, we set N = 41 and τ = 20µs. We check the system performance changing with N f and T FW when λ = 0.5 frames/µs and 7.0 frames/µs. Fig. 7 clearly shows that with the increase of N f , η decreases linearly while L remains almost unchanged when λ = 0.5 frames/µs, and η converges from 0 to 0.09 while L first slightly decreases and then grows by four times when λ = 7.0 frames/µs. This result suggests that T FW and N f should be as small as possible, such that the system can save more power when the traffic rate is small, while keeping the queue length short when the traffic rate becomes high. From (34), we have N f = λ * (T BtF + T FW ). In addition, since N f is an integer, the smallest possible integer of N f is λ * T BtF , where x is the smallest integer larger than x. This yields the second selection rule for T FW and N f as follows: Rule 2: For a given steady-state traffic rate λ * , threshold N f should be selected as the following formula: According to Rules 1 and 2, we have T FW = λ * T BtF /λ * − T BtF . For example, if λ * = 2.0 frames/µs, Rule 2 will set N f = 2 and T FW = 0.1µs.

B. TRADEOFF IN DS CYCLE
Once T FW and N f are set according to Rules 1 and 2, the dualmode strategy can adaptively implement the LS cycle and the DS cycle. When the interface enters the DS cycle, the vacation time is controlled by N and τ . As Sections I and II mention, the Wakeup condition of the DS cycle of the dual-mode strategy is quite similar to that of the 10G EEE [16]. In the following, we show that the impact of N and τ on the performance of the dual-mode strategy is also similar to what happens in the 10G EEE [16]. As Section I mentions, a 10G EEE interface uses N and τ to control the duration of the LPI mode. According to Little's Law, there are λ * τ arrivals during time interval τ on average. Thus, Ref. [16] recommends setting N − 1 = λ * τ or (N − 1)/τ = λ * , such that N and τ can compensate each other under different traffic rates as follows: • When λ < λ * , the waiting time of the first arrival during the vacation tends to reach τ before N frames arrive. In this case, timer τ triggers the DS Wakeup with high probability, and thus the delay of the frame that arrives during the vacation time is bounded.
• When λ > λ * , N frames tend to arrive before timer τ expires, and the Wakeup operation is almost always triggered by N , such that the queue length at the end of the vacation is bounded. On the other hand, Ref. [16] also shows that the duration of the LPI mode increases with N and τ . If N and τ are large, the LPI mode is long such that the fraction of energy consumed by the transition operations is small, and thus power efficiency is large. As N and τ go to infinity, the power efficiency converges to a constant related to the traffic load [16]. However, the power efficiency is improved at the expense of delay performance. If N and τ are large, the frames that arrive during the LPI mode have to wait for a long time. Thus, given (N − 1)τ = λ * , Ref. [16] suggests selecting the values of N and τ to meet a preset delay requirement.
To demonstrate that N and τ have a similar effect on the performance of the dual-mode strategy, Fig. 8 plots the power efficiency, the mean delay, and the mean queue length with the following parameter setups: 1) N f = 2, T FW = 0.1µs, N = 41, and τ = 20µs, which corresponds to the τ &N policy in Fig. 8; 2) N f = 2, T FW = 0.1µs, N = 41, and τ → ∞, which corresponds to the N policy in Fig. 8, namely the interface only uses N to regulate the vacation time of the DS cycle; 3) N f = 2, T FW = 0.1µs, N → ∞, and τ = 20µs, which corresponds to the τ policy in Fig. 8, namely the interface only uses τ to regulate the vacation time of the DS cycle. As Fig. 8 shows, when λ < N −1 τ = 2.0 frames/µs, the N policy suffers a large delay, since it takes a long time to accumulate N arrivals in the buffer when the frame arrivals are sparse. When λ > N −1 τ , the τ policy suffers a serious queue length performance, since the frame arrivals are intensive and the frames accumulate rapidly during the interval τ . In contrast, the τ &N policy is able to adaptively adopt N or τ to wake up the interface according to the traffic rate. It uses τ to trigger the DS Wakeup when λ < N −1 τ , and uses N to trigger the DS Wakeup when λ > N −1 τ . Thus, the τ &N policy achieves the minimum of the mean delay and the power efficiency of the N policy and the τ policy, i.e., we have where D τ = lim N →∞ D and η τ = lim N →∞ η are the mean delay and the power efficiency of the τ policy, and D N = lim τ →∞ D and η N = lim τ →∞ η are those of the N policy. In particular, according to the results in Fig. 8, there are and when λ = N −1 τ . This point is quite similar to that of 10G EEE studied in [16]. Thus, we use the rule similar to EEE 1 presented in [16]: VOLUME 8, 2020  Rule 3: For a given steady-state traffic rate λ * , thresholds N and τ should be selected as: We proceed to determine the values of thresholds N and τ on the basis of Rule 3. Fig. 9(a) and 9(b) depict η and D versus N , where λ * = 2.0 frames/µs, T FW = 0.1µs, N f = 2, and τ varies with N according to (45). Fig. 9(a) shows that η increases with N , and converges to an upper bound when N → ∞. This can be demonstrated by setting limit N , τ → ∞ on (32). In (32), the mean cycle time of the DS cycle C d → ∞ as N → ∞, and thus the upper bound of the power efficiency η is In the example Fig. 9(a) gives, the power efficiency converges to η * = 0.72 when N and τ approach infinity. On the other hand, as Fig. 9(b) indicates, the mean delay linearly increases with N and τ . Therefore, as Fig. 9(c) plots, there is a tradeoff between D and η with the increase of N and τ . In particular, the power efficiency η can be remarkably improved with a small delay cost when η is small, but η can only be enhanced slightly even if we trade a large delay cost when η is close to its upper bound. Such a tradeoff is also quite similar to the situation in 10G EEE. Given the steady-state traffic rate λ * and Rule 3, we assume that the delay requirement is D * . According to (43), at the traffic rate λ, there is equation (46), as shown at the bottom of this page. We thus have the following rule: Rule 4: In terms of a given delay requirement D * , threshold N can be selected via (46).

C. PERFORMANCE UNDER BURSTY TRAFFIC
In practice, the traffic in Ethernet links is bursty by nature. For example, the test data in [29] indicates that the traffic   Fig. 10.
rate of a link in commercial data center networks exhibits a time-of-day feature: the traffic rate almost periodically fluctuates around an average value in the long run. In other words, though the instantaneous traffic rate can change fast and irregularly, the statistical average traffic rate varies very slowly [29], [30]. In the following, we further demonstrate via simulation that the dual-mode strategy can perform well under the bursty traffic, as long as the thresholds are set according to the statistical average traffic rate. As Fig. 6 illustrates, the advantage of the dual-mode strategy is that, given thresholds N f and T FW , it can adaptively select sleep modes according to the instantaneous input traffic. Consider the case where the input traffic rate λ varies around the steady state traffic rate λ * = N f /(T BtF + T FW ) = 2.0 frames/µs. We can see from Fig. 2(b) and Fig. 6 that (1) if λ < λ * , the dual-mode strategy will select the DS cycle more frequently than the LS cycle, such that the interface can save more power; (2) if λ > λ * , the dual-mode strategy will enter the LS cycle more often than the DS cycle, such that the interface can suppress the queue length.
To demonstrate this benefit clearly, we simulate the dual-mode strategy under the bursty traffic. We assume that the input traffic is a two-state Markov-modulated Poisson Process (MMPP) [31], where the frame arrival process alternates between two kinds of periods: high-rate period and low-rate period. The high-rate period and the low-rate period are exponentially distributed with mean durations 1/α and 1/β, respectively. The traffic rate is λ h during the high-rate period and λ l during the low-rate period. Thus, the average traffic rate of the two-state MMPP is [32] The burstiness of the two-state MMPP is defined as clearly, burstiness b increases with the decrease of α and β.
In reality, the timescales of the high-rate period and the low-rate period range from a few microseconds to several seconds [33] in different applications. Moreover, the low-rate period is usually several times longer than the high-rate period [34]. We thus set α : β = 4 : 1, λ h = 8.0 frames/µs, and λ l = 0.5 frames/µs to generate the bursty input traffic with λ * = 2.0 frames/µs in our simulation. Assume that the mean delay requirement is D * = 12µs. According to Rules 1 VOLUME 8, 2020 We observe the behavior of the dual-mode strategy in three consecutive cycles of the two-state MMPP in Fig. 10(a), where 1/α = 1ms, 1/β = 4ms. Fig. 10(b) clearly displays that the dual-mode strategy almost selects the DS cycle during the low-rate periods and the LS cycle during the high-rate periods. Consequently, as Table 1 shows, the dual-mode strategy can suppress the queue length during all these cycles, and achieve a high power efficiency when the input traffic rate is low.
As a comparison, Table 1 also provides the simulation results of the DS single-mode strategy with T FW = 0.1µs, N = 41, τ = 20µs, and the FW single-mode strategy with N f = 2, N = 41, and τ = 20µs. Table 1 clearly shows that, though the DS single-mode strategy can achieve almost the same power efficiency with the dual-mode strategy during the low-rate periods, its queue length is much longer than that of the dual-mode strategy during the high-rate periods. The FW single-mode can always suppress the queue length very well, but its power efficiency is much lower than those of the dual-mode strategy and the DS single-mode strategy.
We thus conclude that the dual-mode strategy can make a good performance compromise between the DS single-mode and the FW single-mode strategies under the bursty traffic. Fig. 11 further plots the power efficiency and the mean delay when burstiness b changes from 0 to 280. From Fig. 11, one can see that the power efficiency almost keeps unchanged, while the mean delay decreases and finally converges to 3.9µs with the increase of b. Note that the two-state MMPP traffic with b = 0 reduces to the Poisson traffic. This indicates that our analytical results cannot accurately delineate the performance of the dual-mode strategy under the bursty traffic. Even so, Fig. 11 shows that the mean delay of the dual-mode strategy under bursty input traffic is upper bounded by the analytical result. This implies that our rules proposed based on our analytical results are conservative under the bursty input traffic. Since the instantaneous Ethernet traffic changes irregularly and the burstiness varies over time [30], we conclude from Fig. 11 that our rules can provide worst-case performance guarantee.
Therefore, the dual-mode strategy can be applied in practice as follows. The Ethernet interface monitors the input traffic and calculates the statistical average traffic rate every fixed time period, e.g., one week or one month [35]. According to such collected history information, it estimates the statistical average traffic rate of the next month, and sets the thresholds using Rules 1 through 4 according to the estimated value.

V. CONCLUSION
In this paper, we propose an analytical model based on the concept of conditional sleep-mode to analyze the dual-mode strategy designed for the 100G EEE protocol. The key idea of our approach is to calculate the performance of the dual-mode strategy as the weighted average of that of the DS cycle and the LS cycle. Using this approach, we derive the power efficiency, the mean delay, and the mean queue length. Our simulation results confirm that our analysis is quite accurate. In addition, our analysis shows that the FW thresholds (i.e., N f and T FW ) mainly control the probabilities and the durations of the DS mode and the FW mode, while the DS thresholds (i.e., N and τ ) balance the power efficiency and the mean delay of the DS cycle. Based on this observation, we provide four parameter selection rules, which are easy to implement and can help the dual-mode strategy to select the suitable sleep mode according to the fluctuation of traffic rate.

APPENDIX A DIFFICULTY OF EXACT ANALYSIS OF DUAL-MODE STRATEGY
As Section II mentions, thresholds N f and T FW in the dual-mode strategy make the distribution of the number of arrivals during a vacation depend on that during the FW mode. If we directly apply the model in [16] to the dual-mode strategy, the analysis will be very complex. In the following, we elaborate on this difficulty.
According to [16], modeling of the dual-mode strategy starts from the distribution of the number of arrivals during a vacation, based on a tree of all possible events generated by the arrivals during the vacation time, as Fig. 12 shows. The most difficult part for the derivation of such distribution is to calculate the following distribution a(n) = Pr{there are n arrivals during V SL }, where V SL is the time interval from the beginning of the vacation to the time that the FW Wakeup or the DS Wakeup is triggered. Such difficulty is mainly incurred by the analysis of Events 3 and 4. To avoid tediousness, we take Event 4 as an example to illustrate the difficulty.
In the following, we are going to derive the conditional probability defined as follows: a(n|Event 4) Pr{there are n arrivals during V SL given that Event 4 happens}.
To facilitate our discussion, we divide V SL into three parts, as Fig. 12 shows. The first part, denoted by I 0 , is the time interval from the beginning of the vacation to the time that the first frame arrives, namely the first arrival interval. It is clear that I 0 is an exponentially distributed random variable as follows The second part, denoted by Y , is the duration from the first arrival instant to the end of the FW mode, and thus, The last part, denoted by Z , is the period from the end of the FW mode to the end of the DS mode, and thus, Let n y and n z be the number of arrivals in the intervals Y and Z , respectively. If N f ≤ n < N , n y must be less than N f − 1, to guarantee that the interface enters the DS mode. In this case, we have equation (50), as shown at the bottom of the next page. If 1 ≤ n ≤ N f − 1, n y ≤ n − 1 is sufficient to ensure that the interface enters the DS mode. We thus have equation (51), as shown at the bottom of the next page. We can see from (50) that the thresholds T FW and N f result in intractable convolution and integral when N f ≤ n < N .

APPENDIX B DERIVATION OF h d (n)
Here, we employ the method in [16] to calculate h d (n), the distribution of the number of arrivals during a vacation time of the DS single-mode strategy. We divide the vacation time into two parts. The first part is the period from the beginning of the vacation to the instant that the DS Wakeup is triggered, and the second part is the duration of the DS Wakeup T DtB . Let a d (n) be the distribution of the number of arrivals during the first part and b d (n) be that during the second part. The distribution h d (n) can be expressed by We derive the distribution a d (n) by analyzing the mutually exclusive events that may occur during the vacation time, as Fig. 13 shows. In particular, we consider the following four cases: 1. n = 0. The interface will not wake up if no arrival comes during the vacation time, and thus a d (0) = 0.
As Events 4 and 6 show, when the waiting time of the first arrival frame reaches τ before N frames arrive, the DS mode is ended with less than N frames in the VOLUME 8, 2020 Substituting the expressions of a d (n) and b d (n) into (52), we can obtain the distribution h d (n).

APPENDIX C DERIVATION OF h f (n)
In this part, we derive h f (n), the distribution of the number of arrivals during a vacation of the FW single-mode strategy. We divide the vacation time of the FW single-mode strategy into two parts. The first part is from the beginning of the vacation time to the instant when the FW Wakeup is triggered. The second part is the duration of the FW Wakeup T FtB . Let a f (n) be the distribution of the number of arrivals during the first part and b f (n) be that during the second part. The distribution h f (n) can be expressed by We first derive the distribution a f (n) based on the event tree in Fig. 14. We consider the following three cases: 1. n = 0, 1, · · · , N f − 1.
The interface does not wake up until N f frames are accumulated in the buffer. We have a f (n) = 0.
2. n = N f + 1, N f + 2, . . . If there are more than N f arrivals during transition time T BtF , as Event 1 in Fig. 14 shows, the interface wakes up immediately when the BtF terminates. Therefore, we have a f (n) = e −λT BtF (λT BtF ) n n! .
In Events 2 and 3, the interface wakes up from the FW mode with exactly N f arrivals in the buffer. In Event 2, exact N f frames arrive during the BtF, and the interface wakes up immediately when the BtF ends. In Event 3, there are less than N f arrivals during the BtF, and the N f th frame arrives and triggers the FW Wakeup after the interface enters the FW mode. Accordingly, we can obtain a f (n) for n = N f as follows As the duration of the FW Wakeup T BtF is a constant, the distribution of the number of arrivals during T BtF is given by b f (n) = e −λT BtF (λT BtF ) n n! , n = 0, 1, 2, . . .
Finally, substituting the expressions of a f (n) and b f (n) into (58) yields the distribution h f (n).