Time-Stable Performance in Parallel Queues With Non-Homogeneous and Multi-Class Workloads

Motivated by applications in data centers, we consider a scenario where multiple classes of requests arrive at a dispatcher at time-varying rates which historically has daily or weekly patterns. We assume that the underlying environment is such that at all times the load from each class is very high and a large number of servers are necessary which, for example, is fairly common in many data centers. In addition, each server can host one or more classes. Design, control and performance analysis under such heterogeneous and transient conditions is extremely difficult. To address this shortcoming we have suggested a holistic approach that includes a combination of sizing, assignment, and routing in an integrated fashion. Our proposed approach decomposes a multi-dimensional and non-stationary problem into a one-dimensional, simpler and stationary one, and achieves time-stability by introducing an insignificant number of dummy requests. Based on time-stability, our suggested approach can provide performance bounds and guarantees for time-varying and transient system. Moreover, we can operate the data centers in an energy-efficient manner via suggested approach.