Randomized Load Balancing in Finite Regimes

Randomized load balancing is a cost efficient policy for jobscheduling in parallel server queueing systems whereby, with everyincoming job, a central dispatcher randomly polls some servers andselects the one with the smallest queue. By exactly deriving thejobs' delay distribution in such systems, in explicit and closedform, Mitzenmacher [13] proved the so-called 'power-of-two' result, which states that the random polling of only two serversyields an exponential improvement in delay over randomly selectinga single server. Such a fundamental result, however, was obtainedin an asymptotic regime in the total number of servers, and doesdo not necessarily provide accurate estimates for practical finiteregimes with small or moderate number of servers. In this paper weobtain stochastic lower and upper bounds on the jobs' averagedelay in non-asymptotic/finite regimes, by extending ideas foranalyzing the particular case of the Join-the-Shortest-Queue (JSQ) policy. Numerical illustrations indicate not only that the (lower) bounds are remarkably accurate, but also that the asymptoticapproximation can be misleading in scenarios with a small numberof servers, and especially at very high utilizations.