An “A+” Heuristic for Dispatching in Large-Scale Systems with Unknown Server Speeds

How to dispatch jobs to servers is a question of critical importance in today’s computer systems. While there is a long history of literature on dispatching, much of this work focuses on settings where servers are homogeneous in their speeds—an unrealistic assumption in modern computer systems. Dispatching policies that are heterogeneity-aware typically assume that the dispatcher has access to detailed information about all servers’ speeds; this may also be unrealistic. In this paper, we study the setting in which server speeds are heterogeneous and the dispatcher has no information about server speeds. We propose a novel heuristic, called server accomplishment, that can be used as a proxy for detailed server speed information in heterogeneity-aware dispatching policies. Using extensive simulation studies, we demonstrate that our accomplishment-based dispatching policies are able to bridge most of the performance gap between heterogeneity-unaware policies, such as JSQ(d), and policies that make use of detailed server speed information.