Cluster load balancing for fine-grain network services

This paper studies cluster load balancing policies and system support for fine-grain network services. Load balancing on a cluster of machines has been studied extensively in the literature, mainly focusing on coarse-grain distributed computation. Fine-grain services introduce additional challenges because system states fluctuate rapidly,for those services and system performance is highly sensitive to various overhead. The main contribution of our work is to identify effective load balancing schemes for fine-grain services through simulations and empirical evaluations on synthetic workload and real traces. Another contribution is the design and implementation of a load balancing system in a Linux cluster that strikes a balance between acquiring enough load information and minimizing system overhead. Our study concludes that: 1) Random polling based load-balancing policies are well-suited for fine-grain network services: 2) A small poll size provides sufficient information for load balancing, while an excessively large poll size may in fact degrade the performance clue to polling overhead; 3) Discarding slow-responding polls can further improve system performance.