Skip to Main Content
Load balancing in publish/subscribe (pub/sub) broker networks is challenging as the workload is multi-dimensional and content-dependent. In this paper we present the framework design of a middleware, called Shuffle, to achieve optimal load balancing in a pubbroker network. Shuffle features a suite of active workload management schemes within a single overlay topology on message parsing, matching, delivery, and forwarding, the four types of workload in a publishsubscribe service affected by two inputs-streaming events and stored subscriptions. Shuffle leverages its traffic randomization scheme and Chord, a DHT substrate, to build overlay trees for active workload aggregation and distribution, and we show the optimality property of the load balancing scheme upon any input traffic distribution on individual Shuffle aggregation trees. We also show the NP-hardness of the workload management problem when it has to be done among multiple correlated aggregation trees, and present a heuristic accordingly. Through extensive simulations we validated the design of Shuffle upon dynamic and heavy workload.
Date of Conference: 19-23 May 2008