Adaptive Scheduling for Staged Applications: The Case of Multiple Processing Units

Staged design has been introduced as a programming paradigm to implement high performance Internet services that avoids the pitfalls related to conventional concurrency models. However, this design presents challenges concerning resource allocation to the individual stages, which have different demands that change during execution. On the other hand, processing resources have been shown to form the bottleneck in a variety of Internet-based applications. For this reason, parallel processing hardware techniques have been employed in order to cope with the massive concurrency and the increasing demands for performance aspects in these applications. Recently, the rise of multi-core technology introduces a hierarchic parallelism in modern server machines that has to be considered when allocating processing units in order to improve the utilization of these resources. This paper, introduces an adaptive policy to allocate processing units in Internet services that are based on the staged architecture. The proposed approach takes the hierarchic parallelism in account and adapts the resources assigned to the individual stages dynamically based on the observed demand of each stage using a feedback loop. Simulation results demonstrate that our approach achieves a competitive system throughput, avoids overhead that is associated with parallel processing, and successfully adapts resource allocation to dynamic changes in workload characteristics.