An Adaptive Scheduling Policy for Staged Applications

The performance of Web servers and application servers is a crucial factor for the success of the underlying business activity. Current commercial servers (such as Apache and Microsoft’s IIS) usually employ a thread-based concurrency policy for executing request simultaneously. However, with this policy the throughput tends to collapse when massive overload occurs. Event-based concurrency has been proposed as an alternative, but-–similar to the thread-based approach–-it suffers in its pure form from a potential memory access bottleneck. This is avoided by a server based on a Staged Event-Driven Architecture (SEDA) which combines the two approaches to take advantage of locality in data and code within each state.  In this paper, we present a three layer observer/ controller architecture for dynamic performance control of SEDA-based application servers. Control is achieved by a strategy which adapts the resources assigned to each stage based on the observed throughput and queue length of each stage using a feedback loop. To validate our strategy and to compare with other strategies, we implemented a simulation environment. The results gained demonstrate that our approach successfully self-optimizes the CPU time allocation and achieves a competitive system throughput while avoiding performance degradation under overload and dynamic changes in the system.