The impact of concurrency gains on the analysis and control of multi-threaded Internet services

With the proliferation of Internet services, many solutions have emerged to provide quality-of-service (QoS) guarantees when the demands for the hosted services exceed the server's capacity. We take an analytical approach to answering key questions in the design and performance of application-level QoS techniques, especially those that are based on the multi-threading or multi-processing abstraction. Key to our analysis is the integration of the effects of concurrency into the interactions between multi-threaded services. To this end, we extend traditional time-sharing models to develop the multi-threaded round-robin (MTRR) servers, a more accurate model of operation of typical multi-threaded Internet services. For this model, we first develop powerful, yet computationally-efficient, mathematical relationships that describe the performance (in terms of throughput and response time) of multi-threaded services. We then apply optimization techniques to derive the optimal allocation of threads given specific QoS objective functions. Using realistic workloads on a typical Web server, we show the efficacy and accuracy of the proposed new methodology.