Tell-tale tails: Decomposing response times for live internet services

Internet services use a wide range of software during request processing. One very slow software component can increase response time significantly. Or, many slightly slow components can conspire to increase response time. This paper 1) describes an approach to model the slowdown caused by each software component and 2) diagnoses root causes of tail response times in live services. Our approach uses Independent Component Analysis (ICA) to decompose the response time of independent, parallel requests into normalized software delays. Our approach models delays with less than 17% error for a wide range of software. It also captures the degree of data parallelism for each component, a measure of normalized energy footprint. We applied our approach to 33 services hosting real users, e.g., Google, Bing and Twitter. We observed that the services with slowest tail response times were affected by many components conspiring to slow down response time. We also found that energy footprint and tail response time were correlated.