TailCut: Power Reduction under Quality and Latency Constraints in Distributed Search Systems

Web search constitutes an important class of data-intensive online services in data centers. Optimizing search systems for energy efficiency, timely response and high search quality (i.e., how relevant the returned results are to a search query), however, is very challenging, as a search system involves a distributed architecture with hundreds of thousands of index serving nodes (ISNs) that return searching results to an aggregator through multiple interdependent retrieval stages in a partition-aggregate fashion. In this paper, we discover through experiments two important characteristics that can affect the system performance: (1) response time and energy consumption are greatly impacted by a small fraction of queries with long processing times; (2) the quality contribution of the ISN is independent of the query processing time. Based on our observation, we propose TailCut, which judiciously discards long query executions and enables ISN-aggregator coordination to minimize energy consumption subject to latency and quality constraints. Our experimental results show that TailCut can achieve up to 39% power saving, while satisfying the tail latency and quality constraint.