Minimizing electricity cost for geo-distributed interactive services with tail latency constraint

Cross-border data movement has become increasingly more costly, and even been prohibited due to data sovereignty requirements. Consequently, geo-distributed interactive services, which rely on geographically distributed data sets, is quickly emerging as an important class of workloads in data centers and resulting in soaring electricity costs. While numerous geographic load balancing (GLB) techniques exist to exploit differences in electricity prices for cost savings, they do not apply to emerging geo-distributed interactive services due to two major limitations. First, they assume that each request is processed only in one data center, whereas each geo-distributed interactive request must be processed at multiple data centers simultaneously. Second, they primarily focus on meeting average latency constraints, whereas tail latencies (i.e., high-percentile latencies) are more suitable to ensure a consistently good user experience. In this paper, we make an early effort to optimize GLB decisions for geo-distributed interactive services, exploiting spatial diversity of electricity prices to minimize the total electricity cost while meeting a tail latency constraint. Our solution employs a novel data-driven approach to determine the tail latency performance for different GLB decisions, by profiling the network latency and data center latency at a low complexity. We run trace-based discrete-event simulations to validate our design, showing that it can reduce the electricity cost by more than 7% while meeting the tail latency constraint compared to the performance-aware but cost-oblivious approach.