XLA-LB: Latency Aware Load Balancing Using XDP

In distributed systems, stateless services are replicated across multiple machines to ensure scalability and performance. These services, referred to as servers, can experience fluctuations in service time latencies due to varying incoming request rates, which can affect overall system efficiency. Therefore, an effective load-balancing strategy is crucial for distributing requests based on the service time latencies of these servers.Traditional software load balancers are efficient, but have limitations in performance due to protocol stack overhead, where packet processing and user-space interactions consume significant CPU cycles. In contrast, we present an in-kernel packet processing using XDP (eXpress Data Path) which we refer to as XLA-LB (XDP Latency-Aware Load Balancer). XLA-LB dynamically forwards requests based on service time latencies, which are obtained by periodically sending probe requests to the servers. XLA-LB is implemented using a combination of user-space and XDP programs. The evaluation demonstrates improved service time latencies and packet forwarding performance compared to other load balancing strategies.