Load Balancer Filter-Based Approach To Enable Distributed API Rate Limiting

Efficient and accurate rate limiting is crucial for managing API traffic in distributed systems, ensuring fair resource allocation, preventing abuse, maintaining reliability, and enabling monetization. Unlike single-node approaches, distributed rate limiting poses challenges in maintaining consistent API rate limits across multiple nodes. This research explores implementing distributed API rate limiting through load balancers, evaluating six algorithms integrated via Lua filters that enforce client-specific global rate limits while balancing performance and accuracy. To synchronize client states across load balancers, three mechanisms are assessed: Redis, MySQL, and Conflict-Free Replicated Data Types (CRDTs). Latency, throughput, and throttling deviation serve as key metrics to evaluate the results. To reduce the potential state synchronization overhead among load balancers, this research introduces a novel asynchronous batch-quota-based implementation. Experiments with a microservices benchmarking application and API traffic simulation in Google Cloud Platform (GCP) demonstrate that the load balancer-based rate limiting framework introduces minimal performance impact, with latency overhead remaining below 1% for certain configurations, proving its high viability. The Sliding Window Log algorithm had the lowest throttling deviation, while all algorithms showed negligible performance differences. For state synchronization, CRDTs exhibited the lowest latency overhead, followed by Redis and MySQL, with all three offering comparable consistency. These findings provide practical insights for API providers when selecting rate-limiting strategies for distributed environments.