Adaptive Dynamic Scaling and Request Routing Optimization in the Multi-Edge Cluster Collaboration

With the rapid proliferation of mobile devices, a growing number of intelligent applications are being deployed at the network edge, placing immense strain on the processing capabilities of edge computing. Therefore, resource-constrained edge servers frequently experience overload due to highly dynamic workloads. To address this, one approach involves forwarding user requests to the cloud or other edge servers, albeit at the cost of increased transmission latency. Alternatively, dynamic scaling of edge clusters can be employed to enhance processing capacity, thereby mitigating latency but at the expense of additional service configuration and hosting expenses. By integrating their complementary benefits, we study the joint optimization problem of dynamic scaling and request routing within a multi-edge cluster collaborative framework, which fully exploits cluster resources to manage the temporal and spatial varying edge workloads. This collaborative framework aims to minimize overall request latency while satisfying an acceptable time-averaged budget cost. However, the complex coupling between scaling and routing decisions, along with the uncertainty of future system information (e.g., user request workloads) impedes the derivation of an optimal offline policy over the long term. Thus, considering the different decision granularities, we employ the two-timescale Lyapunov optimization technique to decouple the original problem into a series of independent online optimization problems with the current system state. In particular, we make cluster scaling decisions in each large timescale and request routing decisions in each small timescale. Given that the decoupled large-timescale subproblems involve NP-hard mixed-integer linear programming, we design an edge resource-aware greedy rounding algorithm to efficiently produce approximate optimal solutions. Finally, both rigorous theoretical analysis and extensive trace-driven evaluations demonstrate the superiority of our proposed algorithm over its counterparts.