CAFGD: Reduce Fragmentation in Large-Scale Multi-tenant Clusters for GPU Sharing Workloads

GPU fragmentation has emerged as a critical obstacle to maximizing resource utilization in large-scale multi-tenant GPU clusters, despite advancements in GPU sharing techniques. This issue stems from the tidal nature of load fluctuations, the hierarchy of resource allocation, and variations in task runtime and resource requests. To address these challenges, we propose the Cluster-Aware Fragmentation Gradient Descent (CAFGD) algorithm, which formulates resource allocation as a hierarchical multi-dimensional packing problem with time dimension. CAFGD dynamically evaluates GPU fragmentation by considering task life cycles, balances CPU and GPU resources through a counterweight approach, and optimizes the allocation structure during low-demand periods with an optimistic waiting strategy aligned to cluster periodicity. Experiments on real-world traces with over 50,000 tasks demonstrate that CAFGD reduces unallocated GPU resources by more than 13.93% compared to existing packing-based schedulers, showcasing its effectiveness in enhancing cluster efficiency and GPU utilization.