Evaluation of an Interference-free Node Allocation Policy on Fat-tree Clusters

Interference between jobs competing for network bandwidth on a fat-tree cluster can cause significant variability and degradation in performance. These performance issues can be mitigated or completely eliminated if the resource allocation policy takes the network topology into account when allocating nodes to jobs. We implement a fat-tree network topology aware node allocation policy that allocates isolated partitions to jobs in order to eliminate inter-job interference. We compare the impact of this node allocation policy to a topology-oblivious policy with respect to the execution time of individual jobs with different communication patterns. We also evaluate the cluster's quality of service using metrics such as system utilization, schedule makespan, and job wait time for both policies. The results obtained for production workloads indicate that a topology-aware node allocation can provide interference-free execution without negatively impacting the cluster's quality of service.