AMG: An Auto-Tuning Method for Spark GraphX Running on GPU Cluster

With the explosive growth of social networking services and traffic information, graph structured data has promoted the development of analytical tools. GraphX is an API of Apache Spark for graph analytics. NVIDIA has created a library named RAPIDS to enable the execution of Spark 3.0 including GraphX on GPUs. It has a lot of configurable options related to GPU which can significantly affect GraphX programs' performance. However, the default options of Spark GraphX may not lead to the optimal performance for a given GraphX program and its input data and thus they need to be carefully tuned. To address these issues, we propose an automatic tuning approach named AMG to efficiently identify the optimal option for a given GraphX program-input pair running on a GPU cluster. AMG innovates two techniques: 1) it leverages Random Forest to construct performance models; 2) it employs Bayesian Optimization to seek the optimal option for a given GraphX program-input pair. We use three typical GraphX programs to evaluate AMG. The experimental results show that compared with the default options, AMG dramatically improves the performance by a factor of 3.99× on average and up to 4351×.