Performance Optimization of High-Performance LINPACK Based on GPU-Centric Model on Heterogeneous Systems

Nowadays, computing environments are becoming more and more multifaceted. The emergence of the accelerator has made heterogeneous computing become the mainstream computing mode in the field of High-Performance computing. Especially, with the rapid improvement of GPU in data parallel processing capability and storage bandwidth, the presence of multiple GPUs on a system presents an opportunity for applications to exploit parallelism and concurrency, and improves overall system performance. However, current heterogeneous applications are mostly designed to be CPU-centric, which depends heavily on the process capability of CPU and the transmission rate of PCI-E, inefficient data movement between host memory and device memory decreases the parallel computing capability of the entire computer system severely. In this paper, we develop a GPU-centric model for optimizing High-Performance LINPACK(HPL) benchmark on multi-core and multi-GPU system. The raw data is directly generated in the GPU memory and the computational tasks are redistributed between CPU and GPU according to its own computational characteristics. Additionally, tuning techniques such as multi-stream concurrency, asynchronous iteration and dynamic workload partitioning are also employed to enhance performance. And the experimental outcome proves that the optimization based on the GPU-centric model is valid and effective.