Parallelizing MPI Using Tasks for Hybrid Programming Models

Many-core architectures such as the Intel® Xeon PhiTM provide dozens of cores and hundreds of hardware threads. For these machines, a basic MPI implementation is inefficient, as it does not take advantage of the shared data across the ranks on the same node. A hybrid-programming model called MPI+X, where MPI is used between the nodes and a threading model X (Pthreads or OpenMP*, among others) can better utilize the compute and memory resources. In this paper, we investigate how the MPI library can run in parallel without competing with the threads of X that are running the application, and in particular without oversubscribing the available hardware resources. We assess whether MPI can use tasks that can be executed by idle threads in X. To prototype such a system, we have chosen X to be OpenMP. Our experimental results show that parallelizing MPI calls using OpenMP tasks can provide significant speedups.