Synchronization in Parallel Programming Models for Heterogeneous Many-Cores

Even though the architecture and programming models of heterogeneous many-core processors significantly differ from the conventional multi-core processors, their overall performance is promising for future computing systems. The application programs should be suitably parallel to unlock such potential and match the underlying heterogeneous platform. Therefore, high-level programming constructs should be provided by parallel programming models for heterogeneous many-cores to avoid recurrent programming errors while communicating in heterogeneous many-core systems. Synchronization is one of the key problems in building shared-resource-based parallel software. In this article, we survey mainstream and novel parallel programming models that handle this troublesome issue for heterogeneous Many-Cores: OpenMP, CUDA, OpenCL, Go, Kokkos, OmpSs, and XcalableMP. We also discuss potential research directions in the area.