A Parallel Skeleton Library for Multi-core Clusters

A parallel skeleton library is a collection of parallel computations that abstract generic and recurring patterns within parallel programs and conceal parallel behaviors as skeletons. It enables users to develop parallel programs as if they were sequential ones by composing suitable skeletons. However, many existing parallel skeleton libraries for distributed environments do not take into account the potential performance of multi-core CPUs, because they operate under the premise that each node (computer) has a single-core CPU. To resolve this problem, this paper proposes the design and implementation of a parallel skeleton library for multi-core clusters. The proposed library adopts a two-stage dynamic task scheduling strategy; the first is among nodes and the second is among cores. This scheduling strategy enables the library to appropriately balance the load both between nodes and cores. The library also dynamically fuses successive skeleton calls to reduce the cost of control flows and increase the locality of data. The proposed skeletons are implemented from scratch for matrices within a parallel skeleton library called SkeTo by using the template techniques in C++ language. We confirmed that our implementation was efficient through various benchmarks.