Concurrent Collections on Distributed Memory Theory Put into Practice

Finding and expressing scalable parallelism is a non-trivial task, in fact it is one of the most difficult parts of software development. Concurrent Collections (CnC) is a novel programming model which aims to make this easy. Its higher level abstractions expose available parallelism implicitly through specifying the semantically required dependencies between individual computation kernels. It has been shown conceptually to be deterministic, independent of the target platform and to separate program semantics from tuning. While abstractly evident, there have been no concrete implementations yet which show that these concepts are actually generally exploitable in practice. We developed an implementation of CnC which exposes these benefits in a single model for both shared and distributed memory. Additionally, we provide a tuning interface which allows defining and optimizing distribution plans easily and flexibly. Unlike most approaches, our implementation allows changing the distribution without altering the computation code itself. This makes the development very productive because it separates the concerns of program semantics and tuning. Last but not least, we show that the new mechanisms not only preserve CnC's deterministic model but are also capable of providing competitive performance. We ported several applications and ran them on a cluster of multi-cores. Our results show that CnC performance matches and often outperforms that of existing state-of-the-art models.