Low-overhead scheduling of nested parallelism

Nested parallelism has the potential not only to permit more parallelism than non-nested parallelism, but to result in better load balancing. However, nested parallelism will not be profitable unless the overhead of scheduling nested parallel constructs can be made nonprohibitive. Previous implementations of nested parallel constructs have been fairly expensive and therefore have not been able to exploit fine-grained nested parallelism. In this paper, we describe a run-time system that schedules a large subset of nested parallel constructs—those that run until completion without blocking—with very little overhead. Our run-time system is built around a novel scheduling policy and work queue. The scheduling policy permits efficient stack-based local-memory storage allocation for task data, which is particularly efficient for multiprocessor architectures with both shared and local memory, such as the RP3. The shared, nonlocking work queue allows processors to obtain tasks in just a few instructions, without sacrificing load balancing.