A Non-blocking Programming Framework for Pipeline Application on Multi-core Platform

Many applications meet certain programming patterns like pipeline, fork-join, do-all etc. While tools such as OS threads and OpenMP allow programmers only to express task or data parallelism, special support for programming patterns is distinctly lacking. Intel threading building blocks (TBB) is developed to address this problem, but its scheduler is general and not optimized for any of its parallel algorithms which include pipeline specially. In this paper, we provide a non-blocking framework for pipeline application on multi-core platform. We target linear pipeline in which each filter has one entrance and one exit. We design a novel work-stealing scheduler optimized specially for pipeline application: first, priority based stealing, priority is calculated for each filter in pipeline so that a worker can find the optimal "victim" easily when it needs to steal, second, multiple tasks can be stolen at a time so that much stealing time is reduced. A non-block queue is used to store intermediate result to reduce lock overhead and increase scalability. We apply our framework to four case studies, including text filter, two fish, ferret, ded up. And our framework reduces execution time of TBB by 72% in best case and 20% on average on an 8 core machine.