VersaPipe: A Versatile Programming Framework for Pipelined Computing on GPU

Pipeline is an important programming pattern, while GPU, designed mostly for data-level parallel executions, lacks an efficient mechanism to support pipeline programming and executions. This paper provides a systematic examination of various existing pipeline execution models on GPU, and analyzes their strengths and weaknesses. To address their shortcomings, this paper then proposes three new execution models equipped with much improved controllability, including a hybrid model that is capable of getting the strengths of all. These insights ultimately lead to the development of a software programming framework named VersaPipe. With VersaPipe, users only need to write the operations for each pipeline stage. VersaPipe will then automatically assemble the stages into a hybrid execution model and configure it to achieve the best performance. Experiments on a set of pipeline benchmarks and a real-world face detection application show that VersaPipe produces up to $6.90 \times (2.88 \times$ on average) speedups over the original manual implementations. CCS CONCEPTS • General and reference $\rightarrow$ Performance; • Computing methodologies $\rightarrow$ Parallel computing methodologies; • Computer systems organization $\rightarrow$ Heterogeneous (hybrid) systems;