An Automatic Pipeline Parallel Acceleration Framework for Neural Network Models on Heterogeneous Computing Platforms

With the widespread use of heterogeneous computing platforms in the AI field, neural network models on heterogeneous computing platforms are experiencing problems such as low execution efficiency and high deployment difficulty. To solve these problems, this paper proposes an automatic pipeline parallel acceleration framework for neural network models on heterogeneous computing platforms. Firstly, the framework divides neural network computing into multiple work task stages. It uses the software pipeline design method and multi-threading technology to realize the parallel execution of multiple work task stages in neural network model computing. Secondly, the framework provides a set of universal programming interfaces for automatic pipeline parallel acceleration function. Finally, the framework realizes the rapid and efficient deployment of the neural network model on the heterogeneous computing platform without the need for specialized knowledge. Experiments have verified that the framework can bring good speedup ratios while ensuring the accuracy of the model. In particular, the average speedup ratio on the object detection model can reach 1.36 times. With the increase of the image size on object detection models, the speedup ratio increases. Therefore, the automatic pipeline parallel acceleration framework can effectively improve the intelligent processing capability of the heterogeneous computing platform.