Parallel Computation Of Deep Neural Network Optimized Based On ML-Metis

In recent years, accelerated computation of deep neural networks(DNN) has become a hot topic. Using Metis, a graph partitioning algorithm, to divide the DNN data flow graph, and then perform model parallelization on DNN is a common method to improve the speed of its calculation. However, because Metis cannot capture all the characteristics of a data flow graph, it is often inefficient. This paper proposes an improved graph partitioning algorithm, ML-Metis, which can take into account the memory reuse and intermediate data of the data flow graph during partitioning, and provide an efficient parallel strategy for DNN.