Parallel matrix multiplication algorithms in Supercomputing

Nowadays high-performance computing is gradually implementing Exa-scale computing, and the performance of single node has reached several T-flops. Communication problem has become one of the main concerns of parallel matrix multiplication algorithms. This paper surveys the research on PMM algorithms on supercomputers around the world. We first stress the significance of PMM (Parallel Matrix multiplication) algorithms in high performance scientific computing, followed by a background introduction on widely employed PMM algorithms. We introduce the researches on parallel techniques for PMM algorithms from two aspects: classical parallel matrix multiplication and Strassen's fast matrix multiplication. Further, we present the detailed complexity analysis comparison of each algorithm. Finally, we summarize the paper and discuss potential directions of future work.