Multiplication of Sparse Matrices and their Transpose Using Compressed Sparse Diagonals

Matrix-matrix multiplication is one of the most important kernel in linear algebra operations with a multitude of applications in scientific and engineering computing. Sparse matrix computation on modern High-performance Computing (HPC) architecture presents challenges such as load balancing, data locality optimization, and computational scalability. Data structures to store sparse matrices are designed to minimize overhead information as well as to optimize the operations count and memory access. In this study, we present a new data structure, “compressed sparse diagonal” (CSD), to efficiently store and compute with general sparse matrices. The CSD builds upon the previously developed diagonal storage - an orientation-independent uniform scheme to compute with “structured” matrices [1]. Compared with the widely used compressed sparse row/column (CSR/CSC), the CSD scheme avoids explicit transposition operation when multiplying a matrix with its transpose. The results from preliminary numerical experiments with the aforementioned types of matrices demonstrate the CSD scheme's effectiveness in matrix-transposed matrix multiplication.