HISC/R: An Efficient Hypersparse-Matrix Storage Format for Scalable Graph Processing

The need to analyze increasingly larger graph datasets has driven the exploration of new methods and unique system architectures for graph processing. One such method moves away from the typical edge- and vertex-centric approaches and describes graph algorithms using linear-algebra operations, bringing the added benefits of predictable data-access patterns and ease of implementation. The performance of this approach is limited by the sparse nature of graph adjacency matrices, which leads to inefficient use of memory bandwidth, and reduced scalability in distributed systems. In order to maximize the scalability and performance of these linear-algebra systems, we require new sparse-matrix storage formats capable of maximizing memory throughput and minimizing latency, while maintaining low storage overhead. In this paper, we present an overview of a novel sparse-matrix storage format called Hashed-Index Sparse-Column/Row (HISC/R) which guarantees constant-time row or column access complexity at low storage overhead, while also supporting online non-zero element insertions and deletions. We evaluate the performance of HISC/R using randomly generated Kronecker graphs, demonstrating a 19% reduction in memory footprint, and 40% reduction in memory reads, for sparse matrix/matrix multiplication compared to competing formats.