Identifying Important Data Transformations for Synthesizing Effective Lossless Compressors

Efficient data compression is essential in high-performance computing, particularly when managing large-scale datasets. Lossless compression algorithms generally incorporate multiple stages, i.e., data transformations. Our LC synthesis framework contains an extensive library of such transformations that it can combine in any order. We have used some of these transformations to build state-of-the-art CPU/GPU-compatible compressors such as SPspeed and SPratio. In this paper, we evaluate the effectiveness of these transformations in achieving high compression ratios on single- and double-precision datasets. We present a comprehensive ranking of these transformations and explore their impact across different datasets. Our analysis reveals that not all transformations are equally effective across datasets, even across datasets produced by the same simulation code, underscoring the need for customized compressors to maximize compression ratios. Using this ranking, we are able to eliminate unimportant transformations, enabling us to more quickly search for and find well-compressing algorithms with more stages. Additionally, we study the importance of these transformations in each compression-algorithm stage and show that the various stages prefer distinct transformations.