Transform-based Lossy Compression for HPC Big Datasets

As many IoT devices generate an enormous and varied amount of data that need to be processed in a brief space of time, storing and processing IoT big data becomes a huge challenge. While lossy compression can drastically reduce data volume, finding an optimal balance between volume reduction and information loss is not an easy task. Motivated by this, we analyze the characteristics of data compressed and present relationships between compression ratio and data characteristics to use an effective lossy compression algorithms.