Understanding the impact of lossy compressions on IoT smart farm analytics

As the volume of data collected by various IoT stations increases, Big Data management and analytics becomes a huge challenge for IoT applications. Although Big Data can potentially benefit from data compression techniques, the chances are that compression will reduce a negligible amount of data such that it would not worth the effort. The insight of this paper is that only lossy compression can unleash the power of compression to IoT because, compared with its counterpart (lossless one), it can significantly reduce the data volume by taking advantages of spatiotemporal patterns. However, lossy compression faces the challenge of compressing too much data thus losing the data fidelity, which might affect the quality of analytics outcomes. To understand the impact of lossy compression on IoT data management and analytics, we evaluate several classification algorithms on agricultural sensor data reconstructed based on energy concentration. Specifically, we applied three transformation based lossy compression mechanisms to five real-world sensor data from IoT weather stations. Our experimental results indicate that there is a distinctive relationship between energy concentration on the transformed coefficients and compression ratio as well as the amount of error introduced. While we observe a general trend where the higher energy concentration the lower compression and error rates, we also observe that the impact on classification accuracy varies among data sets and algorithms we evaluated.