Local-to-Cloud Database Synchronization via Fine-Grained Hybrid Compression

With the increasing migration of business operations to the cloud, cloud service providers are facing a growing demand for faster database synchronization across diverse network conditions. Thus, compression methods are predominantly employed over the synchronized database binlog files to reduce the volume of data to be transmitted across the network. However, previous solutions typically rely on using one single compression method. This can result in data compression rates failing to align well with network bandwidth, causing data to wait for compression or transmission, thereby leading to inferior performance. To address the above issues, we propose a fine-grained hybrid adaptive compression system that (1) parses binlog files into multiple fine-grained blocks, and (2) applies a hybrid combination of multiple compression methods to seamlessly align compression speed with the network bandwidth. We have conducted extensive evaluations which demonstrate that, compared to the cutting-edge compression methods like ZSTD, LZ4, and Snappy, our approach can cut down the average latency by 66% and improve the synchronization throughput by 2.45×.