A Low-Complexity and High-Throughput Hardware Design for Lempel-Ziv 4 Compression Algorithm

The Lempel-Ziv (LZ) 4 compression algorithm, widely used in data transmission and storage, faces the challenge of high-speed implementation and increased complexity in the era of big data. Therefore, this paper proposes a single-core parallel architecture for LZ4 algorithm with high throughput and low complexity. Firstly, to enhance throughput, two innovative approaches are introduced from the perspective of parallelism and frequency with an acceptable compression ratio loss: each parallelization window is restricted to performing a single match, bridging the gap between actual and theoretical parallelism; the feedback loop in the circuit is broken by utilizing the spatial correlation between adjacent matches for higher frequency. Secondly, two optimization schemes are employed on resource-consuming modules to achieve low complexity. Multi-port hash tables using Live Value Table (LVT) are improved based on inherent data characteristics, significantly reducing the hardware resource consumption while ensuring excellent scalability on hash table depth and frequency. The match comparison operation is moved ahead, further reducing the logic resources by 64.36%. Finally, our design is implemented on FPGA and ASIC platforms. Experimental results on FPGA demonstrate that the proposed architecture achieves a throughput of 17.39 Gb/s, exhibiting a 2.86 $\times $ improvement over the state-of-the-art, along with a 6.46 $\times $ enhancement in area efficiency. Further optimizations including Canonic Signed Digit (CSD) coding and computational reuse on the ASIC platform result in a $45\times $ improvement in area efficiency.