A FPGA-based FSE Accelerator with Dynamic Table for ZSTD compression

Finite state entropy (FSE) is a crucial component of the powerful Zstandard (ZSTD) lossless compressor. However, previous hardware implementations of FSE have used a static table method, causing major performance (compression ratio) losses. Therefore, in this paper, we propose a pipelined dynamic table FSE accelerator on FPGA to mitigate the compression performance loss. We present a fully pipelined dynamic table building design, which adjusts the state table size by setting four compression levels through configurable parameters. To better adapt to the pipelined design, we improved the normalization algorithm and proposed a parallel design scheme to reduce the encoding module’s waiting time. Our accelerator implemented on a Xilinx Kintex-7 KC705 FPGA board runs at 180MHz. To our knowledge, this is the first hardware accelerator of a dynamic table FSE. Experimental results show that our accelerator achieves throughputs of 735MB/s to 982MB/s, which is 15× to 38× faster than Intel(R) Core(TM) i7-13700K CPU at 3.40 GHz. Compared to previous state-of-the-art FPGA implementation, our accelerator improves the compression ratio by 23.89% (Silesia corpus) and 28.75% (Canterbury corpus) at essentially the same throughput.