MSBF-LSTM: Most-significant Bit-first LSTM Accelerators with Energy Efficiency Optimisations

Long short-term memory (LSTM) recurrent networks are frequently applied to sequence processing problems such as speech recognition and video classification. In this paper, we propose a novel LSTM network implementation based on most-significant bit-first (MSBF) arithmetic on FPGAs, called MSBF-LSTM, to improve the energy efficiency of LSTM inference engines. Furthermore, an LSTM model compression strategy with incremental network quantization is proposed to achieve high accuracy with low-precision weights. Hardware implementations conducted on the Xilinx UltraScale+ Zynq xczu9eg FPGA demonstrate that MSBF-LSTM achieves 1.51× better energy efficiency compared with the state-of-the-art FPGA-based LSTM designs.