Deep Nibble: A 4-bit Number Format for Efficient DNN Training and Inference in FPGA

This paper introduces a compact number format (Deep Nibble) and a resource- and performance-efficient dot product core (Deep Nibble Unit – DNU) designed to address the performance and memory bottlenecks of deep learning on resource-constrained devices. Deep Nibbles are 4-bit signal-exponent numbers generated by the proposed normalized stochastic logarithmic quantization (NSLQ) method. This method mitigates the gradient vanishing and exploding problems in training deep neural networks (DNNs) using narrow data types. DNU is a Deep Nibble-based core designed to accelerate MAC operations of forward and backward passes of deep learning systems. Compared to FP32 formats, the experimental results show that Deep Nibble supports the training of DNN models with an acceptable accuracy degradation while reducing the memory footprint by 8×. Compared to state-of-the-art, the FPGA implementation of the DNU reduces the area per MAC and increases the throughput per area by 3× and 2.3×, respectively.