BISQ: A Bit-level Sparse Quantized Accelerator For Embedded Deep Neural Networks

Deep Neural Networks (DNNs) are pivotal in modern applications, yet their inference phase remains computationally demanding. Quantization, the process of reducing the precision of numerical representations, holds immense promise for alleviating the computational burden of DNNs. However, existing approaches often sacrifice accuracy or fail to exploit the potential efficiency gains fully. Thus, a crucial research gap exists in developing quantization techniques that maintain accuracy while significantly enhancing efficiency. We introduce a novel quantization method and architecture to optimize DNN computations efficiently.By exploiting the quantized bit patterns and performing bit-level multiplications, we potentially reduce the number of computations needed by multiplications during inference across a wide selection of reduced-precision neural networks. Our quantization method and the proposed optimized operations produce numerically identical results, do not affect overall accuracy, and boost the energy efficiency of inference in DNNs. Measurements demonstrate that for convolutional layers, our proposed method is (4.2× to 10.4×) faster, (3.2× to 8.6×) more energy-efficient, and has 10× less area than the state-of-the-art 8-bit fixed-point representation in 45nm technology.