Bit-Balance: Model-Hardware Codesign for Accelerating NNs by Exploiting Bit-Level Sparsity

Bit-serial architectures can handle Neural Networks (NNs) with different weight precision, achieving higher resource efficiency compared with bit-parallel architectures. Besides, the weights contain abundant zero bits owing to the fault tolerance of NNs, indicating that bit sparsity of NNs can be further exploited for performance improvement. However, the irregular proportion of zero bits in each weight causes imbalanced workloads in the Processing Element (PE) array, which degrades performance or induces overhead for sparse processing. Thus, this article proposed a channel-wise bit-sparsity quantization method that keeps the non-zero bit number of each weight in each channel from exceeding a certain threshold and clusters the channels with the same threshold to balance the workloads in PE array with little accuracy loss. Then, we co-designed a sparse bit-serial architecture, called Bit-balance, to improve overall performance, supporting weight-bit sparsity and adaptive bitwidth computation. The whole design was implemented with 65 nm technology at 1 GHz and performs at 447-, 37-, 59-, 240-, and 19-frame/s for AlexNet, VGG-16, ResNet-50, GoogleNet, and Yolo-v3 respectively. Compared with sparse bit-serial accelerator, Bitlet, Bit-balance achieves 1.6
$\boldsymbol{\times}$
2.1
$\boldsymbol{\times}$
energy efficiency (frame/J) and 2.3
$\boldsymbol{\times}$
3.6
$\boldsymbol{\times}$
resource efficiency (frame/mm
${}^{\mathbf{2}}$
).