A Low-Power Arithmetic Element for Multi-Base Logarithmic Computation on Deep Neural Networks

Computational complexity and memory intensity are crucial in deep convolutional neural network algorithms for deployment to embedded systems. Recent advances in logarithmic quantization has manifested great potential in reducing the inference cost of neural network models. However, current base-2 logarithmic quantization suffers from performance upper limit and there is few work that studies hardware implementation of other bases. This paper presents a multi-base logarithmic scheme for Deep Neural Networks (DNNs). The performance of Alexnet is studied with respects to different quantization resolutions. Base $-\sqrt{2}$ logarithmic quantization is able to raise the ceiling of top-5 classifying accuracy from 69.3% to 75.5% at 5-bit resolution. A segmented logarithmic quantization method that combines both base-2 and base $-\sqrt{2}$ is then proposed to improve the network top-5 accuracy to 72.3% in 4-bit resolution. The corresponding arithmetic element hardware has been designed, which supports base $-\sqrt{2}$ logarithmic quantization and segmented logarithmic quantization respectively. Evaluated in UMC 65nm process, the proposed arithmetic element operating at 500MHz and 1.2V consumes as low as 120 $\mu \mathbf{W}$. Compared with 16-bit fixed point multiplier, our design achieves 58.03% smaller in area, with 73.74% energy reduction.