Stochastic Computing for Low-Power and High-Speed Deep Learning on FPGA

Stochastic Computing (SC) presents a low-cost and low-power alternative to conventional binary computing. In SC, continuous values are represented by stochastically generated bit streams. By performing simple hardware-friendly bit-wise operations on these streams, complex calculations can be realized very efficiently. However, the inherent randomness and approximation used in SC can result in undesirable computational errors. As Convolutional Neural Networks (CNNs) are inherently error-tolerant, SC could be embedded in them to gain higher speed and lower power without significant accuracy loss. In this paper, we propose using SC techniques to approximate multiplication operations on fixed-point weights and biases during training of CNNs. By employing such techniques, we demonstrate near state-of-the-art learning performance for the MNIST and CIFAR-10 datasets, while achieving significant resource and speed improvements when implementing the deep networks on a Field Programmable Gate Array (FPGA). For MNIST, we demonstrate that SC compared to conventional computing, will result in almost 3 times increase in learning speed with only 1.37% degradation in validation accuracy. Similarly, for CIFAR-10, training is accelerated 3.5 times with a degradation of 3.39%. We also show that our FPGA implementations of CNNs adopting stochastic multipliers consume over 17 times less power than their GPU counterparts.