Towards Efficient On-Chip Learning using Equilibrium Propagation

With the growing research and application of deep learning, there are increasing demands in the ability to re-train or improve models with new data in the field. The popular backpropagation algorithm are very effective when training large models offline, however, it requires considerable computational resources. Equilibrium Propagation is an energy based learning algorithm for neural networks proposed as an alternative to the traditional back propagation algorithm. With the forward and backward phase using almost the same computation, the algorithm is an interesting candidate for implementing on-chip learning. As a first step towards building the hardware, we apply quantization on the algorithm to study the feasibility of a digital implementation. We then introduce a hardware oriented network pruning method to reduce the number of computations and the memory usage by a factor of 2.7. This paper lays the foundation for the implementation of Equilibrium Propagation on digital hardware, and an provides alternative angle to the problem of on-chip learning.