Optimization Techniques for Conversion of Quantization Aware Trained Deep Neural Networks to Lightweight Spiking Neural Networks

In this paper, we present spiking neural network (SNN) conversion technique optimized for converting low bit-width artificial neural networks (ANN) trained with quantization aware training (QAT). Conventional conversion technique suffers significant accuracy drop on QAT ANNs due to different activation function used for QAT ANNs. To minimize such accuracy drop of the conventional conversion, the proposed technique uses Spike-Norm Skip, which selectively applies threshold balancing. In addition, subtraction based reset is used to further reduce accuracy degradation. The proposed conversion technique achieves an accuracy of 89.92% (0.68% drop) with a 5-bit weight on CIFAR-10 using VGG-16.