A-Connect: An Ex Situ Training Methodology to Mitigate Stochasticity in Neural Network Analog Accelerators

Analog-based neural network accelerators outperform digital-based accelerators in energy efficiency by trading accuracy. Analog computation is susceptible to hardware stochastic variability, incurring in limited signal-to-noise and aggravating for compact and low power applications. Here we introduce A-Connect, an ex situ statistical methodology to improve analog neural network resilience against stochastic variability. Our methodology achieves state-of-art performance in analog environments with heavy stochasticity levels by injecting noise during the neural network forward propagation and considering the same injected noise during the backward propagation. Furthermore, we developed a Keras/Tensorflow library with fully-connected and convolutional layers versions using our training methodology, which can be coupled easily to standard machine learning platforms. We present simulation results applying the A-Connect methodology to popular DNN models, like LeNet-5 for MNIST dataset, AlexNet, VGG-16, and ResNet-20 for the CIFAR-10 dataset, and ResNet-18 for CIFAR-100 dataset. When validating the CIFAR-10 or CIFAR-100 recognition tasks, the results with the A-Connect methodology showed an improvement over the baseline model of around 15 to 68 percentage points for the median accuracy at a 70% of stochastic variability. The deviation of the results with A-Connect is around 20X lower than the baseline at this level of stochasticity. A-Connect also showed the best performance when compared to other ex situ approaches, while having comparable results to in situ, and hybrid (i.e., using ex situ and in situ approaches) methods in the literature. We anticipate that the A-Connect methodology could enable emergent memory technologies, such as ReRAM and PCM, for accurate computation-in-memory applications.