Technology Aware Training in Memristive Neuromorphic Systems for Nonideal Synaptic Crossbars

The advances in the field of machine learning using neuromorphic systems have paved the pathway for extensive research on possibilities of hardware implementations of neural networks. Various memristive technologies such as oxide-based devices, spintronics, and phase change materials have been explored to implement the core functional units of neuromorphic systems, namely the synaptic network, and the neuronal functionality, in a fast and energy efficient manner. However, various nonidealities in the crossbar implementations of the synaptic arrays can significantly degrade performance of neural networks, and hence, impose restrictions on feasible crossbar sizes. In this paper, we build mathematical models of various nonidealities that occur in crossbar implementations such as source resistance, neuron resistance, and chip-to-chip device variations and analyze their impact on the classification accuracy of a fully connected network (FCN) and convolutional neural network (CNN) trained with Backpropagation algorithm. We show that a network trained under ideal conditions can suffer accuracy degradation as large as 59.84% for FCNs and 62.4% for CNNs when implemented on nonideal crossbars for relevant nonideality ranges. This severely constrains the sizes for crossbars. As a solution, we propose a technology aware training algorithm, which incorporates the mathematical models of the nonidealities in the backpropagation algorithm. We demonstrate that our proposed methodology achieves significant recovery of testing accuracy within 1.9% of the ideal accuracy for FCNs and 1.5% for CNNs. We further show that our proposed training algorithm can potentially allow the use of significantly larger crossbar arrays of sizes 784 × 500 for FCNs and 4096 × 512 for CNNs with a minor or no tradeoff in accuracy.