Fault tolerance via weight noise in analog VLSI implementations of MLPs-a case study with EPSILON

Training with weight noise has been shown to be an effective means of improving the fault tolerance of multilayer perceptrons (MLPs). This paper investigates the use of weight noise used during MLP training to compensate for the inherent errors encountered in VLSI implementations. Weight adaptation is conducted solely in software, eliminating the need for costly in-the-loop training. The particular VLSI implementation considered here is the EPSILON processor card developed at Edinburgh University. Both software and hardware experiments demonstrate the effectiveness of this approach. This case study with the EPSILON processor card highlights what we believe to be a number of common inadequacies with custom designed hardware. In particular, the limitations of EPSILON in terms of its dynamic range performance has been shown to be a problem. In summary, we show that networks trained with weight noise are fault-tolerant but also require an increased dynamic range to exploit this property.