Fault tolerance in neural networks: Neural design and hardware implementation

This paper provides a review of fault tolerance in artificial neural networks from a neural design perspective towards hardware implementations. We claim that an integrative outlook of these fields can benefit research to obtain truly fault tolerant physical neural networks that implement parallel and distributed neural computing and the associated learning algorithms in hardware, and/or in the quest to find fault-tolerant computing solutions rooted in the neural paradigm. As the underlying semiconductor technologies are getting less and less reliable, the probability that some components of computing devices fail also increases, preventing designers from realizing the full potential benefits of on-chip exascale integration. Neural computing principles remain elusive, yet they stand as source of a promising fault-tolerant computing paradigm. We present main concepts, and a taxonomy of well established techniques to exploit and improve, by design, the intrinsic but limited fault tolerance of neural networks, and we discuss fault tolerance of neural network hardware implementations targeted to FPGA devices.