Improving tolerance of neural networks against multi-node open fault

Neural networks are not intrinsically fault tolerant and their fault tolerance has to be improved by employing extra mechanisms. During the last decades, some simple fault types of feedforward neural networks have been widely investigated. In this paper, a rather complicated fault type, i.e. a multi-node open fault where several hidden nodes are out of work at the same time, is formally analyzed, and an approach named T3 is proposed. The ground of T3 is the recognition that the performance of trained neural networks does not linearly decrease with the increasing of the severity of fault. T3 utilizes a validation set to build the fault curve of a trained network. It then locates the inflection point of the fault curve and repeatedly trains the network according to the corresponding fault rate so that the redundancy are appended to the network appropriately. Experiments show that T3 can improve the tolerance against multi-node open fault of some feedforward neural networks at the expense of relatively small redundancy.