Fault and Delay Tolerance in Deep Learning Framework under GPU

Deep Neural networks are considered to be robust to the variation of the circuit components and the interconnection faults. A faulty network would terminate normally but generate an erroneous output. The tolerance of these neural networks is dependent on various factors such as the fault model, the network size and the training method. The fault tolerance to a certain extent can be obtained by introducing redundancy in the design of the network. In this work, we have considered Convolutional Neural Network and designed a fault-tolerant system with soft errors as fault in the network and introduced Triple Modular Redundancy to detect and rectify the fault. We have implemented three different methods to detect and rectify the fault. The third method proved to be the efficient as it uses 2/3rd of the GPU resources and takes only 2/3rd of the GPU execution time to detect and rectify the fault. The exceptional delay due to memory read or write in the system especially in one or more threads degrades the performance of GPU. To overcome such delay we designed a delay tolerant framework that effectively makes use of the available cores in the GPU to speed up the computation. Thus irrespective of the delay caused, the system identifies the effective configuration of the kernel launch to minimize the delay. We have discussed five different scenarios and found that the efficient configuration proved to reduce the overall delay by 66 percent in the worst case and provides constant delay time in the best case.