NN-ECC: Embedding Error Correction Codes in Neural Network Weight Memories using Multi-task Learning

Neural networks (NNs) have shown outstanding performance in various domains, leading to widespread deployment on various hardware devices. They require large memories to store the NN weight parameters, which are susceptible to numerous permanent and transient faults. Therefore, error detection and correction mechanisms with certain guarantees should be provided to ensure reliable NN operation, especially in safety-critical applications. Error Correction Codes (ECCs) are a common approach to protecting memories against these failures, but they impose significant memory overheads. This work proposes NN-ECC, a multi-task learning objective that integrates linear block ECC into the NN parameters during training. The proposed NN-ECC does not increase the total number of NN parameters and eliminates all storage requirements for parity bits of different ECCs. Unlike existing methods, which selectively eliminate the storage overhead with limited error correction and necessitate specific weight distributions to utilize redundant weight bits for ECC parity, our approach is versatile and can accommodate various ECC schemes with different correction capabilities without imposing constraints on weight distributions. Moreover, the proposed NN-ECC does not deteriorate the baseline accuracy due to ECC encoding.