Effective Application-level Error Modeling of Permanent Faults on AI Accelerators

The deployment of Machine Learning (ML) applications extensively leverages Matrix Multiplication (MM) operations on modern and advanced accelerators, like Graphic Processing Units (GPUs), which employ Tensor Core Units (TCUs) to optimize MM’s execution efficiently. However, reliability concerns arise in devices with cutting-edge semiconductor technologies (7 nm or less), as faults can compromise some structures (e.g., TCUs) during their operation. In safety-critical applications, this can lead to wrong DNN outcomes and cause unpredictable and unacceptable actions. Thus, the impact evaluation of such faults is crucial to ensure that TCUs and GPUs meet the safety standard requirements (e.g., ISO26262). Currently, the reliability assessment of complex applications concerning hardware faults involves fault injection (FI) campaigns. Unfortunately, low-level FI campaigns might be computationally prohibitive for GPUs when these execute massive applications like DNNs. In this work, we propose an error modeling approach to accurately describe corruptions from permanent faults on TCUs, during the operation of MMs. This approach enables realistic reliability evaluations of computationally expensive MM-based workloads, resulting in a huge acceleration (up to 225X) compared with hardware-level FIs. Our experimental results show a very good accuracy (up to $93 \%$ correlation between our error modeling approach and FI campaigns conducted on TCUs).