Confidence Driven Compact Testing of Compute-in-Memory Based Language Models

Language models (LM) have revolutionized natural language processing (NLP) tasks such as question answering and code generation. However, escalating energy costs have accelerated the use of analog compute-in-memory (CiM) based architectures for energy-efficient language model inference. A key barrier to widespread adoption of such architectures is posed by device-to-device and cycle-to-cycle conductance variations within analog crossbars of CiM based systems that degrade text generation quality. To ensure reliable operation of analog CiM-based LMs (also referred to as device under tests or DUTs), fast and accurate functional testing of CiM structures is necessary post-manufacture. We first assess the impact of conductance variations within crossbars on a LM’s log-perplexity, an evaluation metric for the quality of text generation. To derive a functional test, we propose confidence driven compact testing of CiM-based LMs. In this framework, we down-select text sequences from a benchmarking dataset to create a compact sequence set. During testing, we iteratively apply sequences from the compact sequence set to the CiM based LM and estimate a confidence interval of the DUT’s log-perplexity. Depending on the log-perplexity confidence interval (LPCI) and a predefined performance (log-perplexity) threshold, a DUT is classified as "good" or "bad". The combination of confidence driven testing and sequence down-selection achieves 14.8-272x speedup compared to exhaustive testing while maintaining 0% critical test escape for GPT-2.