Investigation of Effects of Non-Volatile Memory-Based Computation-in-Memory Non-Idealities and Model Size on Performance and Robustness of Small Language Model During Inference Phase

This paper comprehensively evaluates the effect of employing non-volatile memory (NVM)-based computation-in-memory (CiM) for small language model acceleration. The effect of memory non-idealities on the performance of the small language models, along with the similarity between CiM-based CNNs and language models, is documented with similarities between error tolerance of large language models and convolutional neural networks. The study also establishes a trend between the type of NVM non-idealities, model performance, model size, and noise tolerance.