NeuroSim Agent: Automated Compute-In-Memory Accelerator Deployment with Transferable Reinforcement Learning and Dynamic Design Space Pruning

Compute-In-Memory (CIM) is a compelling candidate for accelerating deep neural network (DNN) applications by significantly reducing data transfer overhead between memory and compute units. However, designing an optimal CIM architecture that satisfies hardware constraints while maximizing performance typically requires extensive design space exploration (DSE) involving thousands of simulations before tape-out. Reinforcement learning (RL) offers a powerful alternative by automating the exploration process and identifying optimal hardware configurations. Nevertheless, two key challenges hinder the adoption of RL-based approaches in CIM design: (1) long interaction time with simulators, especially for complex DNN workloads, and (2) high retraining costs when adapting to changes in technology nodes or workloads. This work addresses these challenges and proposes an agentic version of the NeuroSim tool, essentially an RL-based CIM design agent with (1) feature transfer from the pretrained base model and (2) dynamic design space pruning informed by current and prior experience. In the case studies on performance optimization for ResNet-18 and DenseNet-40 under 22nm technology with emerging non-volatile memories, the proposed CIM agent shows 2.2x to 7.3x average runtime speedup over brute-force search, with negligible performance degradation (<0.1%). Furthermore, when transitioning from 22nm to 14nm technology, our method achieves a 3.2× average runtime reduction for throughput optimization on ResNet-18 compared to brute-force search.