A 1131-kb/mm2 14.0-to-53.3-TOPS/W 8-bit Analog-Assisted Digital Compute-in-Memory With Hybrid Local-Refresh eDRAM for Attention Computing

This article presents an eDRAM-based analog-assisted digital computing-in-memory (CIM) core that offers superior power and area efficiency for dynamic sparse attention computing. For different queries, it uses efficient low-precision analog pre-computing to filter out irrelevant in-memory key groups. As a result, each query only needs to attend to a small subset of the most relevant in-memory key groups in the digital domain, significantly boosting overall energy efficiency. At the cell level, a hybrid local-refresh eDRAM cell is proposed, which integrates analog, digital, and local refresh functions into a 4T1C cell, enabling the analog-assisted digital CIM to achieve high storage density. In addition, to reduce the cost of filtering out irrelevant in-memory key, a time-domain comparator that performs comparisons directly in the analog domain is adopted. For the digital operations, a near-memory time-shared multiply-accumulate (MAC) block design is employed, further enhancing the macro’s density. Fabricated in a 28-nm CMOS process, the prototype demonstrates a high storage density of 1131 kb/mm² and improved 8-bit digital-mode energy efficiencies from 14.0 to 53.3 TOPS/W.