CINELL: An Energy-Efficient Compute-In/Near-Memory eDRAM Processor for Sparse Transformer-Based Large Language Models

Large language models (LLMs) based on transformer architecture have significantly advanced various AI applications. However, the multihead self-attention (MHSA) in a transformer requires extensive computation and memory bandwidth. Techniques such as sparse attention and attention formula reordering have been explored recently to mitigate these challenges. However, existing processors still struggle to process LLMs efficiently. To overcome these limitations, an energy-efficient compute-in/near-memory (CINM) embedded dynamic random access memory (eDRAM) processor CINELL is proposed, incorporating the following three key features: 1) an attention block fusion computation (ABFC), which maximizes data reuse in the attention map, reduces external memory access (EMA) by 85.86%, and improves hardware utilization to 86.1%; 2) a CINM architecture designed to address the imbalance between memory and computation, while the heterogeneous pipeline reduces the system latency by 77.27%; and 3) a compute-in-memory (CIM) array that supports the cross-read operation (CRO) addresses data direction conflicts, achieving a 98.44% reduction in latency, while dual-row computation (DRC) with reduced adder logic (RAL) improves energy efficiency by $1.58\times $ . Designed in 28-nm CMOS technology, the CINELL processor achieves 36.28–58.05 TOPS/W and an F1-score of 92.41% on the SQuAD 1.1v using the BigBird-large model.