A 28-nm Computing-in-Memory Processor With Zig-Zag Backbone-Systolic CIM and Block-/Self-Gating CAM for NN/Recommendation Applications

Computing-in-memory (CIM) chips have demonstrated promising energy efficiency for artificial intelligence (AI) applications such as neural networks (NNs), Transformer, and recommendation system (RecSys). However, several challenges still exist. First, a large gap between the macro and system-level CIM energy efficiency is observed. Second, several memory-dominate operations, such as embedding in RecSys and non-linear functions in NN/Transformer, require heavy off-chip access. To solve these challenges, this work presents an energy-efficient CIM-CAM processor for both NN and RecSys applications with three main innovations. First, we propose a reconfigurable Zig-Zag memory access structure, enabling flexible adjacent connections between SRAMs and CIM arrays to minimize SRAM access power. Second, a backbone-systolic (BKS) CIM array is implemented to avoid a long accumulation path with better data reuse between CIM macros. Third, a distribution-aware block-/self-gating (DABG) content-addressable-memory (CAM) is proposed to reduce off-chip access and on-chip search power for the memory-dominate embedding operations in RecSys. The fabricated 28-nm CIM-CAM chip achieves 37.9–81.5 TOPS/W for NN and 12.3–56.1 nJ/request for RecSys and highlights the improvement of system-to-macro efficiency ratio (SMER) by 9.2% over the trendline of the state-of-the-art system-level CIM processors.