A Hybrid In-Memory-Searching and In-Memory-Computing Architecture for NVM Based AI Accelerator

A hybrid architecture combining in-memory-searching (IMS) and in-memory-computing (IMC) is proposed to accelerate neural network inference computation. The high speed NOR-flash IMS unit, with large unit cell discharge time ratio (>10
⁶
) and fast sensing time (<1ns), provides coarse filtering function to reduce IMC loading. The 3D-NAND-flash IMC unit for multiplication-accumulation (MAC) operation calculates the likeliness scores between the inquiry and the IMS selected instances in the NAND array. With IMS’ coarse filtering function the efficiency for the IMC scoring function can be improved up to 1000X in our demonstration. Adding augmented instances to the abundant IMC unit can further improve the accuracy and the reliability of the system. This is enabled by enhancing the tolerance to inquiry variation and memory fail bits.