A Hierarchical-Hybrid Floating-Point Compute-in-Memory Macro Using FP-DAC and FP-ADC for Edge-AI Devices

The floating-point (FP) format offers superior precision compared to its integer counterpart for multiply-and-accumulate (MAC) computations. With the rapid progress of artificial intelligence (AI), FP-MAC operations are increasingly moving from the cloud to edge devices. However, the complex nature of FP processes constrains their energy efficiency. Compute-in-memory (CIM) shows promise for reducing power consumption, but FP-CIM still encounters challenges related to mantissa and exponent computations. This article presents an energy-efficient FP-CIM macro designed for edge-AI devices. It addresses energy efficiency bottlenecks through three key features: 1) an FP digital-to-analog converter (FP-DAC) simplifies the pre-alignment logic, thereby reducing undesired power overhead throughout the FP process; 2) an FP analog-to-digital converter (FP-ADC) that substitutes uniform quantization steps with FP steps, eliminating redundant conversion power through adaptive quantization range (AQR); and 3) a hierarchical-hybrid structure (HHS) optimizes the tradeoff between accuracy and energy efficiency in FP-CIM using a coarse-to-fine hybrid strategy. Fabricated in a 65-nm CMOS process, the prototype FP-CIM macro achieves 54.4 TFLOPS/W in the BF16 mode and 130.9 TFLOPS/W in the FP8 mode.