Circuits to Systems: Co-Designing Efficient AI Hardware

The evolution of artificial intelligence (AI) has been fundamentally enabled by the co-development of circuit-level innovation and system-level architecture. From early neuromorphic circuits that mimicked biological neurons using CMOS and VLSI techniques, to modern accelerators that integrate memory-centric computing with algorithm-hardware co-design, the foundation of AI hardware continues to expand. This paper offers a layered perspective on the past, present, and future of AI systems. We begin by reviewing the development of neuromorphic circuits, including Processing-in-Memory (PIM) and Content-Addressable Memory (CAM), which address the memory-compute bottleneck. We then examine key milestones in system co-design, such as GPUs and systolic arrays for deep learning, and explore how sparsity and quantization have driven algorithm-architecture synergy. Finally, we highlight emerging challenges posed by large language models (LLMs), including memory bandwidth, sequential decoding, and low-bit quantization, and discuss future directions for circuit and system co-design in the era of foundation models. This work aims to inform and inspire next-generation research in efficient and scalable AI hardware.