Comma: A Communication-Minimized Model-Architecture Framework for Efficient Convolution Acceleration

Convolutional neural networks (CNNs) are still indispensable nowadays, but they bring a dominantly large overhead of data communication. Current works mainly focus on prior off-chip or intuitive/heuristic access optimization and lack a theoretically holistic analysis. Therefore, this article proposes an overall on- and off-chip communication-minimized CNN acceleration framework. First, we derive the layer-wise off-chip communication analytic expressions (AEs) based on different data reuse strategies and give their lower bound (LB), reaching 6%–18% lower than that proposed in work CLB. For practical off-chip access, we achieve 22%–35% reduction compared with the adaptive access strategy of work Sense and 11% with the dataflow selection approach of work Versa. Second, on-chip communication AEs and LB are derived within the tile range, and overall on- and off-chip access analysis is conducted to obtain the unified energy LB (ELB). Guided by our theoretical model, we design window-primitive (WP) dataflow and an energy-efficient systolic-cross-line (SCL) CNN accelerator. Finally, a holistic framework, Comma, is proposed to streamline the workflow of model-architecture co-design. SCL is implemented in 65-nm technology and achieves $4.2\times $ pJ/Multiply and ACcumulation (MAC) energy reduction at $1.4\times $ less on-chip memory area compared with Eyeriss, and 1.4~ $1.9\times $ reduction at 3~ $4\times $ less area compared with work CLB; as for systolic architecture, Sense, it achieves 29%–44% energy reduction with 4% area overhead. For the near memory processing (NMP) situation, we reduce around $2\times $ access energy compared with the systolic NMP architecture.