A Data Flow Scheduler for Multi-precision SIMD Processing Engine Systolic Array CNN Accelerators

Conventional inference data-flow techniques for DNN accelerators often involve layer-wise mapping, resulting in frequent off-chip memory accesses. To address this issue, an efficient scheduling mechanism is proposed for a systolic array architecture with a multi-precision fixed-point Processing Engine (PE) featuring SIMD capabilities. By restructuring input feature maps and kernel data, the approach optimizes data reuse for both on-chip and off-chip memory accesses. The Channel-First scheduling mechanism significantly reduces DMA (memory read) overheads, enhancing efficiency. Analysis shows memory read overhead reductions of 3×, 6×, and 62× for input feature maps of LeNet-5, AlexNet, and VGG16 models, respectively. For kernel weights, reductions of 10×, 214×, and 371× are achieved. Software emulation of SIMD-enabled PEs across different precisions (8-bit, 16-bit) shows minimal accuracy loss: less than 1% and 2% for LeNet on MNIST and AlexNet, VGG16 on CIFAR-10 respectively, in comparison with Float32 precision.