CIMAX-Compiler: An End-to-End ANN Compiler for Heterogeneous Computing-in-Memory Platform

As artificial neural network (ANN) being widely adopted in edge applications, such as voice recognition, and face detection, etc. Computing-In-Memory (CIM) accelerators have received much attention because they are faster and more energy efficient. However, existing compilers are mainly applicable for traditional backend devices rather than emerging hardware architectures like CIM accelerator. In this paper, we propose an end-to-end neural network compiler, named CIMAX-Compiler. It can convert an ANN model in Open Neural Network Exchange (ONNX) format to executable codes, which runs on a heterogeneous embedded system composed of an MCU and a CIM accelerator. CIMAX-Compiler greatly reduces model deployment effort from tens of engineer-hours to less than a second. In addition, we applied several optimization techniques such as operator fusion and convolution compression to further improve compiled code performance. Experimental results show that the optimized code can speed up ANN model inference performance by more than 2× compared with the base-line layer to layer implementation.