Paddle-Mlir: A Compiler Based on MLIR for Accelerating PaddlePaddle on Edge Intelligence Devices

AI accelerators provide a promising influence for the deployment of deep learning model on edge devices, and an effective compiler can reduce great workload when deploying the models. Multi-level intermediate representations (MLIR) framework offers a general and efficient solution for compiling and deploying deep learning models on AI hardware because of its reusable and extensible compiler infrastructure. This work utilizes the MLIR framework to compile PaddlePaddle models, and then uses the back-end compiler to generate the target machine codes (bmodel) which are then deployed on an AI hardware--BM1684X. The experimental results show that the compilation of the YOLOv5s model from the PaddlePaddle framework into the MLIR file taking 4.636 seconds without any loss of accuracy, whose time cost is 20% less than compiling the YOLOv5s model from ONNX. The bmodel file, using the FP16 strategy, is executed on the BM1684X processor within 6.508 milliseconds, from loading the model's inputs to outputs. Moreover, employing model in local non-quantization strategy can reduce the entire computing time by 40% with a loss of 1 % accuracy.