ONNC-Based Software Development Platform for Configurable NVDLA Designs

With the proliferation of deep learning and the increasing pressure to deploy inference applications at the edge, many AI chip makers integrate the open source NVIDIA Deep Learning Accelerator (NVDLA) design in their AI solutions. Lack of open source compiler support and having only limited configurability support in the software stacks erect a barrier for developers to freely explore the NVDLA design space at system level. This paper presents an ONNC-based software development platform that includes the first open source compiler for NVDLA-based designs, a virtual platform with various CPU models as well as configurable NVDLA models, and auxiliary tools for debugging. The platform is tightly coupled with the hardware design tradeoffs and provides extendibility for compiler optimization, more CPU types, and more NVDLA hardware configurations. It lifts many restrictions of software development for those who like to leverage the NVDLA design in inference applications.