Structure of Deep Learning Inference Engines for Embedded Systems

For the last several years, various types of deep learning applications have been introduced. Most deep learning related research and development have been done on servers or PCs with GPUs. Recently there have been a number of moves to apply those applications to the industrial sector. When deep learning techniques are applied to actual targets, we can face some spatial and environmental constraints unlike the laboratory environment. In this paper, we describe requirements when deep learning applications run for embedded systems. We introduce our ongoing project on developing a deep learning framework for embedded systems, especially automotive vehicles. Generally, deep learning application development process can be divided to two steps: training a data model with a big data set and executing the data model with actual data. In our framework, we focus on the execution step. We try to design an inference engine to satisfy the operational requirements for embedded systems. We describe our design direction and the structure. We also show preliminary evaluation result.