An Energy-Efficient Embedded Deep Neural Network Processor for High Speed Visual Attention in Mobile Vision Recognition SoC

An energy-efficient Deep Neural Network (DNN) processor is proposed for high-speed Visual Attention (VA) engine in a mobile vision recognition SoC. The proposed embedded DNN (E-DNN) processor realizes VA to rapidly find Region-Of-Interest (ROI) tiles of potential target objects to reduce ~ 70% of recognition workloads of a vision recognition SoC. Compared to the previous scale-invariant feature transform (SIFT) based VA models, the proposed E-DNN VA model reduces the execution time by ~ 90%, which results in 73.4% reduction of the overall object recognition (OR) processing time. Also, the proposed E-DNN VA model shows ~4% higher OR accuracy for 113-object database (13 laboratory object database + COIL-100 objects database) than the previous model shows. Highly-parallel 200-way PEs are implemented in the E-DNN processor with 2D-axis layer sliding architecture, and only ~3 ms of the E-DNN VA latency can be obtained. In addition, the dual-mode configurable PE architecture is proposed to support both Convolution Neural Network (CNN) and Multi-Layer Perceptron (MLP) by utilizing the same hardware resources for high energy efficiency. As a result, the implemented E-DNN processor achieves only 1.9 nJ/pixel energy efficiency which is 7.7× smaller than the state-of-the-art VA accelerator which showed 14.6 nJ/pixel energy efficiency.