A 0.23mW Heterogeneous Deep-Learning Processor Supporting Dynamic Execution of Conditional Neural Networks

A deep-learning processor is presented for achieving ultra-low-power operation in mobile applications. Using a heterogeneous architecture that includes a low-power always-on front-end and a selectively-enabled high-performance backend, the processor dynamically adjusts computational resources at runtime to support conditional execution in neural networks and meet performance targets with increased energy efficiency. Featuring a reconfigurable datapath and a memory architecture optimized for energy efficiency, the processor supports multilevel dynamic activation of neural network segments, performing object detection tasks with 5.3× lower energy consumption in comparison with a static baseline design. Fabricated in 40nm CMOS, the processor test-chip dissipates 0.23m W at 5.3 fps. It demonstrates energy scalability up to 28.6 TOPS/W and can be configured to run a variety of workloads, including severely-power-constrained ones such as always-on monitoring in mobile applications.