An Energy-Efficient CNN Accelerator for Multi-object Real-Time Semantic Segmentation in Autonomous Vehicle

An energy-efficient convolutional neural network (CNN) accelerator is proposed for real-time segmentation in autonomous electric vehicle (AEV) system. The computation of semantic segmentation with high-resolution images makes it difficult for real-time operation in time-critical and resource-constrained AEV. To facilitate real-time implementation in AEV, this paper proposes two key features: 1) A compressed multi-object Depth-fused Trilateral Network (DTN) with dilated convolution and depthwise separable convolution that reduces 90% of the overall computation of baseline [1] and achieves 94.73% accuracy on KITTI Road dataset; 2) An energy-efficient CNN accelerator, which supports 5 types of CONV’s, achieving 1.33× higher throughput than the previous processor [2]. Finally, the proposed processor is designed in 28 nm CMOS technology. It consumes 65.7 mW of power and achieves 2.91 TOPS/W of energy efficiency. As a result, the system realizes 72.2 and 37 frames-per-second of semantic segmentation for road and multi-objects with high resolution.