A 4K Vision Computing Platform with Convolutional Neural Network Engine on FPGA

This paper presents the design of a smart vision computing platform with convolutional neural network accelerator implemented on FPGA. Vision signals are first captured by image sensor and processed in image signal processing (ISP) which includes domain conversion, 3D noise reduction, color enhancement, etc. A realtime H.265/HEVC main profile video encoder is proposed to process up to 4096×2160@30fps 4:2:0 encoding with five-stage pipeline architecture. A computer vision engine (CVE) is presented to offload the CPU for tasks such as moving and intrusion detection, which is able to achieve 1920×1080@30fps performance. To cope with more complex vision tasks such as object detection for autonomous driving, we also designed a convolutional neural network engine (CNNE) for realtime object detection and recognition. The CNNE is implemented on Xilinx FPGA with overall throughput of 1TOPS when running at 250MHz.