An Implementation of a 3D Image Filter for Motion Vector Generation on an FPGA Board

Object detection and tracking technology are essential in many ways with the increasing demand for remote video conferencing and video surveillance. This paper focuses on a method that combines an object detection method based on a CNN (Convolutional Neural Network) with a motion-vector-based object tracking method in specified frames instead of the whole frame stream. Although this method can improve the throughput with a small memory requirement, it requires 3D image filters to reduce the noise in each motion vector. The immense computing power needed for 3D image filters prevents this method from being popularly used. To address this problem, we propose to implement the technique on an FPGA and offload the filters and CNN engine on the FPGA hardwired logic. We implemented slide window-based image filters on a Zynq Ultrascale+ FPGA board and achieved 23.42 times performance improvement for the median filter and 55.46 times for the average filter, respectively. Considering the resource usage, it appeared that we could implement these 3D image filters design with YOLO on the target FPGA.