A Low-Latency Feed-Forward Architecture for Image Filtering via Row-by-Row Processing

This paper presents a novel accelerator architecture for real-time image filtering applications that require high throughput and low latency. The proposed architecture consists of two systolic arrays: a Row Convolution (RConv) Array and a Column Convolution (CConv) Array. The RConv Array processes one row of the input image and computes one row of an intermediate image per clock cycle which is then input to CConv Array. The CConv Array computes one row of the output image per clock cycle. A novel aspect of the architecture is that the line delays within the convolution architecture are used efficiently due to the parallelism. This leads to a significant reduction in the memory requirement and latency of the architecture. While the row convolution in the RConv Array is implemented using a direct-form FIR filter, the column convolution in the CConv Array is implemented using the transpose-form FIR filter. Thus, the processing elements in the CConv array are implicitly pipelined and do not require additional pipelining delays. The parallelized row-by-row processing significantly reduces the memory overhead per output pixel. The proposed architecture is then generalized where multiple rows of the image can be processed in a clock cycle, leading to further proportional reduction in latency and memory requirements. The proposed architecture is fully feed-forward and can be pipelined further as needed. We show that using a Xilinx Virtex-7 2000T FPGA clocked at 100 MHz, the architecture achieves $16.0\times $ , $86.3\times $ , and $943\times $ improvements in the LUT-time2, FF-time2, and DSP-time2 products over a baseline serial architecture for the non-separable case with Image size $128\times 128$ and filter size $11\times 11$ . The proposed architecture can operate at a throughput of 12.8 Gpixels/s whereas the baseline can only operate at 100 Mpixels/s.