PEDSA: High-Throughput Pipeline-Based FPGA Accelerator for Convolutional Encoder–Decoder Segmentation Networks

In the era of artificial intelligence (AI), rapidly growing data and computing demands stimulate a shift toward more intelligent processing at the edge in the Internet of Things (IoT). AI application scenarios, such as image segmentation, pose new challenges to the computing capability of edge hardware, which cannot be solved by traditional AI accelerators with the traditional processing element (PE) architecture. Recently, the streaming architecture has received more attention due to its higher performance. To improve the throughput of edge platforms for segmentation tasks, customizing streaming accelerators for segmentation models is now necessary. Based on these motivations, we proposed pipelined encoder-decoder segmentation model accelerator (PEDSA), a fully pipelined streaming accelerator for convolutional encoder-decoder segmentation networks. PEDSA maps all the layers in the network into a pixel-level pipeline. All operations, especially upsampling (unpooling, deconvolution, etc.) which involves complex data rearrangement, are integrated into a regular, concise, and fully on-chip data flow. On Xilinx field-programmable gate arrays (FPGAs), our acceleration of SegNet-Basic and U-Net reached performances of 2676.47 and 7646.29 GOPS, respectively, outperforming previous accelerators for this kind of network. This work provides new ideas for the deployment of segmentation algorithms at the edge.