A Resource-Sharing & Pipelined Design Scheme for Dynamic Deployment of CNNs on FPGAs

In this paper, we present a new design scheme for dynamic deployment of CNNs on FPGAs, to adaptively implement large CNN models on resource limited FPGAs while keeping low latency and high performance. We first propose a FPGA-based pipeline model named Resource-Sharing Pipeline in the design scheme. The novel pipeline model dynamically reconfigures pipeline stages to FPGA so that FPGA resources are shared by multiple pipeline stages at the different time. The computing parallelism of convolutional layers and the memory access efficiency are also optimized in the design scheme for performance and latency enhancement. In experiment we map all convolutional layers of VGG-16 to Xilinx VC709 platform using our new design scheme. The results show that the implementation reaches a performance of a 187.0ms latency and an 820.8GOP/s throughput under 100MHz clock frequency, achieving a promising promotion over previous works.