Customized Instruction on RISC-V for Winograd-Based Convolution Acceleration

Convolution operation accounts for the major work-load in convolutional neural networks (CNN). However, standard instruction set for RISC-V processor cannot efficiently perform the matrix convolution between kernel and input matrices. In this paper, we construct a custom instruction under the RISC-V ISA that can perform the F(2×2,3×3) convolution within one single execution. Particularly, optimized by the Winograd algorithm, the operation only needs 16 multiplications instead of 36 multiplications as needed by standard ISA. Benefit from this cycles, as compared to 140 cycles using standard instructions. Thenew instruction, F(2×2,3×3) can be calculated within 19 clock power consumed during convolution operation is also reduced significantly.