Efficient frequency domain CNN algorithm

Deep Learning techniques like Convolutional Neural Networks (CNN) are getting popular for image classification with broad usage spanning across automotive, industrial, medicine, robotics etc. Typical CNN network consists of multiple layers of 2D convolutions, non-linearity, spatial pooling and fully connected layer, with 2D convolutions constituting more than 90% of total computations. The Fast Fourier Transform (FFT) based approach for convolution is promising in theory, but not used in practice due to growth in memory sizing of coefficients storage. The paper proposes new frequency domain algorithm which avoids memory size growth compared to traditional FFT based approach for performing 2D convolution. The proposed algorithm performs Fourier Transform (FT) of coefficients On-The-Fly (OTF) instead of offline calculation on PC. The proposed algorithm consists of expands, OTF-FT and pruning blocks that do efficient 2D convolution in the frequency domain. The proposed algorithm is compared with the FFT-based algorithm for the coefficient transformation. As per simulations, assuming typical network configuration parameters, the proposed algorithm is 4-8X faster compared to FFT based approach for the co-efficient transform.