Hardware Acceleration Implementation of Three-Dimensional Convolutional Neural Network on Vector Digital Signal Processors

Convolutional neural networks (CNNs) have achieved great success in the field of computer vision. Researchers are currently focusing on more complicated three-dimensional (3D) CNNs and applying them to video processing. However, 3D CNNs have a huge number of calculations and parameters, which must be operated on an efficient computing platform. Digital signal processors (DSPs) are widely used in mobile terminals, high-performance workstations, and supercomputers. As a programmable computing platform, the DSP contains numerous multiplier-accumulator units (MACs) and supports vectorized calculation methods, which is very conducive to the efficient implementation of CNNs. To this end, it is of great significance to study the implementation of CNNs on a DSP. However, the current research on CNN acceleration on a DSP is still insufficient. This paper uses the FT-Matrix as the target DSP platform, which is a vector single instruction/multiple data (SIMD) architecture, and proposes a complete 3D CNN vectorized mapping method on the FT-Matrix, a block access scheme, and a hybrid convolutional layer optimization mapping method based on a sliding window and 3D Winograd algorithm. Our method can also be extended to other DSP platforms. Experimental results show that the algorithm can be executed accurately and efficiently on the DSP.