Selective Data Transfer from DRAMs for CNNs

Convolutional Neural Networks (CNN) have changed the direction of image and speech signal processing. They have become prolific in applications such as self driving cars and voice assistants like Siri and Alexa. Since the success of AlexNet, many deep learning networks like GoogleNet, ResidualNet etc have been introduced. These networks are highly competent in image classification however, they are very large in size and have a lot of parameters, for example, AlexNet has 60M parameters [1]. On-Off chip data transfer is a great engineering challenge that needs to be addressed while implementing CNN in hardware. Hardware Acceleration and parallelization are constrained by energy cost of reading and writing data from main memory. For one forward pass during inference, one image needs to go through to all the CNN layers and be classified into a softmax determined category. During this process, the memory bandwidth is used by three types of payloads that need to be moved on and off-chip - input image, intermediate feature maps, and filter weights. This research is focused on reducing weight-related memory traffic that occurs between off-chip memory and on-chip buffers during inference. In this paper, we propose a technique named `weight sharing selective transfer' (WS-ST) that uses processing in-memory architecture to selectively transfer weights from the DRAM memory structure to the computation unit. We emulate a PIM architecture by having a self populating FIFO as a part of Selective DRAM controller and a weight selector logic near the DRAM. The DRAM is modeled in the FPGA as a dual port static RAM in order to analyze the effectiveness of the weight selector. We observe a 30% decrease in memory transfer traffic compared to a non-selective approach for AlexNet in redundancy analysis and use the selective DRAM to implement it. The power saving of 2% of dynamic power as a result of the selective transfer are reported in the results.