Accelerating the Inference Phase in Ternary Convolutional Neural Networks Using Configurable Processors

The need to automate tasks has led to state-of-the-art algorithms such as Convolutional Neural Networks (CNNs), which can learn representations from images. CNNs are computationally expensive. Hence, they are mostly implemented on Graphical Processing Units (GPU), which consume high power, a large area and have high cost. Due to these factors, it is difficult to implement CNNs in embedded systems where there are severe space and power constraints. Application Specific Instruction-Set Processors (ASIPs) provide an alternative to GPUs and are low power consuming devices. However, floating-point operations and storage are expensive and do not lend themselves well to an ASIP implementation. A more efficient approach to implement CNNs on ASIPs involves expressing the data and parameters with binary, ternary or fixed-point representations. In this paper, we discuss our implementation of the inference phase of ternary CNNs on an Xtensa LX6 ASIP and measure its performance for the MNIST dataset. The complex computations are parallelized with custom instructions developed using the TIE programming language. Compared to a sequential implementation, our results indicate a speed up of 4.7× with an added area of only 20%.