Caffe Barista: Brewing Caffe with FPGAs in the Training Loop

As the complexity of deep learning (DL) modelsincreases, their compute requirements increase accordingly. De-ploying a Convolutional Neural Network (CNN) involves twophases: training and inference. With the inference task typicallytaking place on resource-constrained devices, a lot of research hasexplored the field of low-power inference on custom hardwareaccelerators. On the other hand, training is both more compute-and memory-intensive and is primarily performed on power-hungry GPUs in large-scale data centres. CNN training onFPGAs is a nascent field of research. This is primarily due tothe lack of tools to easily prototype and deploy various hardwareand/or algorithmic techniques for power-efficient CNN training. This work presentsBarista, an automated toolflow that providesseamless integration of FPGAs into the training of CNNs withinthe popular deep learning framework Caffe. To the best of ourknowledge, this is the only tool that allows for such versatile andrapid deployment of hardware and algorithms for the FPGA-based training of CNNs, providing the necessary infrastructurefor further research and development.