Tutorial: Designing an Edge Inferencing Accelerator with High- Level Synthesis

Convolutional neural networks are computationally intensive. A single inference can require billions of multiply/accumulate operations. In the datacenter, where ample power, space, and cooling are available, high powered CPUs or GPUs can be used. However, in the embedded space, edge systems often do not have the power or compute capability to perform these calculations. Further, edge applications often have hard real-time requirements. This combination necessitates the use of hardware accelerators for performing inferencing calculations. A bespoke accelerator will deliver the highest performance and efficiency, outperforming configurable IP by an order of magnitude or more. But transforming an algorithm developed in a machine learning framework, often in Python, into silicon can be a daunting task. Traditional hardware design is performed at the register transfer level (RTL). When designing in RTL, hardware engineers describe every register, every operator, and every connection in the design. At this level of detail, it is impractical for the designer to explore significantly different implementation architectures. High-Level Synthesis (HLS) allows a hardware engineer to operate at a higher level of abstraction – specifically, at the algorithmic level – and use the facilities of the HLS tool to automate the conversion of the algorithm description into synthesizable RTL. This enables a broad set of architectures to be created and evaluated. And at the algorithmic level, the description of the hardware is much closer to the original Python, making the conversion faster, easier, and more natural.