Design Considerations for Edge Neural Network Accelerators: An Industry Perspective

AI acceleration is one of the most actively researched fields in IP and system design. The introduction of specialized AI accelerators in the cloud and at the edge has made it possible to deploy large scaled AI solutions that automate tasks that were previously not possible without AI. The growth of big data and the compute horsepower needed to process this data to provide business intelligence is key for several companies in gaining a competitive edge. AI workloads are both data and compute intensive and improving the efficiency often requires an end-to-end solution. In this perspective paper, we identify key considerations for the design of AI accelerators. The focus of this paper is on Deep Neural Networks (DNN) and how to build efficient deep neural network accelerators through microarchitectural exploration, energy efficient memory hierarchies, flexible dataflow distribution, domain-specific compute optimizations and finally hardware-software co-design techniques. The importance of interconnect topology and the impact of its scaling to the physical design of an AI accelerator is also a key consideration that is described in this paper. In the future, the energy efficiency of these accelerators may rely on approximation computing, compute-in-memory and runtime flexibility for significant improvement.