Tiny neural network search and implementation for embedded FPGA: a software-hardware co-design approach

Embedded FPGA has been widely adopted as a hardware platform for implementing artificial intelligence (AI) systems on the edge due to its flexibility, versatility and efficiency. In general, there are three steps to deploy edge AI algorithms on embedded FPGAs (Fig. 1a): (a) manually train a neural network (NN) model for a specific dataset; (b) prune and quantize the NN model; (c) design a hardware accelerator architecture to implement the NN model. Such an approach, however, usually results in inferior local-optimal solutions because of the mutual influence between the algorithm and hardware. Recently, neural architecture search (NAS) has been intensively exploited to automatically identify an optimal NN architecture on given constraints [1–6]. In particular, hardware-aware NAS (HW-NAS), a NAS strategy that takes hardware constraints (e.g., on-chip memory and DSP) and performance requirements (e.g., FPS and energy efficiency etc.) into consideration (Fig. 1b), has drawn great interest in finding an optimal NN architecture to maximize the algorithm accuracy and hardware efficiency. Nevertheless, current HW-NAS designs, mainly for general AI scenarios, generally have complicated search space (e.g., oversized NN model) and are unaffordable for edge AI devices (limited resources and energy).