TRIM: A Design Space Exploration Model for Deep Neural Networks Inference and Training Accelerators

There is increasing demand for specialized hardware for training deep neural networks (DNNs), both in edge/IoT environments and in high-performance computing systems. The design space of such hardware is very large due to the wide range of processing architectures, DNN configurations, and dataflow options. This makes developing DNN processors quite complex, especially for training. We present TRIM, an infrastructure to help hardware architects explore the design space of DNN accelerators for both inference and training in the early design stages. The model evaluates at the whole network level, considering both interlayer and intralayer activities. Given applications, essential hardware specifications, and a design goal, TRIM can quickly explore different hardware design options, select the optimal dataflow and guide new hardware architecture design. We validated TRIM with FPGA-based implementation of DNN accelerators and ASIC-based architectures. We also show how to use TRIM to explore the design space through several case studies. TRIM is a powerful tool to help architects evaluate different hardware choices to develop efficient inference and training architecture design. Experimental results show that TRIM is a powerful tool for rapidly exploring the design space of DNN architectures for training and inference.