Application Aware Tuning of Reconfigurable Multi-Layer Perceptron Architectures

Production FPGA implementations of Multi-Layer Perceptron (MLP) inference typically address the growing performance demands by, (i) to address memory boundedness, storing neuron weights on-chip, e.g., Microsoft Brainwave, and, (ii) to address compute boundedness, generating the largest possible arrays of multipliers and accumulators. These approaches of maximizing device utilization, irrespective of application model, can actually result in higher latencies due to the tight coupling of different function modules. For example, generic parameter sizing of a given component can force an increase in complexity and latency of multiple other modules in the design. In real-time applications, this can result in the overall computation failing to make a deadline. In our work we begin by creating a testbed for low-latency MLP inference, which we then use to explore the application-aware optimization space for compute-bound MLP inference engines. The optimization process begins by identifying modules in the critical path and their connectivity. We then use this information to determine key parameters and their ideal values. In addition, we automate hardware generation using OpenCL to ensure standard optimizations are applied. We find that correct parameter sizing can reduce latency by 20% on average. For the MNIST, Poker, and ECP-Candle benchmarks, we implement inference models using the Arria10X115 FPGA and achieve an average speedup of 1.47x over the NVIDIA Tesla P100 GPU.