Efficient 8-bit Matrix Multiplication on Intel Agilex-5 FPGAs

Matrix multiplication is a fundamental operation in many fields including artificial intelligence and machine learning, and it often requires significant computational resources. FPGAs have always been a great platform for accelerating such calculations thanks to their inherent parallelism and flexibility regarding data movement. The newly released Agilex-5 FPGA devices introduce several AI-specific hardware features that complement the traditional DSP Block functionality. The fixed-point Tensor Mode of the DSP Block exposes twenty 8-bit signed multipliers organized into two 10-element dot products, having one set of inputs fed from internal DSP Block registers. In this paper, based on the philosophy of efficiently utilizing the low-level features of the new DSP Block, we make use of these structures and construct a flexible 8-bit matrix multiplication engine. The generic matrix-multiplication architecture presented here achieves at steady-state 100% compute resource utilization (no idle states). In the 810-DSP configuration, the engine achieves over 750MHz on an Agilex-5 (fastest speedgrade) device with a throughput of 24.75TOPs and an energy efficiency of 1.65TOPs/W.