ProTEA: Programmable Transformer Encoder Acceleration on FPGA

Transformer neural networks (TNN) have been widely utilized on a diverse range of applications, including natural language processing (NLP), machine translation, and computer vision (CV). Their widespread adoption has been primarily driven by the exceptional performance of their multi-head self-attention block used to extract key features from sequential data. The multi-head self-attention block is followed by feedforward neural networks, which play a crucial role in introducing non-linearity to assist the model in learning complex patterns. Despite the popularity of TNNs, there has been limited numbers of hardware accelerators targeting these two critical blocks. Most prior works have concentrated on sparse architectures that are not flexible for popular TNN variants. This paper introduces ProTEA, a runtime programmable accelerator tailored for the dense computations of most of state-of-the-art transformer encoders. ProTEA is designed to reduce latency by maximizing parallelism. We introduce an efficient tiling of large matrices that can distribute memory and computing resources across different hardware components within the FPGA. We provide run time evaluations of ProTEA on a Xilinx Alveo U55C high-performance data center accelerator card. Experimental results demonstrate that ProTEA can host a wide range of popular transformer networks and achieve near optimal performance with a tile size of 64 in the multi-head self-attention block and 6 in the feedforward networks block when configured with 8 parallel attention heads, 12 layers, and an embedding dimension of 768 on the U55C. Comparative results are provided showing ProTEA is 2.5× faster than an NVIDIA Titan XP GPU. Results also show that it achieves 1.3 – 2.8× speed up compared with current state-of-the-art custom designed FPGA accelerators.