THEF: A Privacy-Preserving Framework for Transformer Inference leveraging HE and TEE

The transformer has profoundly driven the development of natural language processing and computer vision tasks. However, their widespread deployment has raised concerns about the potential leakage of data privacy. To address the issue, homomorphic encryption (HE) is employed as a secure technique to process user private inputs over ciphertexts. As HE only supports linear operations, non-linear functions are required to be approximated into polynomials, which leads to heavy communication and computation overhead. In this paper, we propose THEF (Tee-And-HE-based Framework), an efficient and secure inference framework for transformer inference, which uses HE to ensure data security and leverages Trusted Execution Environment (TEE) to reduce communication and computation cost. We also design a number of optimization methods, including expanding encoding matrix multiplication scheme, batch acceleration and adaptive scheme, to improve performances of the THEF. Finally, we compare the performances of THEF and the state-of-the-art methods. The experimental results demonstrate that THEF maintains a level of accuracy close to that of plaintext model, reduces end-to-end communication cost by 80% and achieves significant improvement in end-to-end run-time.