Resource-saving Design Scheme Based on Look-Up Table for Activation Functions

In the field of deep learning, model quantization is an important technique aimed at reducing the storage requirements and computational complexity of models to enable efficient deployment on resource-constrained devices. However, traditional model quantization methods often face high computational demands when dealing with exponential operations, especially those involving activation functions. To address this issue, this paper proposes a LUT-based method for simplifying exponential operations. Look-Up Table (LUT) is a technique that precomputes and stores function values in a table for fast querying. In this paper, LUT is employed to approximate the exponential function, significantly alleviating the computational demands of exponential operations in models. To validate the effectiveness of this method, a series of experiments were conducted. The results demonstrate that using the proposed method for activation functions improves computational speed by 22% compared to using only pseudo-quantization for activation functions. Additionally, in practical applications, this method substantially reduces the L2 error during the quantization process, further demonstrating its superiority in model quantization.