Approximating Nonlinear Activation Function Using Genetic Programming

Modern advancements in hardware implementation of Deep Neural Networks (DNNs) have underscored the need for efficient and accurate nonlinear activation function approximations to reduce computational complexity in resource-constrained environments like FPGA. Traditional methods face challenges in achieving an optimal balance between accuracy and hardware efficiency. To address this, we propose PSO-GP, an automated framework that formulates the approximation as an optimization problem. In this framework, Genetic Programming (GP) approximates complex functions in linear functions, while Particle Swarm Optimization (PSO) identifies optimal interval partition to minimize error. The experimental results demonstrate that PSO-GP significantly improves accuracy and reduces hardware overhead compared to state-of-the-art methods.