Approximate Swish Activation Function for Low-Energy Yet Low-Error VLSI Implementations

This paper presents a hardware-friendly approximation of the Swish activation function and its VLSI implementation. The proposed design relies on approximating Swish with a piecewise linear function purposely devised to operate only on power-of-two factors and to increase the reuse of computational paths between different segments. This choice allows replacing complex exponentiation, division and multiplication units with simpler shifting and addition operations. Experimental results demonstrate that, when implemented in a 28 nm FDSOI technology, our 16-bit design saves more than $65 \%$ and $57 \%$ of area and power, respectively, with respect to the best competitor. This outcome is achieved with a Top-1 accuracy degradation of just $0.31 \%$ on the CIFAR100 dataset using the VGG-BC model.