Implementation of RRAM based Swish Activation Function and its Derivative on 28nm FD-SOI

We propose a novel analog implementation of RRAM based swish activation function and its derivative circuit for inference and training of neuromorphic architectures. The swish activation function takes advantage from the sigmoid and ReLU non-linear functions for attaining maximum accuracy and speed for a neural network. This paper introduces the circuit design details of the swish activation function and its derivative implemented using RRAM devices and passive components implemented on 28nm FD-SOI (Fully Depleted Silicon on Insulator) technology from ST Microelectronics. The total power consumed by the swish and its derivative circuit is 376μW and 535μW respectively for the device operating between 0.5V-0.7V carried out at 27°C. The approximate area is 100.8μm
²
and 288μm
²
for swish and its derivative circuit. The output results attained for both the circuits have better characteristics and high performance as compared to other memristor based designs which uses bulk CMOS process. The device on/off ratio using RRAMs turns out to be 6.42 and 4.87 which is good as compared to other memristor based implementations. The circuit design and simulation were accomplished using STM 28nm FD-SOI technology on cadence tool. The swish activation and derivative circuit can be applied for training of neural circuits in order to evaluate the learning ability of an artificial neural network which can later be implemented as neuro based SoCs.