Investigating Robustness of Trainable Activation Functions for End-to-end Deep Learning Model in Autonomous Vehicles

This paper investigates the utilization of trainable activation functions (TAF) in vision-based autonomous vehicles, focusing on regression tasks. Activation functions play a crucial role in neural networks, introducing nonlinearity essential for learning abstract features through nonlinear transformations. While standard activation functions (SAF) like ReLU and ELU have demonstrated effectiveness, they often lack adaptability to diverse domains. This work investigates the robustness of TAF such as CosLU, ReLUN, SinLU, DeLU, HeLU, and ScaledSoftSign using deep learning models for regression tasks. We aim to explore the robustness of these functions under adversarial conditions. We present a case study on imitation learning using these activation functions in autonomous driving systems and evaluate their performance under various adversarial scenarios. Experiments conducted in Udacity Simulator demonstrate the robustness of each trainable activation function compared to AF. Among the TAF, we found that the deep learning models trained with ReLUN, CosLU, and SinLU respectively showed resilience against adversarial perturbations. This research contributes to the broader understanding of trainable activations in regression tasks for the vision-based autonomous vehicles domain.