Multiclass Classification Using Arctangent Activation Function and Its Variations

Deep learning have been applied in life changing areas. Wide range of areas shows how successful deep learning is. There are several reasons why deep neural networks works well. The most importantly, activation functions since they are very powerful for solving non-linear problems. For that reason, it became a focus point for artificial intelligence researchers who want to improve the performance of neural networks. Special irrational numbers like pi and the golden ratio are shown up themselves in many areas such as art, geometry, architecture, etc. The wide range of occurrences of the pi and golden ratio inspire us to apply them to activation functions. This document is written for comprehensive explanation and comparison of activation functions which mainly focuses on arc tangent and its' variations defined in the paper. Experimental results are showed that variations which are obtained using irrational numbers pi and golden ratio, and also self-arctan, give promising results. Especially arctan with golden ratio have given better results. Multi-class classification problem was taken consider in the paper.