Approximation by fully complex MLP using elementary transcendental activation functions

Recently, we presented 'fully' complex multi-layer perceptrons (MLPs) using a subset of complex elementary transcendental functions as the nonlinear activation functions. These functions jointly process the inphase (I) and quadrature (Q) components of data, while taking full advantage of well-defined gradients in the error back-propagation. The characteristics of these elementary transcendental functions are categorized and their common almost everywhere (a.e.) bounded and analytic properties are investigated. More importantly, it is proved that fully complex MLPs are a.e. convergent and therefore are capable of universally approximating any nonlinear complex mapping to an arbitrary accuracy. Numerical examples demonstrate the benefit of isolated essential singularity included in a subgroup of elementary transcendental functions in achieving arbitrarily close approximation to the desired mapping.