Fisher Information Matrix and its Application of Bipolar Activation Function Based Multilayer Perceptrons With General Gaussian Input

For the widely used multilayer perceptrons (MLPs), there exist singularities in the parameter space where Fisher information matrix (FIM) degenerates on these subspaces. The singularities seriously influence the learning dynamics of MLPs which have attracted many researchers’ attentions. As FIM plays key role in investigating the singular learning dynamics of MLPs, it is very important to obtain the analytical form of FIM. In this paper, for the bipolar activation function based MLPs with general Gaussian input, by choosing bipolar error function as the activation function, the analytical form of FIM are obtained. Then the validity of obtained results are verified by taking two experiments.