Matrix Neural Network with Kernel Activation Function and Its Online Combined Learning

In the paper, we introduce a matrix neural network with a kernel activation function that is meant to process information in real-time mode. The distinctive feature of the system is during learning not only synaptic weights but also the architecture of the system tunes. At the same time, there are such criteria to be minimized as mean square error and empirical risk one. The proposed system combines the advantages of support vector machines, probabilistic neural networks, and radial basis function networks that allow tuning the proposed system under conditions of a small training dataset.