A Joint Stochastic Gradient Algorithm and Its Application to System Identification with RBF Networks

Mean-square-error (MSE) and minimum-error-entropy entropy (MEE) criteria play significant roles in adaptive filtering and learning theory. Nevertheless, both the criteria have their respective shortcomings. In this paper, we propose a more general and effective stochastic gradient algorithm under joint criterion of MSE and MEE, and derive the approximate upper bound for the step size in the adaptive linear neuron (ADALINE) training. In particular, we demonstrate the superiority of this joint adaptive algorithm by applying it into system identification with radial basis function (RBF) networks.