Recursive Renyi's entropy estimator for adaptive filtering

Recently we have proposed a recursive estimator for Renyi's quadratic entropy. This estimator can accurately converge the results for stationary signals or track the changing entropy of nonstationary signals. We demonstrate the application of the recursive entropy estimator to supervised and unsupervised training of linear and nonlinear adaptive systems. The simulations suggest a smooth and fast convergence to the optimal solution with a reduced complexity in the algorithm as compared to the batch training approach using the same entropy-based criteria. The presented approach also allows on-line information theoretic adaptation of model parameters.