Learning from examples with Renyi's information criterion

This paper discusses a novel algorithm to train linear or nonlinear systems with information theoretic criteria (entropy or mutual information) directly from a training set. The method is based on Renyi's quadratic definition of entropy and a distance measure based on the Cauchy-Schwartz inequality.