Estimating the entropy of a signal with applications

We present a new estimator of the entropy of continuous signals. We model the unknown probability density of data in the form of an AR spectrum density and use regularized long-AR models to identify the AR parameters. We then derive both an analytical expression and a practical procedure for estimating the entropy from sample data. We indicate how to incorporate recursive and adaptive features in the procedure. We evaluate and compare the new estimator with other estimators based on histograms, kernel density models, and order statistics. Finally, we give several examples of applications. An adaptive version of our entropy estimator is applied to detection of law changes, blind deconvolution, and source separation.