By Topic

Strong optimality of the normalized ML models as universal codes and information in data

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

1 Author(s)
Rissanen, J. ; IBM Res. Div., Almaden Res. Center, San Jose, CA, USA

We show that the normalized maximum-likelihood (NML) distribution as a universal code for a parametric class of models is closest to the negative logarithm of the maximized likelihood in the mean code length distance, where the mean is taken with respect to the worst case model inside or outside the parametric class. We strengthen this result by showing that, when the data generating models are restricted to be the most “benevolent” ones in that they incorporate all the constraints in the data and no more, the bound cannot be beaten in essence by any code except when the mean is taken with respect to the data generating models in a set of vanishing size. These results allow us to decompose the code of the data into two parts, the first having all the useful information in the data that can be extracted with the family in question and the rest which has none, and we obtain a measure for the (useful) information in data

Published in:

Information Theory, IEEE Transactions on  (Volume:47 ,  Issue: 5 )