By Topic

A Metric Between Probability Distributions on Finite Sets of Different Cardinalities and Applications to Order Reduction

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

1 Author(s)
Vidyasagar, M. ; Erik Jonsson Sch. of Eng. & Comput. Sci., Univ. of Texas at Dallas, Richardson, TX, USA

In this paper we define a metric distance between probability distributions on two distinct finite sets of possibly different cardinalities. The metric is defined in terms of a joint distribution on the product of the two sets, which has the two given distributions as its marginals, and has minimum entropy. Computing the metric exactly turns out to be NP-hard. Therefore an efficient greedy algorithm is presented for finding an upper bound on the distance. We then study the problem of optimal order reduction in the metric defined here. It is shown that every optimal reduced-order approximation must be an aggregation of the original distribution, and that optimal reduced order approximation is equivalent to finding an aggregation with maximum entropy. This problem also turns out to be NP-hard, so again a greedy algorithm is constructed for finding a suboptimal reduced order approximation. Taken together, all the results presented here permit the approximation of an independent and identically distributed (i.i.d.) process over a set of large cardinality by another i.i.d. process over a set of smaller cardinality. In future work, attempts will be made to extend this work to Markov processes over finite sets.

Published in:

Automatic Control, IEEE Transactions on  (Volume:57 ,  Issue: 10 )