A Bayesian analysis of entropy optimization for uncertainty modeling

Both the linearly-constrained minimum cross-entropy (LCMXE) method and the Bayesian parameter estimation procedure start with a prior distribution for an unknown quantity, then absorb the new information, and finally produce a posterior distribution. The authors establish an equivalence relationship between them by identifying certain statistical experiments embedded in LCMXE. To study the embedded experiments, they take a dual approach to understand the LCMXE method. The space of absolutely continuous distributions is considered as its domain. This LCMXE is a continuous convex programming problem involving an uncountable number of variables. An unconstrained dual program is derived with a finite number of variables by using the geometric programming approach with one simple inequality. This equivalence and the dual relationship between LCMXE and maximum likelihood estimation (MLE) provide a condition under which MLE and the Bayesian procedure produce the same inference.<>