On Bayesian Exponentially Embedded Family for Model Order Selection

In this paper, we derive a Bayesian model order selection rule by using the exponentially embedded family (EEF) method, termed Bayesian EEF. It shows that the Bayesian EEF can use vague proper priors and improper noninformative priors to be objective in the elicitation of parameter priors. Moreover, the penalty term of the rule is shown to be the sum of half of the parameter dimension and the estimated mutual information between the parameter and observed data. This helps to reveal the EEF mechanism in selecting model orders and may provide new insights into the open problem of choosing an optimal penalty term for model order selection from information-theoretic viewpoints. The Bayesian EEF that uses a g-prior is derived for the linear model. The Bayesian EEF is extended for nonlinear models by using Jeffreys' prior. Interestingly, it coincides with the EEF rule derived by a frequentist strategy. This shows another relationship between the frequentist and Bayesian philosophies for model selection.