Keep it simple stupid — On the effect of lower-order terms in BIC-like criteria

We study BIC-like model selection criteria. In particular, we approximate the lower-order terms, which typically include the constant log ∫√det I(θ) dθ, where I(θ) is the Fisher information at parameter value θ. We observe that the constant can sometimes be a huge negative number that dominates the other terms in the criterion for moderate sample sizes. At least in the case of Markov sources, including the lower-order terms in the criteria dramatically degrades model selection accuracy. A take-home lesson is to keep it simple.