Consistent Order Estimation and Minimal Penalties

Consider an i.i.d. sequence of random variables whose distribution
f*
lies in one of the nested families of models
Mq
,
q
≥ 1. The smallest index
q*
such that
Mq*
contains
f*
is called the model order. The aim of this paper is to explore the consistency properties of penalized likelihood model order estimators such as Bayesian information criterion. We show in a general setting that the minimal strongly consistent penalty is of order η(
q
)loglog
n
, where η(
q
) is a dimensional quantity. In contrast to previous work, an a priori upper bound on the model order is not assumed. The results rely on a sharp characterization of the pathwise fluctuations of the generalized likelihood ratio statistic under entropy assumptions on the model classes. Our results are applied to the geometrically complex problem of location mixture order estimation, which is widely used but poorly understood.