Optimal Confidence Sets for the Multinomial Parameter

Construction of tight confidence sets and intervals is central to statistical inference and decision making. This paper develops new theory showing minimum average volume confidence sets for categorical data. More precisely, consider an empirical distribution
$\widehat{p}$
generated from
$n$
iid realizations of a random variable that takes one of
$k$
possible values according to an unknown distribution
$p$
. This is analogous to a single draw from a multinomial distribution. A confidence set is a subset of the probability simplex that depends on
$\widehat{p}$
and contains the unknown
$p$
with a specified confidence. This paper shows how one can construct minimum average volume confidence sets. The optimality of the sets translates to improved sample complexity for adaptive machine learning algorithms that rely on confidence sets, regions and intervals.