Measuring Dependencies of Order Statistics: An Information Theoretic Perspective

This work considers a random sample X
1
,X
2
,…,X
n
drawn independently and identically distributed from some known parent distribution P
X
with X
(1)
≤ X
(2)
≤ … ≤ X
(n)
being the order statistics of the sample. Under the assumption of an invertible cumulative distribution function associated with the parent distribution P
X
, a distribution-free property is established showing that the f-divergence between the joint distribution of order statistics and the product distribution of order statistics does not depend on P
X
. Moreover, it is shown that the mutual information between two subsets of order statistics also satisfies a distribution-free property; that is, it does not depend on P
X
. Furthermore, the decoupling rates between X
(r)
and X
(m)
(i.e., rates at which the mutual information approaches zero) are characterized for various choices of (r,m). The work also considers discrete distributions, which do not satisfy the previously-stated invertibility assumption, and it is shown that no such distribution-free property holds: the mutual information between order statistics does depend on the parent distribution P
X
. Upper bounds on the decoupling rates in the discrete setting are also established.