Distribution Construction Approaches for Constrained Entropy Vectors and Converse Results

Properties of the Shannon entropy vectors are instrumental for analyzing the limits of reliable communication for various communication models. However, the set of all Shannon entropy vectors is not yet known even for three random variables. Often, there are constraints on the entropy vectors of random variables due to the underlying communication model, e.g., functional dependence, independence, and Markov chain constraints. Moreover, the entropy vectors may be constrained to belong to a specific class for a given model, e.g., belonging to the class of quasi-uniform entropy vectors due to code design constraints. This paper presents distribution construction approaches and converse results for such constrained entropy vectors. We present three approaches for constructing distributions such that corresponding entropy vectors satisfy given constraints: (i) by exploiting functional dependence constraints, (ii) by quasiuniform constructions, and (iii) by using independent random vectors. In particular, these approaches are utilized to show the looseness of the known inner bounds for constrained entropy vectors for three random variables. We also present converse results for entropy vectors constrained by basic equalities and quasi-uniformity.