Information-distilling quantizers

Let X and Y be dependent random variables. We consider the problem of designing a scalar quantizer for Y to maximize the mutual information between its output and X, and study fundamental properties and bounds for this form of quantization. Our main focus is the regime of low I(X; Y), where we show that for a binary X, there always exists an M-level quantizer attaining mutual information of Ω(-M · I(X;Y)/log(I(X;Y)) and that there exist pairs of X, Y for which the mutual information attained by any M-level quantizer is O(-M · I (X;Y)/ log (I(X;Y))).