The rate-distortion function for source coding with side information at the decoder

Let
\{(X_{k}, Y_{k}) \}^{ \infty}_{k=1}
be a sequence of independent drawings of a pair of dependent random variables
X, Y
. Let us say that
X
takes values in the finite set
\cal X
. It is desired to encode the sequence
\{X_{k}\}
in blocks of length n into a binary stream of rate
R
, which can in turn be decoded as a sequence
\{ \hat{X}_{k} \}
, where
\hat{X}_{k} \in \hat{ \cal X}
, the reproduction alphabet. The average distortion level is
(1/n) \sum^{n}_{k=1} E[D(X_{k},\hat{X}_{k})]
, where
D(x,\hat{x}) \geq 0, x \in {\cal X}, \hat{x} \in \hat{ \cal X}
, is a preassigned distortion measure. The special assumption made here is that the decoder has access to the side information
\{Y_{k}\}
. In this paper we determine the quantity
R \ast (d)
, defined as the infimum ofrates
R
such that (with
\varepsilon > 0
arbitrarily small and with suitably large
n
)communication is possible in the above setting at an average distortion level (as defined above) not exceeding
d + \varepsilon
. The main result is that
R \ast (d) = \inf [I(X;Z) - I(Y;Z)]
, where the infimum is with respect to all auxiliary random variables
Z
(which take values in a finite set
\cal Z
) that satisfy: i)
Y,Z
conditionally independent given
X
; ii) there exists a function
f: {\cal Y} \times {\cal Z} \rightarrow \hat{ \cal X}
, such that
E[D(X,f(Y,Z))] \leq d
. Let
R_{X | Y}(d)
be the rate-distortion function which results when the encoder as well as the decoder has access to the side information
\{ Y_{k} \}
. In nearly all cases it is shown that when
d > 0
then
R \ast(d) > R_{X|Y} (d)
, so that knowledge of the side information at the encoder permits transmission of the
\{X_{k}\}
at a given distortion level using a smaller transmission rate. This is in contrast to the situation treated by Slepian and Wolf [5] where, for arbitrarily accurate reproduction of
\{X_{k}\}
, i.e.,
d = \varepsilon
for any
\varepsilon >0
, knowledge of the side information at the encoder does not allow a reduction of the transmission rate.