Directed Information Subject to a Fidelity. Applications to Conditionally Gaussian Processes

This paper is concerned with the minimization of directed information over conditional distributions that satisfy a fidelity of reconstructing a conditionally Gaussian random process by another process, causally. This information theoretic extremum problem is directly linked, via bounds to the optimal performance theoretically attainable by non-causal, causal and zero-delay codes of data compression. The application example includes the characterization of causal rate distortion function for conditionally Gaussian random processes subject to a mean-square error fidelity.