On Compressive Sensing in Coding Problems: A Rigorous Approach

We take an information theoretic perspective on a classical sparse-sampling noisy linear model and present an analytical expression for the mutual information, which plays a central role in a variety of communications/signal processing problems. Such an expression was addressed previously by bounds, by simulations, and by the (nonrigorous) replica method. The expression of the mutual information is based on techniques used, addressing the minimum mean square error analysis. Using these expressions, we study specifically a variety of sparse linear communication models, which include coding in various settings, accounting also for multiple access channels, broadcast channels, and different wiretap problems. For those, we provide single-letter expressions and derive achievable rates, capturing the communications/signal processing features of these contemporary models.