Theory and design of dyadic subband tree structures with embedded PDF-optimized quantizers

We develop a methodology for the analysis of signal quantization effects in critically sampled dyadic subband tree structures using a nonlinear gain-plus-additive-noise model for the probability density function (PDF)-optimized quantizer. We constrain the two-band nonquantized and uncompensated structure at each level to be perfect reconstruction (PR). We develop an equivalent uniform filter bank followed by its polyphase structure described by primitive submatrices and compute a rigorously correct mean squared error (MSE) in the frequency domain using cyclostationary concepts in terms of: (1) the allocated quantizer bits; (2) the filter coefficients; (3) an embedded compensation parameter vector. This MSE is then minimized over all three items above. Our optimization method is applied to the specific case of a four-channel dyadic tree with average bit rate constraint. This tree is represented by an eight-channel polyphase equivalent whose interchannel signals are correlated. We show how to represent rigorously the correlation of random noise between channels due to the embedded quantizers. Our design of paraunitary and biorthogonal structures with identical and nonidentical stages is performed, compared, and validated by computer simulation under the assumption of uncorrelated cross band noise. The nonidentical stage biorthogonal filter bank turned out to have the best performance in MSE sense, but the most robust structure is the nonidentical stage paraunitary filter bank.