Derivation of Mutual Information and Linear Minimum Mean-Square Error for Viterbi Decoding of Convolutional Codes Using the Innovations Method

We apply the innovations method to Viterbi decoding of convolutional codes. First, we calculate the covariance matrix of the innovation (i.e., the soft-decision input to the main decoder in a scarce-state-transition (SST) Viterbi decoder). Then a covariance matrix corresponding to that of the one-step prediction error in the Kalman filter is obtained. Furthermore, from that matrix, a covariance matrix corresponding to that of the filtering error in the Kalman filter is derived using the formula in the Kalman filter. This is justified from the fact that Viterbi decoding of convolutional codes has the structure of the Kalman filter. As a result, an upper bound on the average mutual information per branch for Viterbi decoding of convolutional codes is given using these covariance matrices. Also, the trace of the latter covariance matrix represents the (filtering) linear minimum mean-square error (LMMSE) per branch. We show that an upper bound on the average mutual information per branch is sandwiched between half the SNR times the filtering and one-step prediction LMMSEs per branch. In the case of quick-look-in (QLI) codes, from the covariance matrix of the soft-decision input to the main decoder, we can get a matrix. We show that the trace of this matrix has some connection with the linear smoothing error.