Noise compensation in a multi-modal verification system

In this paper we propose an adaptive multi-modal verification system comprised of a modified minimum cost Bayesian classifier (MCBC) and a method to find the reliability of the speech expert for various noisy conditions. The modified MCBC takes into account the reliability of each modality expert, allowing the de-emphasis of the contribution of opinions from the expert affected by noise. Reliability of the speech expert is found without directly modeling the noisy speech or finding the reliability a priori for various conditions of the speech signal. Experiments on the digit database show the total error to be reduced by 78% when compared to a non-adaptive system.