Evaluating instrumental measures of speech quality using Bayesian model selection: Correlations can be misleading!

Choosing among competing models of collected data is crucial for all sciences. In the last decade there has been an increasing tendency to use Bayesian methods throughout many fields. When assessing the performance of instrumental measures of speech quality, classical measures such as correlation coefficients are still used. While these methods have their merits, they discard information about the data distribution, such as variability. They are useful as absolute measures of fit, but often not suitable for comparing different models. This paper uses Bayesian model selection, which does not suffer from these shortcomings, as it takes all information about the distribution of data into account and yields easily interpretable model probabilities. Two instrumental measures of speech quality are evaluated using data obtained in an absolute category rating (ACR) test. The results are compared and discussed. Bayesian methods prove superior for comparing instrumental measures, especially when the correlation of both measures is either poor or nearly identical. The proposed estimation procedure is highly recommended in selection phases for standardization bodies such as ITU-T, ETSI, 3GPP.