Application of multidimensional scaling to subjective evaluation of coded speech

We propose a new procedure for subjective evaluation of coded speech. This procedure has the potential of providing an anchorable measure of quality that contains more information than the single number provided by MOS testing. A stimulus space and the relationship between this space and speech quality are established with multidimensional scaling techniques in a large-scale listening test. In the field, the user uses a method described in this report to position a stimulus under evaluation in this previously-established space, and from this position the user draws conclusions about speech quality. The stimulus space is created by the multidimensional scaling program INDSCAL, which operates on subjective judgments of dissimilarities between samples of speech to create a stimulus space in which distances between stimuli correspond to perceptual dissimilarities. The stimulus space has the additional property that its dimensions correspond to perceptual attributes of the stimuli. In a pilot experiment, stimulus spaces for utterances produced by a male and a female talker were found to be highly correlated. MOS scores obtained in a separate study were found to be highly correlated with position in the stimulus space. We discuss both the physical and perceptual correlates of the three dimensions.