Where packet traces meet speech samples: an instrumental approach to perceptual QoS evaluation of VoIP

With the increasing deployment of real-time Internet services, evaluating the user perception of quality of service (QoS) has gained rapidly increasing importance. In the case of Voice over IP (VoIP), the standard approach of listening-only tests for subjectively assessing a limited number of speech samples, which are supposed to be representative for selected network conditions, does in no way reflect the huge variability of packet loss patterns that may originate from the underlying network. Performing tests by employing objective (instrumental) evaluation methods in a live testbed environment is usually extensive and does not deliver reproducible results, moreover the measurement granularity is bounded by the length of the test speech samples. In this paper, we propose a methodology that circumvents these limitations by employing arbitrary packet traces and successively matching the encoded speech sample with all possible trace fragments. This approach allows for continuous perceptual evaluation of VoIP traffic carried over various QoS-enabled transmission technologies. Results based on traces from testbed measurements reflecting different Web-like cross traffic situations for both the G.729 and iLBC codecs validate our approach and allow interesting insights into the dependence of perceived VoIP quality on underlying technological conditions.