The effect of focused attention on audio-visual quality perception with applications in multi-modal codec design

Understanding the perceptual processes involved in cross-modal interaction is necessary for multi-modal codec design and automatic multimedia quality prediction systems. By using a multi-modal codec, the auditory and visual streams are considered to be a single entity, thus resulting in an improved performance for a given system bandwidth. This paper describes an experiment which examines the effect on focus of attention on quality prediction. It shows that, if attending a certain modality (auditory or visual), one's ability to detect errors in the other modality is impaired compared to the case when one is attending equally to both modalities.