Measuring Human Perception of Audiovisual Errors using EEG

Audiovisual synchronization is an essential indicator of video quality. The degradation in the quality of experience caused by such synchronization errors is often measured using the mean opinion score (MOS). However, this method is susceptible to emotion and bias. Electroencephalography (EEG), as an objective tool, overcomes the drawbacks of subjective testing for evaluating human perception. In this work, we measure the human perception of audio and video synchronization based on the EEG approach through a series of experiments. A scoring model was developed for audio and video under multilevel synchronization errors by extracting the power spectral density (PSD) of EEG signals as features. The use of EEG signals for perceptual ability assessment of audiovisual distortion provides a potential neurally informed approach that is more objective than the high-level cognitive activity in subjective data.