A Multi-Objective Perceptual Aware Loss Function for End-To-End Target Speaker Separation

Recent years the most commonly used loss function in end-to-end monaural target speaker extraction, especially for time domain deep neural networks, is Scale-Invariant Signal-to-Distortion Ratio (SISDR). This objective function shows powerful optimization efficiency in speaker extraction. However, the evaluation result of extraction models trained by SISDR shows deficiency in speech quality and intelligibility especially in perceptual and spectral scales. Hence we propose a multi-objective loss function for end-to-end target speaker network training, consisting of objectives from time domain, spectral domain and the human perceptual field, which is able to make improvement in terms of speech quality and intelligibility in various measures, and at the same time ensure that its performance in SISDR will not drop. We select SpEx+ as our baseline network, and experimental results on the WSJ0-2mix-extr dataset show that our proposed loss function achieves superiority in multiple quality measures compared with the baseline system.