Evaluation of quality of sound source separation algorithms: Human perception vs quantitative metrics

In this paper we look into the test methods to evaluate the quality of audio separation algorithms. Specifically we try to correlate the results of listening tests with state-of-the-art objective measures. To this end, the quality of the harmonic signals obtained with two harmonic-percussive separation algorithms was evaluated with BSS_Eval, PEASS and via listening tests. A correlation analysis was conducted and results show that for harmonic-percussive separation algorithms, neither BSS_Eval nor PEASS show strong correlation with the ratings obtained via listening tests and suggest that existing perceptual objective measures for quality assessment do not generalize well to different separation algorithms.