Deep Transductive Nonnegative Matrix Factorization for Speech Separation

Non-negative matrix factorization (NMF) has attracted great attentions in speech separation as it can preserve the non-negativity property of the magnitude spectrogram of speech signal. However, NMF sometimes performs poorly because it cannot extract the non-linear features in speech. In this paper, we propose a deep transductive NMF model (DTNMF) which incorporates a multi-layer structure into NMF and learns a shared dictionary on source signal of each speaker and the mixture signal to be separated. Since the multi-layer structure enables DTNMF to learn more precise presentation of source signal with the non-linear features extracted, DTNMF significantly enhances the performance of speech separation. Experimental results on Non-negative matrix factorization (NMF) has attracted great attentions in speech separation as it can preserve the non-negativity property of the magnitude spectrogram of speech signal. However, NMF sometimes performs poorly because it cannot extract the non-linear features in speech. In this paper, we propose a deep transductive NMF model (DTNMF) which incorporates a multi-layer structure into NMF and learns a shared dictionary on source signal of each speaker and the mixture signal to be separated. Since the multi-layer structure enables DTNMF to learn more precise presentation of source signal with the non-linear features extracted, DTNMF significantly enhances the performance of speech separation. Experimental results on the popular LibriSpeech dataset show that DTNMF outperforms the representative NMF models for separating the mixture of single-channel speech signals.