Single-Channel Informed Spontaneous Speech Separation by NNMF on a Uniform Filterbank

The paper summarizes two algorithms to process spontaneous conversation, which contain two speech formats, a mixture, and dialogue of two speakers. Mixture format (called overlapped speech), when they are speaking simultaneously. Dialogue is when only one speaker is speaking and the other is silent. The first algorithm is overlapped speech detection. The algorithm processes the spontaneous conversation to produce two segregated mixture and dialog speech segments. The second algorithm supposes that the speaker diarization corpus processes the dialogue speech segments. The corpus outputs two sets of speech segments, first for the first speaker and second for the second speaker. Inputs of the second algorithm are two. The first is mixture speech (output of the first algorithm). Since the corpus output segments belong to the mixture format speakers, these segments assist the separation process (informed speech separation). The two algorithms are tested for 66 spontaneous conversations. The average EMISS of the overlapped speech algorithm is 0.4%. For the speech separation, the average SDR is 3.05 dB, SIR 9.8dB, and SAR 4.65 dB. On a uniform filter bank domain, the mathematical model of this paper’s informed speech separation algorithm exploited Non-Negative Matrix Factorization NNMF.