Weakly DOA Guided Speaker Separation with Random Look Directions and Iteratively Refined Target and Interference Priors

Given a mixture of multiple speech signals, a neural network can extract the talkers individually and sequentially, which may improve the output quality compared to a simultaneous separation of all speakers. To still suppress interfering speech effectively, the residual mixture of the remaining unseparated speakers can be included in the input of the next step. We build upon this approach with a twofold contribution. First, we propose to refine the already extracted speech signals in further optional iterations. This is accomplished by exploiting that the outputs of previous steps provide prior information on interference and target. Experiments indicate a gradual improvement until convergence after about 2 iterations per speaker. Secondly, look directions are defined to control in which order speakers are extracted, thereby resolving the related permutation ambiguity. Whereas supplying the true speaker locations delivers the best results, even a weak guidance with random directions reduces interference leakage significantly.