Deep Clustering with Gated Convolutional Networks

Deep clustering is a recently introduced deep learning-based method for speech separation. The idea is to model and train the mapping from each time-frequency (TF) region of a spectrogram to an embedding space so that the embedding features of the TF regions dominated by the same source are forced to get close to each other and those dominated by different sources are forced to get separated from each other. This allows us to construct binary masks by applying a regular clustering algorithm to the mapped embedding vectors of a test mixture signal. The original deep clustering uses a bidirectional long short-term memory (BLSTM) recurrent neural network (RNN) to model the embedding process. Although RNN-based architectures are indeed a natural choice for modeling long-term dependencies of time series data, recent work has shown that convolutional networks (CNNs) with gating mechanisms also have an excellent potential for capturing long-term structures. In addition, they are less prone to overfitting and are suitable for parallel computations. Motivated by these facts, this paper proposes adopting CNN-based architectures for deep clustering. Specifically, we use a gated CNN architecture, which was introduced to model word sequences for language modeling and was shown to outperform LSTM language models trained in a similar setting. We tested various CNN architectures on a monaural source separation task. The results revealed that the proposed architectures achieved better performance than the BLSTM-based architecture under the same training condition and comparable performance even with a smaller amount of training data.