Two-Branch Network with Selective Kernel Convolution for Time-Domain Speech Enhancement

Speech enhancement methods based on deep neural networks usually take noisy speech as input and clean speech as output. However, there is still a correlation between predicted speech and removed noise, which is not fully utilized. And the common convolutional codec has mostly the same receptive field at each layer, which may be difficult to represent the multi-scale features of speech. In this paper, we propose a two-branch interactive time-domain speech enhancement model combining selective kernel units (TBSK-Net). We introduce Selective Kernel Convolution into each layer of the encoder to adaptively encode the information of noisy speech at multiple scales. The stacked selective kernel units are designed to construct two branches to learn speech and noise features where insert exchange information modules. The results show that the proposed model has obvious advantages in dealing with unknown noise than the baseline and other references. Introducing the Selective Kernel mechanism can effectively improve the enhancement performance. Ablation experiments are also performed to demonstrate the effectiveness of different components in the structure.