A Fused Speech Enhancement Framework for Robust Speaker Verification

Robust speaker verification (RSV) under noisy conditions is still a challenging task. Recently, some task-specific speech enhancement (SE) approaches are proposed and achieve excellent performance on RSV. However, all these works adopt only one kind of SE network and thus can not remove noise from different aspects, limiting the performance of the RSV task. In this letter, we propose a fused SE framework (FSEF) for RSV, which integrates both T-F masking-based and feature mapping-based SE networks to collect complementary information and improve the robustness against noise. Two FESF-RSV systems are constructed based on two kinds of fusion methods: score fusion and feature fusion. In addition, we present a Multi-Scale Attentive Context Aggregation Network (MSACAN) as the backbone structure in the FSEF. The MSACAN can not only extract and fuse multi-scale features adaptively but also enhance speaker characteristics against noise and interfering speakers. Experiments conducted on the noise-simulated VoxCeleb1 dataset demonstrate both the FSEF and the MSACAN can improve the performance of RSV compared to previous approaches.