Adaptation of Multiple Sound Source Localization Neural Networks with Weak Supervision and Domain-adversarial Training

Despite the recent success of deep neural network-based approaches in sound source localization, these approaches suffer the limitations that the required annotation process is costly, and the mismatch between the training and test conditions undermines the performance. This paper addresses the question of how models trained with simulation can be exploited for multiple sound source localization in real scenarios by domain adaptation. In particular, two domain adaptation methods are investigated: weak supervision and domain-adversarial training. Our experiments show that the weak supervision with the knowledge of the number of sources can significantly improve the performance of an unadapted model. However, the domain-adversarial training does not yield significant improvement for this particular problem.

1. INTRODUCTION

Recent studies have shown that deep neural networks (DNNs) have become the state-of-the-art for sound source localization (SSL) and directions of arrival (DOA) estimation [1]–[8]. Although these DNN-based approaches outperform the classical signal processing-based techniques [9]–[11] under certain conditions, they suffer two major drawbacks.