Multi speaker diarization and speech recognition using ECAPA-TDNN and pyannote-v2.1

Speech recognition converts spoken words into text, which powers accessibility tools and virtual assistants. Speaker diarization is the process of identifying and labeling speakers in audio. In order to improve speech recognition accuracy and facilitate efficient clustering in speaker diarization, speaker embeddings are used to capture distinctive voice characteristics. The ECAPA-TDNN model extracts strong speaker embeddings using a channel- and context-dependent attention mechanism, Squeeze-Excitation, and residual blocks. The pyannote.audio toolkit provides a speaker diarization pipeline based on local speaker segmentation, neural speaker embedding, and global agglomerative clustering. This paper presents a novel approach for speaker diarization and speech recognition using two pre-trained models: the ECAPA-TDNN model and the pyannote.audio version 2.1 toolkit. The paper demonstrates the effectiveness of combining these two models on the AMI meeting dataset and various types of audio streams. The proposed system outperforms existing methods in speaker diarization in terms of performance and robustness.