Winner Takes It All: An Efficient Overlap-Aware Hybrid Online Diarization with Partial Backtracking Mechanism

In this paper, we propose technical enhancements to a recently introduced multi-stage overlap-aware method for low-latency online speaker diarization that combines incremental clustering and local diarization. In particular, we introduce a novel transformer-based segmentation model, which improves both the inference efficiency and modeling capability for over-lapped speech. Furthermore, we propose an optimized incremental clustering approach with a partial backtracking mechanism to mitigate the subsequent impact of early temporary errors in streaming systems, while preserving the irreversibility of the outputs. Our system evaluation on AMI, VOX and DIHARD III datasets demonstrates improvements compared to the original method, with a relative decrease in Diarization Error Rate of 13.2%, 15.5%, and 9.7% respectively, along with over 30% reduction in inference time. Finally, we demonstrate a low-cost transition from online to enhanced offline results by incorporating a two-pass clustering method in specific practical scenarios.