Multispeaker Audio-Visual Fusion for Intelligent Interaction Systems

Speaker identification and semantic scene segmentation in complex environments remain significant challenges in the fields of intelligent speech interaction and video understanding. Traditional approaches often rely on a single modality (either audio or video), leading to low recognition accuracy, weak semantic association, and poor real-time performance in noisy or occluded conditions. This paper proposes a multimodal intelligent interaction system that integrates audio-based speaker embeddings and visual lip movement features. A dynamic scene segmentation mechanism is designed to associate speaker identity with visual regions, while an adaptive optimization strategy enhances the system's robustness to varying environments. Experimental results demonstrate that the proposed system maintains high speaker recognition accuracy and segmentation precision even in noisy, multi-speaker scenarios, indicating strong potential for real-world applications.