A Real-Time Sound Source Localization System for Robotic Vacuum Cleaners With a Microphone Array

With the progress of artificial intelligence (AI) technology, home appliances are becoming more advanced to enhance our quality of life. Many smart devices support speech interfaces, including voice commands and user location tracking. However, robotic vacuum cleaners generate strong ego-noise that distorts microphone signals, making it difficult to estimate the user’s location. To solve this problem, we propose a real-time sound source localization (SSL) system for a robotic vacuum cleaner equipped with a microphone array. We design a system that consists of speech enhancement, voice activity detection (VAD), and SSL modules. The speech enhancement module includes TRU-Net-Light, which has lower computation and similar speech enhancement performance to tiny recurrent U-net (TRU-Net). The TRU-Net-Light reduces the number of channels to reduce the model size and applies a frequency-axis multihead self-attention to boost representational capacity. The finite state machine-based VAD is designed to detect voice active periods using the output of a speech enhancement module. Furthermore, we present a mask-weighted difference correlation vector and the singular value decomposition (SVD) with smoother coherence transform (DSVD-SCOT) that achieves robust localization performance in severely noisy environments. In the experimented robotic vacuum cleaner, the localization accuracy of the SSL system was 97.9% and 84.0% for signal-to-noise ratios (SNRs) of −3 and −8 dB, respectively. The proposed system was run in real-time, with a real-time factor (RTF) of 0.378, on a single Kryo 585 Silver core in the RB5 platform. A demo of the proposed system is available at https://youtu.be/3d3Cr-cs9aY.