Wi-Fi Meets Sound: A Deep Fusion Leap Beyond Single-Modality for Device Localization

Wi-Fi-based indoor localization methods have gained significant attention due to the widespread availability of wireless infrastructure and the ability to collect high-resolution data, i.e., channel state information (CSI), at once. However, the Wi-Fi-based localization often suffers from accuracy limitations in complex indoor environments. Some studies have proposed multimodal localization methods using Wi-Fi and sound signals to address this issue. Although multimodal localization enables high accuracy and robust localization in indoor environments, existing systems face challenges due to infrastructure complexity and limited multimodal fusion capabilities. This paper proposes a novel single-anchor multimodal localization system that utilizes neural fusion techniques to address the challenges. More specifically, the proposed method extracts key features from the temporal and spatial domains using the temporal and element encoders, respectively, and fuses them to enable effective modeling of complex relationships between multimodal sensor data and the user’s location. Experimental results using the commercial Wi-Fi access point and microphone array demonstrate that the proposed system achieves a median localization error of less than 30 cm even in non-line-of-sight (NLoS) indoor environments.