Double-Layer Soft Data Fusion for Indoor Robot WiFi-Visual Localization

This article presents a novel WiFi-Visual data fusion method for indoor robot (TIAGO++) localization. Long-term follow-up experiments show that this method can use ten WiFi samples and four low-resolution images ( ${58}\times {58}$ in pixels) to localize an indoor robot with an average error distance of about 1.32 m three months (or 1.7 m seven months) after training data collection. Instead of neural network design, this article focuses on soft data fusion to prevent unbounded errors in visual localization. The proposed soft data fusion includes first-layer WiFi-Visual feature fusion and second-layer decision vector fusion. First, motivated by the excellent capability of the neural network in image processing and recognition, temporal-spatial features are extracted from WiFi data and represented in image form. Second, the image-form WiFi features and the visual features taken by the robot camera are combined together, and jointly exploited by a classification neural network to produce a likelihood vector for WiFi-Visual localization. This is called first-layer fusion. Similarly, these two types of features can be separately exploited by neural networks to produce another two independent likelihood vectors. Third, the three likelihood vectors are fused by the Hadamard product to produce a final likelihood vector. This is called second-layer fusion. The proposed soft data fusion does not apply any threshold or prioritize any data source over the other in the fusion process. It never excludes low-probability candidate positions, which can avoid information loss due to a hard decision. The demo video is available at: https://youtu.be/__AlHGhDNSI and the code will be open to the public after the publication of this work.