WiFi Probes sniffing: an Artificial Intelligence based approach for MAC addresses de-randomization

To improve city services, local administrators need to have a deep understanding of how the citizens explore the city, use the relevant services, interact and move. This is a challenging task, which has triggered extensive research in the last decade, with major solutions that rely on analysing traces of network traffic generated by citizens WiFi devices. One major approach relies on catching the probe requests sent by devices during WiFi active scanning, which allows for counting the number of people in a given area and to analyse the permanence and return times. This approach has been a solid solution until some manufacturer introduced the MAC address randomization process to improve the user's privacy, even if in some circumstances this seems to deteriorate network performance as well as the user experience. In this work we present a novel techniques to tackle the limitations introduced by the randomization procedures and that allows for extracting data useful for smart cities development. The proposed algorithm extracts the most relevant information elements within probe requests and apply clustering algorithms (such as DBSCAN and OPTICS) to discover the exact number of devices which are generating probe requests. Experimental results showed encouraging results with an accuracy of 65.2% and 91.3% using the DBSCAN and the OPTICS algorithms, respectively.