A Novel Method for Efficient Approximate Join Queries on Big Hyperlocal Geo Data

Due to the massive growth of Internet of Things (IoT) devices, as well as urban sensing technologies, there has been an unprecedented volume of geospatial data, with millions of data points generated per minute. This growth introduced significant challenges for spatial join operations. In particular, point-in-polygon (PIP) joins are computationally intensive and scale poorly with large datasets. Traditional approaches often rely on pairwise tests or minimum bounding rectangle (MBR)-based filter-and-refine strategies. Both of which suffer from high computational costs and the possibility of false positives. In this study, we introduce GeoApproxJoin, a novel framework that utilizes two spatial indexing techniques, Geohash & H3, to integrate stratified-like sampling with a classical filter-and-refine strategy to perform fast, approximate PIP joins. The two spatial indexing techniques are compared over a large-scale hyperlocal geospatial dataset. An empirical assessment of the use of our approach on live data via PM2.5 air-quality data sub-joined with neighborhood boundaries in New York City is conducted, and the results suggest that H3 performed better in terms of run-time performance, completing an approximate 23% faster spatial join of 10.3 seconds compared to Geohash's run-time of 13.8 seconds. However, Geohash provides slightly higher accuracy with 8% lower Root Mean Squared Error (RMSE) of .79 compared to H3 of .86.