Performance Study of Large-Scale GPS Trajectory Query Method and Compression Method Based on Spark Platform

Buses and taxis in cities generate a large amount of GPS trajectory data every day. How to query and manage these large-scale trajectory data is a challenging problem. Based on the RDD component of Spark, a big data computing engine, we design and implement a distributed 3D R-Tree to accelerate trajectory query through global index and local index. The experimental results show that the three queries have the shortest latency under the condition of spatial-temporal grid partition, and the latency of trajectory point query, substring query and region query are 2.33s, 3.33s and 6.88s, respectively. In addition, the massive GPS trajectory data generated by these vehicles bring heavy storage cost pressure to the transportation big data platform. The compression processing of GPS trajectory data is imminent. In this paper, the big data computing engine Spark was used. A set of classical trajectory compression algorithms are implemented in parallel. Finally, the method is verified on a very large-scale real data set. Experimental results on compressed datasets show that the proposed method performs well. It only takes 438 seconds to compress 117.5GB GPS trajectory data of 31000 vehicles for one month on a 14-node Spark cluster.