Tracking Language Mobility in the Twitter Landscape

The unprecedented data explosion has drastically changed the data science landscape. At the same time, Big Data analytics have reshaped the design and implementation of the applications that analyse the data. In this paper, we explore the use of Big Data tools for extracting value from Twitter data. We acquire a large set of Twitter data (10TB in size) and process it by relying on Spark DataFrame. The purpose of our analytics pipeline is to study the mobility of languages as captured by the Twitter signal. We study the evolution of languages from both a temporal and a spatial perspective, by applying density-based clustering and Self-Organising Maps techniques. The analysis enabled the detection of tourism trends and real-world events, as perceived through the Twitter lens.