Social media monitoring using ELK Stack

Analyzing data in real-time and processing them online has been an important aspect when coming into social media monitoring. This helps to understand the current trends, political scenarios all over the world and most importantly for the purpose of data collection. When viewing from an organizational perspective, offline data analysis can seem to be more productive when data collection happens to be a onetime process. But the current researches require real-time data analysis but still faces various challenges when continuous data fetching disrupts and affects the overall efficiency of the process. This paper demonstrates a solution to effectively address the challenges of real-time analysis using an ELK (Elastic search, Log stash and Kibana) stack which can be used as a SIEM for small scale organization (with various limitations) where logs are taken from Social media platforms (Twitter), analyzed, processed and conclusions are derived from them.