Big Data Framework for Monitoring Real-Time Vehicular Traffic Flow

The relatively high rate of traffic accidents in Iraq shows the necessity of working on the driver's actions monitoring through the use of vehicle flow data to improve the road safety. Based on this situation, many tools and technologies such as sensors, cameras, and data management can be utilized to monitor traffic conditions and provide real-time information to drivers and transportation authorities. The primary challenges are collecting, processing, analyzing, and visualizing the huge volume of data produced by vehicles and devices. To address these challenges, we proposed and implemented a big data framework for monitoring the data flows generated by vehicles in the city environment. Among the various data generated by vehicles, our framework monitors the latitude and longitude values of the global positioning system (GPS) and speed. The framework's architecture is scalable and fault-tolerant which makes it suitable for handling large-scale data flows generated by many connected vehicles. The results show that it allows for increased throughput, high availability, and fault tolerance and provides full-text search. This framework has been implemented using several big data platforms and tools such as Apache Kafka and Elasticsearch. In addition, the framework's services have been packaged in the container-based virtualization environment to support the reusability and portability of the framework.