Landscape of Monitoring and Visualization of Technologies in DevOps for Classification and Prediction

As Isaac Asimov said “If knowledge can create problems then it is not through ignorance that this, we solve them” which is the main motto for this project. This project wanted to create the central platform where the knowledge could be shared and keeping the saying in mind this project used logs for the research. Despite the logs are created for every instance and applications, studying and using them is rarely done because of their illegibility for people who do not understand that specific application's log profile leading to the use of logs either entirely neglected or rarely used for specific cases. This project is separated into 3 sections : 1) A centralised platform where the logs are stored and manipulated and standardised method for reading them is created using Elasticsearch. 2) Visualisation with the help of kibana is done for interpreting data. 3) Logs are ingested into hadoop eco system from logstash. From logstatsh which increase the retention time period of logs and hense machine learning algorithms for detecting anomaly in the logs can be done easily. This project tries to create a fully encompasing and logterm architectural solution for application managemnet of environments. And a broad perspective solution for visualizing flow of data on a mangeral level.