Temporal Variability Analysis of Domain Names using Graph Techniques on Big Data

The internet domain name system (DNS) is essential to the internet infrastructure, facilitating communication between devices and applications. Over the time, changes in the web graph structure, which represents the interconnections between web pages, can provide valuable insights into the evolution of the internet ecosystem. By analyzing the dynamics of web graph connectivity, we can gain a deeper understanding of how the relationships between domains have evolved over the time. By analyzing the relationships between domain names and their connections in the web graph, we can track changes over the time, identify patterns and clusters, and better understand how the internet landscape is evolving. This research paper focuses on leveraging big data techniques to perform temporal variability analysis of domain names using graph techniques on website data. The main objective of this research is to identify clusters of related web pages using only URL information, and links between them are analyzed using web graphs built from the website database. The research focuses on three main aspects: extracting and cleaning URL data from the website data, analyzing the relationships between domain names to identify how they are related to each other, and finally, visualizing these relationships using graph visualization techniques. We first extract domain names from the dataset and construct a graph with domains as nodes and links representing the relationships between them. Our approach combines graph algorithms such as ensemble page-rank and label propagation to identify significant changes in the domain name system over the time.