Evaluating the Impact of Design Decisions on Passive DNS-Based Domain Rankings

‘Top sites' rankings of the most popular domains are a core resource for the large-scale measurements that are crucial in Web and Internet research. Recent rankings evolved towards using passive DNS traffic data, but this data's suitability for measuring website popularity is poorly understood. In this paper, we holistically evaluate how design decisions influence the composition and desired properties of passive DNS-based domain rankings. We isolate the effects of these decisions by generating a ranking from the ground up using aggregated “post-recursor” passive DNS data. We evaluate the impact of corrections for resolver caching and CDNs, and confirm that measures such as service classification, bucketing, or long-term aggregation produce more reliable rankings. Our goal is to give transparent insight into the process of using passive DNS data for domain rankings, as a framework for the research community to understand how to develop future rankings that address their needs.