Peaking Beyond the Best Route: An Extensive Dataset for Looking Glasses

The Internet relies on the Border Gateway Protocol (BGP)—a policy-based routing protocol—to establish routes. Each Autonomous System (AS) uses BGP to realize its routing policies based on the business agreements that they have with its neighboring ASes. ASes typically do not share their business agreements publicly. Another drawback of BGP is that ASes typically do not see the effects of their routing policies as this information is only visible within other ASes. Yet, for an AS to check their BGP configuration they need to see the effect. Thus, ASes collaborate and operate Looking Glasses (LGs) which are publicly accessible. LGs are websites that allow the users to query one or several routers within the ASes for routing information. This information may be restricted to BGP routes (routing prefix plus AS path) only or other BGP attributes as well, e.g., local preference, MED, and BGP communities. Such LG data is required by many BGP topology inference methods either as input or for validation. The dataset that this paper focuses on collects BGP attributes from more than 149 LGs in 154 ASes from 931 routers via scraping the LGs. Hereby, the difficulties relate to the non-uniformity of the LGs—most interfaces differ, the fluctuating accessibility of the LGs, as well as the different output formats. To overcome this we combined manual configuration with an automated scraping process followed by careful post-processing and manual checks. Our current dataset covers one and a half months of continuous data collection every 4 hours. In this paper, we describe both our collection pipeline as well as initial analysis results which focus on route diversity for ASes with multiple LGs. We find that up to 43% of these ASes use diverse routes to at least one of their peers. Routes can differ in local preference 40%, AS paths 37%, or BGP communities 41%. While the former is expected the latter is surprising.