A Measurement Study on Mainline DHT and Magnet Link

Although Mainline DHT is not an official BitTorrent protocol so far, it's widely used in many BitTorrent clients such as Utorrent, Vuze, BitComet and Xunlei. Despite there are millions of Mainline DHT users, only little research focus on Mainline DHT. In this paper, we present two kinds of measurement results based on crawling Mainline DHT. Firstly, we develop an effective crawler and crawl Mainline DHT over 10 days. We collected 10,149,649 magnet links and received over 264,846,482 get_peers requests from 57,065,499 unique peers. Based on the measurement results, we found that: (1) Russian and China play dominant roles in Mainline DHT which contribute 35% peers in it, and there are 5% Internet users use Mainline in Europe, (2) UTorrent is the most popular clients so far, half of peers use it. (3) We found that most clients use randomly chosen port numbers, only 8.9% use constant port. (4) Zona is a popular Russia peer to peer client, nine in ten top popular magnet links are unique in Zone, and only one is BitTorrent magnet link. (5) The peers' size of torrent almost follows a Zipf distribution. 11% torrents take up 95% peers of the total. 46% torrents only get one peer during our experiment. Secondly, we study two ways to convert magnet link to torrent file: Converting by five popular torrent storage services and converting by Extension protocol. We compare the performance of them in terms of converting rate, converting time, advantages and disadvantages and discover that: (1) The converting rate of torrent storage services and Extension protocol are 46.9% and 22.7%. (2) The average successful converting time of Extension protocol is 75s, and if a magnet spends more than 150s to convert but still couldn't get good peers, it tends to convert unsuccessfully at present. The best scheme is stopping for a while, and waiting good peers on line. (3) NAT is the main reason caused the converting or downloading failure, there are 32.5% peers stay in NAT or firewall. The best solution of NAT user is the popularized of IPv6 or Tunneling. Understanding DHT in depth can improve the performance of current DHT system.