Cuckoo Prefix: A Hash Set for Compressed IP Blocklists

IP blocking has become a vital task for all network attached devices. Every device from Internet of Things, to routers, to application servers requires the ability to filter certain IP addresses from delivering malicious information. Blocking IPs requires storing and checking lists of tens to hundreds of millions of IP addresses. Cuckoo hash sets provide strong performance by offering relatively low numbers of memory accesses per lookup. This makes them optimal for time sensitive applications like networking. Using cuckoo++ hash tables as a baseline, we propose a new data structure known as cuckoo prefix for the purpose of blocking IPs quickly with relatively little space. Leveraging IP subnets allows us to achieve similar throughput rates as implementations such as cuckoo++ with 8 times less memory usage. In addition, in this paper we offer a comparison of throughput and memory usage of several modern hash set and hash table implementations. In particular, we examine linear probing, robin hood hashing, bit sets (including EBVBL), and cuckoo hashing implementations to determine which provides the best throughput at the lowest memory cost.