JPR: Exploring Joint Partitioning and Replication for Traffic Minimization in Online Social Networks

A scalable storage system becomes more important today for online social networks (OSNs) as the volume of user data increases rapidly. Key-value store uses consistent hashing to save data in a distributed manner. As a defacto standard, it has been widely used in production environments of many OSNs. However, the random nature of hashing always leads to high inter-server traffic. Recently, partitioning and replication are respectively proposed in many existing works where the former aims to minimize the inter-server read traffic and the latter aims to optimize the inter-server write traffic. Nevertheless, the separated manners of optimization cannot efficiently reduce the traffic. Because the inter-server read traffic is changed during replication. In this paper, we suggest that performing partitioning and replication simultaneously could provide probability to further optimize traffic. Then we formulate the problem as a revised graph partitioning with overlaps, since overlaps partitioning naturally corresponds to replication. To solve the problem, we propose a Joint Partitioning and Replication (JPR) scheme. Through extensive experiments with a real world Facebook trace, we evaluate that JPR significantly reduces inter-server traffic with slightly sacrificing storage cost compared to hashing, and preserves a good load balancing across servers as well.